psw/norgbackup
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

updated docs

Browse source
This commit is contained in:
Paul Wilde 2024-08-24 20:20:43 +01:00
parent bf97f20bb2
commit 2d2988b9aa
13 changed files with 359 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/content/contributing/_index.md Normal file
View file

@ -0,0 +1,4 @@
+++
title = "Contributing to Norg"
weight = 80
+++

12
docs/content/contributing/contributing.md Normal file
View file

@ -0,0 +1,12 @@
+++
title = "Contributing"
weight = 80
+++
We are very happy to take on an ideas or issues you may have with Norg, and
welcome pull requests that can implement these features within reason.
The source code is published here: [codeberg.org/pswilde/norgbackup](https://codeberg.org/pswilde/norgbackup), which is where you can raise issue and make pull requests.
When creating new features, or adjusting anything within the current state of Norg,
please also update the documentation to reflect these changes.
The documentation uses the [Zola](https://getzola.org) static site generator,
which uses markdown content files in the `docs/content/` directory.

4
docs/content/reference/command_line.md
View file

@ -28,4 +28,8 @@ Options:
-s, --stats Provides statistics at the end of a backup (Borg only)
```
The `-c`, or `--config`, parameter is required. Failing to provide this will only show the above help information.
When specifying the `-r` or `--repository` parameter, you can specify either
the `label` or `path` for the repository defined in the configuration file.

24
docs/content/usage/before_and_after_actions.md Normal file
View file

@ -0,0 +1,24 @@
+++
title = "Before and After Actions"
weight = 90
+++
Norg has the ability to run other commands before and after various points
during its running session.
These times are as follows:
* `before_everything` - Runs before anything else happens, just after starting.
* `after_everything` - Runs after everything has happened, just before closing.
* `before_actions` - Runs before any action for each repository.
* `after_actions` - Runs after any action for each repository.
* `before_backup` - Runs before each backup for each repository.
* `after_backup` - Runs after each backup for each repository.
* `before_extract` - Runs before an extract.
* `after_extract` - Runs after an extract.
* `before_prune` - Runs before a prune job (when run manually).
* `after_prune` - Runs after a prune job (when run manually).
* `before_compact` - Runs before a compact action, for each repository.
* `after_compact` - Runs after a compact aciton, for each repository.
* `before_check` - Runs before a check action, for each repository.
* `after_check` - Runs after a check action, for each repository.
The general use here is to run a database backup/dump to store a database
backup file in a location that is then backed up, but could be used for other things.

33
docs/content/usage/both_borg_and_restic.md Normal file
View file

@ -0,0 +1,33 @@
+++
title = "Using both Borg and Restic"
weight = 999
+++
Allowances have been made to be able to use both Borg and Restic repositories
for the same backup in one configuration file, and this will largely work
correctly. However, caution must be taken when using any direct Borg/Restic
command line parameters as they could cause either backup tool to fail as it
does not understand the parameter given.
A good example of this is the `--stats` parameter. Borg understands this and
outputs backup stats when its backup has completed. However, Restic does not
have a `--stats` parameter and fails to run when it is provided.
Since `Norg version 0.1.6`, we have taken steps to ignore the `--stats` parameter
when provided for restic repositories, so it will work correctly. However,
other parameters could cause a similar effect. We shall try to cater for these
as they are learned about. Please file an issue on our [Codeberg Repo](https://codeberg.org/pswilde/norgbackup/issues) if you discover a parameter which causes either backup tool to fail to run.
## Aliased Commands
Terminology in both Borg and Restic differs, but the result is effectively the same.
Some different yet similar commands are aliased so they can be used interchangeably.
The table below shows some of these:
| Borg Command | Restic Command | Result |
| --- | --- | --- |
| create | backup | creates a backup |
| list | snapshots | lists archives/snapshots |
| extract | restore | restores/extracts a backup |
| delete | forget | removes a archive/snapshot |
| prune | forget (with --prune flag) | removes snapshots as per `--keep-*` config |
You may specify either command and it will work with both except the `forget` command. This will only forget a single snapshot in restic, and delete a single archive with borg.

19
docs/content/usage/configuration.md Normal file
View file

@ -0,0 +1,19 @@
+++
title = "Configuring Norg"
weight = 20
+++
Norg is configured using a [TOML](https://toml.io) format configuration file.
There is an [Example Configuration File](/reference/example-configuration) on the norgbackup website that you can use as a base to create your own backups.
## Many backups from one system
You can run multiple different backups on the same system by specifying different
configuration files using the `-c` [command line parameter](/reference/command-line).
## Scheduling
Norg can be scheduled to run using your system's task scheduler. For the sake of
this example, we are using `cron` to run a backups every day at 8pm and 10pm.
```ini
0 20 * * * norg -c /home/me/myconfig.toml
0 22 * * * norg -c /home/me/myotherbackup.toml
```
This concept should be transferrable to any other scheduling tool.

21
docs/content/usage/create_a_backup.md Normal file
View file

@ -0,0 +1,21 @@
+++
title = "Create a Backup"
weight = 50
+++
Now norg is configured and your repositories are created and initialized, it's
sensible to test everything is working. Run norg with the `create` command to start a backup like this:
```sh
norg -c myconfig.toml create --stats
```
As the command runs, you should check the output of borg or restic to see if files
you expect to be backed up are being backed up.
The `--stats` flag provides statistics of a borg backup once completed. It is
ignored for Restic backups.
The `-c` or `--config` option is required, and must reference a norg configuration file.
## Default Options
If you do not specify `create`, or any other option, norg will default to the `create` option anyway, so the above command can be simplified as so:
```sh
norg -c myconfig.toml
```
With Restic, the terminology for creating a snapshot is `backup`. Norg has both `create` and `backup` as aliases to each other, so either will work as desired.

46
docs/content/usage/extract_a_backup.md Normal file
View file

@ -0,0 +1,46 @@
+++
title = "Extract/Restore a Backup"
weight = 70
+++
> _"People don't want backups. People want Restores."_
What's the point in backing up if you can't restore from it?
Of course, Norg has a restore functionality, and can restore an entire backup to
a directory of your choice.
To protect current data, Norg will only restore to an empty directory, so it is
important to specify a directory that is empty. Norg will create directories that
do not exist.
## Steps
First, find the backup you wish to restore. This can easily be done by using
the [`list`](http://127.0.0.1:1111/usage/inspect-your-backups#list-existing-backups) command.
```sh
norg -c myconfig.toml list
```
Which will give you an output similar to this:
```sh
# For Borg
hostname-2024-08-23T15:46:19.037234 Fri, 2024-08-23 15:46:19
hostname-2024-08-24T15:21:17.738163 Sat, 2024-08-24 15:21:18
# or for Restic
ID Time Host Paths Size
---------------------------------------------------------------
5f2bc4f1 2024-08-23 10:30:25 hostname /home/me 96.361 MiB
e0de9b6c 2024-08-24 11:50:52 hostname /home/me 94.123 MiB
```
When you know which archive or snapshot you want to extract, you can run the
`extract` command with `repository`, `archive` and `destination` parameters.
```sh
# Borg example
norg -c myconfig.toml extract -r MyBorgRepo -a hostname-2024-08-23T15:46:19.037234 --destination /my/restore/location
# Restic example
norg -c myconfig.toml extract -r MyResticRepo -a 5f2bc4f1 --destination /my/restore/location
```
## Further Command Line Reference
Like with all other commands, you can specify Borg/Restic command line parameters after all other parameters to adjust how the extract will run. Please see [Using both Borg and Restic](/usage/both-borg-and-restic/) for more information.

17
docs/content/usage/initialize.md Normal file
View file

@ -0,0 +1,17 @@
+++
title = "Initialize Repositories"
weight = 40
+++
Your backup repositories need to be initialized before they can be used. Norg
has a command to do this for you once you.
To initialize your repositories one by one, you can run the following command:
```sh
norg -c myconfig.toml init -e repokey-blake2
```
This will initialize a Borg repository using the `repokey-blake2` encryption algorithm, making it ready for use.
Restic does not use the same encryption algorithm option, so the above command may fail. However, running the below command again, without the `-e repokey-blake2` option
will initialize a Restic repository correctly, leaving the borg repository untouched.
**TODO**: We plan to resolve the requirement of running this command twice in the future.

50
docs/content/usage/inspect_your_backups.md Normal file
View file

@ -0,0 +1,50 @@
+++
title = "Inspect your Backups"
weight = 60
+++
The output of a backup will be shown in your terminal if running manually.
You could output this into a log file if you wish, like so:
```sh
norg -c myconfig.toml --stats >> mybackup.log
```
This is great for letting you know some basic information about your backup.
## List Existing Backups
Norg provides command line parameters to allow you to list your backups for all
or specific repositories.
```sh
# List all archives/snapshots on all repositories
norg -c myconfig.toml list
# List all archives/snapshots on a specific repository
norg -c myconfig.toml list -r MyRepository
```
When specifying the `-r` or `--repository` parameter, you can specify either
the `label` or `path` for the repository defined in the configuration file.
You can also specify further [Borg or Restic command line options](/usage/both-borg-and-restic#further-args)
(like `--last 3` to show the last 3 archives in a Borg repository). It is advised to
*always* use the `-r` parameter in these cases as specifc flags for Borg do not
work with Restic and vice-versa.
## Mount a Backup
Both Borg and Restic provide the ability to mount an archive (Borg) or repository (Restic) so you can inspect and recover individual files this way. Norg provides the ability to do this via the below command:
```sh
# For Borg repositories, where we mount an Archive
norg -c myconfig.toml mount -r MyBorgRepository -a hostname-date-of-backup /path/to/mount/point
# For Restic repositories, where we mount the entire repository
norg -c myconfig.toml mount -r MyResticRepository /path/to/mount/point
```
It is sensible to always provide the `-r` parameter as you will not be able to mount multiple archive/repositories to the same location.
### Unmount a Backup
Unmounting a backup is different for both Borg and Restic.
Borg has a `umount` command which can be run via norg as follows:
```sh
norg -c myconfig.toml umount /path/to/mount/point
```
Restic however, remains running after a `mount` command has been issues, and awaits
`Ctrl+C` keyboard entry to unmount. In short, there is no `umount` command for Restic.

63
docs/content/usage/monitoring_backup_jobs.md Normal file
View file

@ -0,0 +1,63 @@
+++
title = "Knowing your Backup has Run"
weight = 65
+++
Great, we have backups set up and running automatically, but how do we know
they're running successfully?
Well, Norg can interact with monitoring tools to let you know backups are running...
Or, probably more accuractely, when they fail.
Currently only [Uptime Kuma](https://uptime.kuma.pet) is supported, but it would be
possible to connect to many other types of monitoring service by creating a new `notifiers/{notifier_name}.nim` codefile and linking it all up.
## Supported Monitoring Tools
- [Uptime Kuma](#uptime-kuma)
- [Ntfy](#ntfy) - Planned, Not implemented yet.
- [Healthchecks.io](#healthchecks-io) - Planned, Not implemented yet.
### Uptime Kuma
Uptime Kuma is a self-hosted monitoring tool. It's generally used to call *out* to
a service or server every minute to check it is running. But, it does have the option
of a "Push" type of monitor, which waits for a service to make a `HTTP GET` request to it, and reports on the status of that request, or if that request is missed within a timeframe.
Some basic Uptime Kuma Settings for a backup job would be as follows:
```ini
# These are to be entered into Uptime Kuma and not into your Norg
# configuration.
# Push monitors wait for the client to contact Uptime Kuma instead of Uptime
# Kuma contacting the client. This is perfect for backup monitoring.
Monitor Type = Push
Heartbeat Interval = 90000 # = 25 hours = 1 day + 1 hour
# Wait 6 times the Heartbeat Retry (below) before logging a heartbeat missed.
Retries = 6
# Multiplied by Retries this gives a grace period within which the monitor
# goes into the "Pending" state.
Heartbeat Retry = 360 # = 10 minutes
# For each Heartbeat Interval if the backup fails repeatedly, a notification
# is sent each time.
Resend Notification every X times = 1
```
Your Norg configuration file should have an `uptimekuma` section as below
```toml
...
[uptimekuma]
base_url = "https://uptime.kuma.example/api/push/abcd1234"
states = ["Success", "Failure", "Running"]
```
This will make Norg request your Uptime Kuma service on `Running`, `Failure` and `Success` states. Uptime Kuma could then alert you about this change in state,
depending on how your Uptime Kuma is configured.
### Ntfy
Not implemented yet.
### Healthchecks.io
Not implemented yet.

36
docs/content/usage/redundancy.md Normal file
View file

@ -0,0 +1,36 @@
+++
title = "Backup Redundancy"
weight = 30
+++
Norg has the ability to back up to multiple repositories using both the [BorgBackup](https://borgbackup.com) or [Restic](https://restic.net) backup utilities.
To enable this, simply configure multiple different repositories in your configuration file and norg will take care of the rest.
For example, the `repositories` section of your configuration file may look like this:
```toml
...
[[repositories]]
label = "MyFirstLocation"
path = "/location1"
[[repositories]]
label = "MySecondLocation"
path = "ssh://1234abcd@1234abcd.repo.borgbase.com/./repo"
[[repositories]]
label = "MyThirdLocationUsingRestic"
path = "/location3"
tool = "restic"
[[repositories]]
label = "MyFourthLocationUsingRestic"
path = "rest:https://5678zyxw:9090abab@5678zyxw.repo.borgbase.com"
tool = "restic"
...
```
When running norg with this configuration, it invokes Borg or Restic once per
repository in sequence (i.e. not at the same time) which means in each
repository a new archive or snapshot is made containing all source directories.
This provides redundancy across multiple backup repositories and across backup
tools. You could even configure multiple storage providers to gain redundancy
there as well.

33
docs/content/usage/setup.md
View file

@ -1,14 +1,14 @@
+++
title = "Setting up Norg"
weight = 0
weight = 10
+++
## Prerequisites
Install [Borg](https://borgbackup.readthedocs.io/en/stable/installation.html) or [Restic](https://restic.net/#installation) as these do not get installed automatically.
You will also need to install [Nim](https://nim-lang.org) and `nimble`
You will also need to install [Nim v2](https://nim-lang.org) and `nimble`
- [FreeBSD](https://www.freebsd.org) - `pkg install nimble`
- [Arch Linux](https://archlinux.org) - `pacman -S nimble`
- [Debian](https://www.debian.org) - `apt install nim`
- For other distros you may need to use the official [`choosenim` Installer](https://nim-lang.org/install_unix.html) to get the latest version of nim installed.
## Installation
Norg is listed on [Nimble Directory](https://nimble.directory) so you can install
@ -16,3 +16,30 @@ norg using nimble
```sh
nimble install norg
```
or, if you'd rather not use the directory, you can install direct from our
[Codeberg Repo](https://codeberg.org/pswilde/norgbackup):
```sh
nimble install https://codeberg.org/pswilde/norgbackup
```
#### PATH variable
Once installed, `norg` will be installed in your `$HOME/.nimble/bin/` directory.
You should add this location to your `$PATH` variable so it can be found remains
up-to-date when running `nimble install norg` again.
## Portability
Norg is a single binary executable, so it can be copied wherever you need providing
it has been compiled in the required system and architecture beforehand.
For example, I have multiple FreeBSD servers where I use Norg. I install the latest version on one of these servers, then just copy that binary to all others servers that
need to make use of it.
## Upgrading Norg
Norg can be upgraded easily via nimble:
```sh
# Update nimble directory cache
nimble update
# Upgrade Norg
nimble install norg
```
The latest version of Norg should then be installed in your `$HOME/.nimble/bin/` directory.