psw/web
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
web/ponderings/norg/index.html
Paul Wilde 808b0849e7 updated norg backup
2024-08-23 14:47:22 +01:00

329 lines
22 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html lang="en">
<head>
<title>Norg Backup Utility | Paul&#x27;s Site of Stuff</title>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1">
<meta name="robots" content="noodp"/>
<link rel="stylesheet" href="https://paulwilde.uk/style.css">
<link rel="stylesheet" href="https://paulwilde.uk/color/orange.css">
<link rel="stylesheet" href="https://paulwilde.uk/color/background_blue.css">
<link rel="stylesheet" href="https://paulwilde.uk/font-hack-subset.css">
<meta name="description" content="">
<meta property="og:description" content="">
<meta property="og:title" content="Norg Backup Utility | Paul's Site of Stuff">
<meta property="og:type" content="article">
<meta property="og:url" content="https://paulwilde.uk/ponderings/norg/">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:description" content="">
<meta name="twitter:title" content="Norg Backup Utility | Paul's Site of Stuff">
<meta property="twitter:domain" content="paulwilde.uk">
<meta property="twitter:url" content="https://paulwilde.uk/ponderings/norg/">
<link rel="alternate" type="application/atom+xml" title="RSS" href="https://paulwilde.uk/atom.xml">
<link rel="shortcut icon" type="image/png" href="/favicon.png">
<script defer data-domain="paulwilde.uk" src="https://plausible.io/js/script.js"></script>
</head>
<body class="">
<div class="container">
<header class="header">
<div class="header__inner">
<div class="header__logo">
<a href="https://paulwilde.uk" style="text-decoration: none;">
<div class="logo">
Hello, I&#x27;m Paul
</div>
</a>
</div>
</div>
<nav class="menu">
<ul class="menu__inner">
<li><a href="/">home</a></li>
<li><a href="/aboutme">about me</a></li>
<li><a href="/ponderings">ponderings</a></li>
<li><a href="/iuse">i use …</a></li>
<li><a href="/tags">tags</a></li>
<li><a href="/atom.xml">rss</a></li>
</ul>
</nav>
</header>
<div class="post">
<h1 class="post-title"><a href="https://paulwilde.uk/ponderings/norg/">Norg Backup Utility</a></h1>
<div class="post-meta-inline">
<span class="post-date">
2024-08-18
</span>
</div>
<span class="post-tags-inline">
:: tags:&nbsp;
<a class="post-tag" href="https://paulwilde.uk/tags/backups/">#backups</a>&nbsp;
<a class="post-tag" href="https://paulwilde.uk/tags/borgbackup/">#borgbackup</a>&nbsp;
<a class="post-tag" href="https://paulwilde.uk/tags/development/">#development</a>&nbsp;
<a class="post-tag" href="https://paulwilde.uk/tags/freebsd/">#freebsd</a>&nbsp;
<a class="post-tag" href="https://paulwilde.uk/tags/linux/">#linux</a>&nbsp;
<a class="post-tag" href="https://paulwilde.uk/tags/nim/">#nim</a>&nbsp;
<a class="post-tag" href="https://paulwilde.uk/tags/tools/">#tools</a></span>
<div class="sourcecode">
<a href="https:&#x2F;&#x2F;codeberg.org&#x2F;pswilde&#x2F;norgbackup" target="_blank">source code</a>
</div>
<div class="post-content">
<h1 id="norg">Norg</h1>
<p>A simple, portable, wrapper for the <a href="https://www.borgbackup.org">borg backup</a> and <a href="https://restic.net">restic</a> utilities written in Nim.</p>
<span id="continue-reading"></span>
<p>Inspired by <a href="https://torsion.org/borgmatic">Borgmatic</a></p>
<h2 id="usage">Usage</h2>
<p>Norg uses a <code>toml</code> based config file for configuration. An example configuration would look like this:</p>
<pre data-lang="toml" style="background-color:#212121;color:#eeffff;" class="language-toml "><code class="language-toml" data-lang="toml"><span style="color:#f07178;">source_directories </span><span style="color:#89ddff;">= [
</span><span> </span><span style="color:#89ddff;">&quot;</span><span style="color:#c3e88d;">/home/me/Music</span><span style="color:#89ddff;">&quot;,
</span><span> </span><span style="color:#89ddff;">&quot;</span><span style="color:#c3e88d;">/home/me/Pictures</span><span style="color:#89ddff;">&quot;
</span><span style="color:#89ddff;">]
</span><span style="color:#89ddff;">[[repositories]]
</span><span style="color:#f07178;">label </span><span style="color:#89ddff;">= &quot;</span><span style="color:#c3e88d;">MyBorgRepo</span><span style="color:#89ddff;">&quot;
</span><span style="color:#f07178;">path </span><span style="color:#89ddff;">= &quot;</span><span style="color:#c3e88d;">/my/backup/location</span><span style="color:#89ddff;">&quot;
</span><span>
</span><span style="color:#89ddff;">[[repositories]]
</span><span style="color:#f07178;">label </span><span style="color:#89ddff;">= &quot;</span><span style="color:#c3e88d;">Another Respository at BorgBase</span><span style="color:#89ddff;">&quot;
</span><span style="color:#f07178;">path </span><span style="color:#89ddff;">= &quot;</span><span style="color:#c3e88d;">ssh://1234abcd@1234abcd.repo.borgbase.com/./repo</span><span style="color:#89ddff;">&quot;
</span><span>
</span><span style="color:#89ddff;">[encryption]
</span><span style="color:#f07178;">encryption_passphrase </span><span style="color:#89ddff;">= &quot;</span><span style="color:#c3e88d;">MyReallySecurePassword</span><span style="color:#89ddff;">&quot;
</span><span>
</span><span style="color:#89ddff;">[actions]
</span><span style="color:#f07178;">before_actions </span><span style="color:#89ddff;">= [&quot;</span><span style="color:#c3e88d;">echo before actions</span><span style="color:#89ddff;">&quot;]
</span><span style="color:#f07178;">after_actions </span><span style="color:#89ddff;">= [&quot;</span><span style="color:#c3e88d;">echo after actions</span><span style="color:#89ddff;">&quot;, &quot;</span><span style="color:#c3e88d;">echo actions completed</span><span style="color:#89ddff;">&quot;]
</span><span style="color:#f07178;">before_backup </span><span style="color:#89ddff;">= [&quot;</span><span style="color:#c3e88d;">echo before backup</span><span style="color:#89ddff;">&quot;, &quot;</span><span style="color:#c3e88d;">date</span><span style="color:#89ddff;">&quot;]
</span><span style="color:#f07178;">after_backup </span><span style="color:#89ddff;">= [&quot;</span><span style="color:#c3e88d;">echo after backup</span><span style="color:#89ddff;">&quot;,&quot;</span><span style="color:#c3e88d;">echo backup completed</span><span style="color:#89ddff;">&quot;]
</span><span>
</span><span style="color:#89ddff;">[uptimekuma]
</span><span style="color:#f07178;">base_url </span><span style="color:#89ddff;">= &quot;</span><span style="color:#c3e88d;">https://uptime.kuma.url/api/push/1234abcd</span><span style="color:#89ddff;">&quot;
</span><span style="color:#f07178;">states </span><span style="color:#89ddff;">= [&quot;</span><span style="color:#c3e88d;">Success</span><span style="color:#89ddff;">&quot;,&quot;</span><span style="color:#c3e88d;">Failure</span><span style="color:#89ddff;">&quot;, &quot;</span><span style="color:#c3e88d;">Running</span><span style="color:#89ddff;">&quot;]
</span></code></pre>
<p>You can then run the equivalent <code>borg</code> or <code>restic</code> command to init, create, list, mount and extract your backups.</p>
<p><strong>Using BorgBackup</strong></p>
<pre data-lang="sh" style="background-color:#212121;color:#eeffff;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="font-style:italic;color:#4a4a4a;"># Init your repository
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml init
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Backup your data
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml create
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># List Archives
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml list
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Mount an Archive
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml mount</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">r</span><span style="color:#82aaff;"> MyBorgRepo</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">a</span><span style="color:#82aaff;"> pcname-2024-08-18T15:20:17773204 /home/me/mnt
</span><span style="font-style:italic;color:#4a4a4a;"># Unmount an Archive
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml umount</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">r</span><span style="color:#82aaff;"> MyBorgRepo /home/me/mnt
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Extract an Archive
</span><span style="font-style:italic;color:#4a4a4a;"># You must be in an empty folder for this to work
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml extract</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">r</span><span style="color:#82aaff;"> MyBorgRepo</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">a</span><span style="color:#82aaff;"> pcname-2024-08-18T15:20:17773204
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Or You must set the destination to an empty folder
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml extract</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">r</span><span style="color:#82aaff;"> MyBorgRepo</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">a</span><span style="color:#82aaff;"> pcname-2024-08-18T15:20:17773204</span><span style="color:#89ddff;"> --</span><span style="color:#f78c6c;">destination</span><span style="color:#82aaff;"> /tmp/my_extracted_archive
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Prune all repos
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml prune
</span><span style="font-style:italic;color:#4a4a4a;"># Or specify a particula repo
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml prune</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">r</span><span style="color:#82aaff;"> MyBorgRepo
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Delete an Archive
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml delete</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">r</span><span style="color:#82aaff;"> MyBorgRepo</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">a</span><span style="color:#82aaff;"> pcname-2024-08-18T15:20:17773204
</span></code></pre>
<p><strong>Using Restic</strong> <em>New in v0.1.6</em>
Add a repository with a <code>tool = "restic"</code> option.</p>
<pre data-lang="toml" style="background-color:#212121;color:#eeffff;" class="language-toml "><code class="language-toml" data-lang="toml"><span style="color:#89ddff;">[[repositories]]
</span><span style="color:#f07178;">label </span><span style="color:#89ddff;">= &quot;</span><span style="color:#c3e88d;">MyResticRepo</span><span style="color:#89ddff;">&quot;
</span><span style="color:#f07178;">path </span><span style="color:#89ddff;">= &quot;</span><span style="color:#c3e88d;">/my/restic/backup/location</span><span style="color:#89ddff;">&quot;
</span><span style="color:#f07178;">tool </span><span style="color:#89ddff;">= &quot;</span><span style="color:#c3e88d;">restic</span><span style="color:#89ddff;">&quot;
</span></code></pre>
<p>Then run the appropriate commands</p>
<pre data-lang="sh" style="background-color:#212121;color:#eeffff;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="font-style:italic;color:#4a4a4a;"># Init your repository
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml init
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Backup your data
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml backup
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># List Snapshots
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml snapshots
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Mount a Repo
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml mount</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">r</span><span style="color:#82aaff;"> MyResticRepo /home/me/mnt
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Restore an Archive (restore destination must be empty)
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml restore</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">a</span><span style="color:#82aaff;"> latest</span><span style="color:#89ddff;"> --</span><span style="color:#f78c6c;">destination</span><span style="color:#82aaff;"> /my/restore/location
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Prune a repo
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml prune
</span><span>
</span><span style="font-style:italic;color:#4a4a4a;"># Forget a Snapshot
</span><span style="color:#82aaff;">norg</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">c</span><span style="color:#82aaff;"> myconfig.toml forget</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">r</span><span style="color:#82aaff;"> MyBorgRepo</span><span style="color:#89ddff;"> -</span><span style="color:#f78c6c;">a</span><span style="color:#82aaff;"> a1b2c3d4
</span></code></pre>
<h3 id="command-line-parameters">Command line parameters</h3>
<ul>
<li><code>-c</code>, <code>--config</code>: The configuration file to use</li>
<li><code>-r</code>, <code>--repository</code>: The repository to work on</li>
<li><code>-a</code>, <code>--archive</code>: The Archive to operate on (snapshots for restic)</li>
<li><code>-d</code>, <code>--destination</code>: When extracting/restoring, the destination for the extracted files</li>
</ul>
<h2 id="build-from-source">Build from Source</h2>
<p>Download and build from source</p>
<pre data-lang="sh" style="background-color:#212121;color:#eeffff;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#82aaff;">git clone https://codeberg.org/pswilde/norgbackup
</span><span style="color:#82aaff;">cd norgbackup
</span><span style="color:#82aaff;">nimble install
</span></code></pre>
<p>or just install directly with <code>nimble</code></p>
<pre data-lang="sh" style="background-color:#212121;color:#eeffff;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#82aaff;">nimble install https://codeberg.org/pswilde/norgbackup
</span></code></pre>
<h2 id="system-support">System Support</h2>
<p>Norg should work on any system that can compile <code>nim</code> code with <code>nimble</code>.
Tested on:</p>
<ul>
<li>Arch Linux</li>
<li>Debian Linux</li>
<li>AlmaLinux</li>
<li>FreeBSD</li>
</ul>
<p>But in general all Linux distributions and BSDs should work if borg and/or restic is installed.
Windows support (Restic only until Borg support in Windows is available) is planned with the only real issue being finding the restic executable. Should be an easy fix.</p>
<h2 id="naming-why-norg">Naming. Why "Norg"?</h2>
<p>Well, I don't know. I'm a Star Trek fan so obviously I wanted to keep something
in line with the <a href="https://memory-alpha.fandom.com/wiki/Borg">Borg pseudo-species</a> as the borg backup utility does.
Also, sometimes I feel my code has elements of inexperience but loads of potential... which reminded me of <a href="https://memory-alpha.fandom.com/wiki/Nog">Nog</a>.
So, simply put, <code>Norg</code> is an portmanteau of "Borg" and "Nog".</p>
<h2 id="borg-and-restic-notes">Borg and Restic Notes</h2>
<p>I love both Borg and Restic tools, they are both great and both have their pros and cons. As <a href="https://borgbase.com">BorgBase</a> has repos for both, I felt it only sensible to
provide a tool that can use both.
Providing implementation for both means you could have duplicate backups using
different tools which should provide a certain amount of protection over failures in
a particular tool.
Caution should be taken when using additional flags when you have repositories of
both types in the same configuration file. I have tried to cater for some common flags
that will be converted to the correct type for a particular tool, but this may not always be the case. If in any doubt, it is advised to use the <code>--repository</code> flag for any borg/restic specific flags so as not to cause one the other tool to fail.</p>
<p>Some different yet similar commands should be converted to the correct type. A table below shows some of these:
| Borg Command | Restic Command | Result |
| create | backup | creates a backup |
| list | snapshots | lists archives/snapshots |
| extract | restore | restores/extracts a backup |
| delete | forget | removes a archive/snapshot |
| prune | forget (with --prune flag) | removes snapshots as per <code>--keep-*</code> config |</p>
<p>You may specify either command and it will work with both except the <code>forget</code> command. This will only forget a single snapshot in restic.</p>
<h2 id="why-create-this-when-borgmatic-exists">Why create this when Borgmatic exists?</h2>
<p><code>Borgmatic</code> is absolutely fantastic, and I love it dearly. I even implemented
the <code>Uptime Kuma</code> hook that is in it. However, I got a little impatient waiting
for the version that included the Uptime Kuma hook to arrive in various distributions
package repositories so ended up building borgmatic from source on all computers.
This was a lengthy process, and borgmatic isn't very portable; it requires installation of numerous python packages (and the entire rust language in FreeBSD).
I wanted to make something that had to features I needed, in a single binary I
could move around to whatever computer I needed it on.
Norg was the outcome of this.</p>
<h2 id="work-in-progress">Work in Progress</h2>
<p>Norg is still very much a work in progress, so there will be bugs. Please raise
and issue, or create a pull request for any issues and resolutions you may have.</p>
<h2 id="contact">Contact</h2>
<p>For any issues, please raise an issue here. Otherwise, I can be contacted via
the fediverse at <a href="https://notnull.space/@paul">@paul@notnull.space</a>.</p>
</div>
<div class="pagination">
<div class="pagination__title">
<span class="pagination__title-h">Thanks for reading! Read other posts?</span>
<hr />
</div>
<div class="pagination__buttons">
<span class="button previous">
<a href="https://paulwilde.uk/ponderings/batmon/">
<span class="button__icon">←</span>&nbsp;
<span class="button__text">BatMon</span>
</a>
</span>
</div>
</div>
</div>
<footer class="footer">
<div class="footer__inner">
<a href="https:&#x2F;&#x2F;notnull.space&#x2F;@paul" rel="me">fediverse (gts)</a>
<a href="https:&#x2F;&#x2F;snac.notnull.space&#x2F;paul" rel="me">fediverse (snac)</a>
<a href="https:&#x2F;&#x2F;codeberg.org&#x2F;pswilde" rel="me">codeberg</a>
<a href="https:&#x2F;&#x2F;keyoxide.org&#x2F;85633E30514CC1932E4268460ED12CF710BC42CA" rel="me">keyoxide</a>
</div>
<div class="footer__inner">
<div class="copyright">
<span>©
2024
Paul Wilde</span>
<span class="copyright-theme">
<span class="copyright-theme-sep">:: </span>
Theme: <a href="https://github.com/pawroman/zola-theme-terminimal/">Terminimal</a>
</span>
</div>
</div>
</footer>
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink