2019-02-04 07:20:59 +01:00
|
|
|
---
|
|
|
|
title: How to develop on borgmatic
|
2020-08-21 23:27:47 +02:00
|
|
|
eleventyNavigation:
|
2022-05-20 20:11:35 +02:00
|
|
|
key: 🏗️ Develop on borgmatic
|
2020-08-21 23:27:47 +02:00
|
|
|
parent: How-to guides
|
2024-01-21 20:34:40 +01:00
|
|
|
order: 14
|
2019-02-04 07:20:59 +01:00
|
|
|
---
|
|
|
|
## Source code
|
|
|
|
|
2023-09-05 01:25:10 +02:00
|
|
|
To get set up to develop on borgmatic, first [`install
|
|
|
|
pipx`](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#installation)
|
|
|
|
to make managing your borgmatic environment easy without impacting other
|
|
|
|
Python applications on your system.
|
|
|
|
|
|
|
|
Then, clone borgmatic via HTTPS or SSH:
|
2019-02-04 07:20:59 +01:00
|
|
|
|
|
|
|
```bash
|
2021-09-14 20:32:01 +02:00
|
|
|
git clone https://projects.torsion.org/borgmatic-collective/borgmatic.git
|
2019-02-04 07:20:59 +01:00
|
|
|
```
|
|
|
|
|
|
|
|
Or:
|
|
|
|
|
|
|
|
```bash
|
2021-09-14 20:32:01 +02:00
|
|
|
git clone ssh://git@projects.torsion.org:3022/borgmatic-collective/borgmatic.git
|
2019-02-04 07:20:59 +01:00
|
|
|
```
|
|
|
|
|
2023-09-05 01:25:10 +02:00
|
|
|
Finally, install borgmatic
|
|
|
|
"[editable](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs)"
|
2023-07-12 07:10:57 +02:00
|
|
|
so that you can run borgmatic actions during development to make sure your
|
2023-09-05 01:25:10 +02:00
|
|
|
changes work:
|
2019-02-04 07:20:59 +01:00
|
|
|
|
|
|
|
```bash
|
2023-03-28 21:45:39 +02:00
|
|
|
cd borgmatic
|
2023-09-18 07:46:33 +02:00
|
|
|
pipx ensurepath
|
2023-09-05 01:25:10 +02:00
|
|
|
pipx install --editable .
|
2019-02-04 07:20:59 +01:00
|
|
|
```
|
|
|
|
|
2024-03-11 00:18:49 +01:00
|
|
|
Or to work on the [Apprise
|
|
|
|
hook](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#apprise-hook),
|
|
|
|
change that last line to:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
pipx install --editable .[Apprise]
|
|
|
|
```
|
|
|
|
|
2023-08-07 08:44:31 +02:00
|
|
|
To get oriented with the borgmatic source code, have a look at the [source
|
|
|
|
code reference](https://torsion.org/borgmatic/docs/reference/source-code/).
|
|
|
|
|
2019-02-04 07:20:59 +01:00
|
|
|
|
|
|
|
## Automated tests
|
|
|
|
|
2023-09-05 01:25:10 +02:00
|
|
|
Assuming you've cloned the borgmatic source code as described above and you're
|
|
|
|
in the `borgmatic/` working copy, install tox, which is used for setting up
|
|
|
|
testing environments. You can either install a system package of tox (likely
|
|
|
|
called `tox` or `python-tox`) or you can install tox with pipx:
|
2019-02-04 07:20:59 +01:00
|
|
|
|
|
|
|
```bash
|
2023-09-05 01:25:10 +02:00
|
|
|
pipx install tox
|
2019-02-04 07:20:59 +01:00
|
|
|
```
|
|
|
|
|
2023-09-05 01:25:10 +02:00
|
|
|
Finally, to actually run tests, run tox from inside the borgmatic
|
|
|
|
sourcedirectory:
|
2019-02-04 07:20:59 +01:00
|
|
|
|
|
|
|
```bash
|
|
|
|
tox
|
|
|
|
```
|
|
|
|
|
|
|
|
### Code formatting
|
|
|
|
|
|
|
|
If when running tests, you get an error from the
|
|
|
|
[Black](https://black.readthedocs.io/en/stable/) code formatter about files
|
|
|
|
that would be reformatted, you can ask Black to format them for you via the
|
|
|
|
following:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
tox -e black
|
|
|
|
```
|
|
|
|
|
2019-05-16 21:06:55 +02:00
|
|
|
And if you get a complaint from the
|
|
|
|
[isort](https://github.com/timothycrosley/isort) Python import orderer, you
|
|
|
|
can ask isort to order your imports for you:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
tox -e isort
|
|
|
|
```
|
|
|
|
|
2023-04-01 20:03:59 +02:00
|
|
|
Similarly, if you get errors about spelling mistakes in source code, you can
|
2023-04-01 23:38:52 +02:00
|
|
|
ask [codespell](https://github.com/codespell-project/codespell) to correct
|
|
|
|
them:
|
2023-04-01 20:03:59 +02:00
|
|
|
|
|
|
|
```bash
|
|
|
|
tox -e codespell
|
|
|
|
```
|
|
|
|
|
|
|
|
|
2019-02-04 07:20:59 +01:00
|
|
|
### End-to-end tests
|
|
|
|
|
|
|
|
borgmatic additionally includes some end-to-end tests that integration test
|
2019-12-12 01:43:01 +01:00
|
|
|
with Borg and supported databases for a few representative scenarios. These
|
|
|
|
tests don't run by default when running `tox`, because they're relatively slow
|
2023-04-17 00:41:17 +02:00
|
|
|
and depend on containers for runtime dependencies. These tests do run on the
|
|
|
|
continuous integration (CI) server, and running them on your developer machine
|
2024-03-11 03:14:18 +01:00
|
|
|
is the closest thing to dev-CI parity.
|
2019-12-12 01:43:01 +01:00
|
|
|
|
2023-04-17 00:41:17 +02:00
|
|
|
If you would like to run the full test suite, first install Docker (or Podman;
|
|
|
|
see below) and [Docker Compose](https://docs.docker.com/compose/install/).
|
|
|
|
Then run:
|
2019-02-04 07:20:59 +01:00
|
|
|
|
|
|
|
```bash
|
2024-03-06 17:32:55 +01:00
|
|
|
scripts/run-end-to-end-tests
|
2019-02-04 07:20:59 +01:00
|
|
|
```
|
|
|
|
|
2023-04-17 00:41:17 +02:00
|
|
|
This script assumes you have permission to run `docker`. If you don't, then
|
|
|
|
you may need to run with `sudo`.
|
2019-12-12 01:43:01 +01:00
|
|
|
|
2023-04-10 23:26:54 +02:00
|
|
|
|
|
|
|
#### Podman
|
|
|
|
|
|
|
|
<span class="minilink minilink-addedin">New in version 1.7.12</span>
|
|
|
|
borgmatic's end-to-end tests optionally support using
|
|
|
|
[rootless](https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md)
|
|
|
|
[Podman](https://podman.io/) instead of Docker.
|
|
|
|
|
|
|
|
Setting up Podman is outside the scope of this documentation, but here are
|
|
|
|
some key points to double-check:
|
|
|
|
|
2023-04-22 19:07:40 +02:00
|
|
|
* Install Podman and your desired networking support.
|
2023-04-10 23:26:54 +02:00
|
|
|
* Configure `/etc/subuid` and `/etc/subgid` to map users/groups for the
|
|
|
|
non-root user who will run tests.
|
|
|
|
* Create a non-root Podman socket for that user:
|
|
|
|
```bash
|
|
|
|
systemctl --user enable --now podman.socket
|
2023-04-17 00:41:17 +02:00
|
|
|
systemctl --user start --now podman.socket
|
2023-04-10 23:26:54 +02:00
|
|
|
```
|
|
|
|
|
|
|
|
Then you'll be able to run end-to-end tests as per normal, and the test script
|
|
|
|
will automatically use your non-root Podman socket instead of a Docker socket.
|
|
|
|
|
|
|
|
|
2019-02-04 07:20:59 +01:00
|
|
|
## Code style
|
|
|
|
|
|
|
|
Start with [PEP 8](https://www.python.org/dev/peps/pep-0008/). But then, apply
|
|
|
|
the following deviations from it:
|
|
|
|
|
|
|
|
* For strings, prefer single quotes over double quotes.
|
|
|
|
* Limit all lines to a maximum of 100 characters.
|
|
|
|
* Use trailing commas within multiline values or argument lists.
|
2023-04-01 18:40:32 +02:00
|
|
|
* For multiline constructs, put opening and closing delimiters on lines
|
2019-02-04 07:20:59 +01:00
|
|
|
separate from their contents.
|
|
|
|
* Within multiline constructs, use standard four-space indentation. Don't align
|
2023-04-01 18:40:32 +02:00
|
|
|
indentation with an opening delimiter.
|
2024-03-11 00:18:49 +01:00
|
|
|
* In general, spell out words in variable names instead of shortening them.
|
|
|
|
So, think `index` instead of `idx`. There are some notable exceptions to
|
|
|
|
this though (like `config`).
|
2019-02-04 07:20:59 +01:00
|
|
|
|
|
|
|
borgmatic code uses the [Black](https://black.readthedocs.io/en/stable/) code
|
2019-05-16 21:06:55 +02:00
|
|
|
formatter, the [Flake8](http://flake8.pycqa.org/en/latest/) code checker, and
|
|
|
|
the [isort](https://github.com/timothycrosley/isort) import orderer, so
|
2019-02-04 07:20:59 +01:00
|
|
|
certain code style requirements will be enforced when running automated tests.
|
2019-05-16 21:06:55 +02:00
|
|
|
See the Black, Flake8, and isort documentation for more information.
|
2019-02-04 07:20:59 +01:00
|
|
|
|
2024-03-11 00:18:49 +01:00
|
|
|
|
2019-05-13 22:56:59 +02:00
|
|
|
## Continuous integration
|
|
|
|
|
2024-03-11 00:18:49 +01:00
|
|
|
Each commit to
|
|
|
|
[main](https://projects.torsion.org/borgmatic-collective/borgmatic/branches)
|
|
|
|
triggers [a continuous integration
|
2024-03-07 05:01:32 +01:00
|
|
|
build](https://projects.torsion.org/borgmatic-collective/borgmatic/actions)
|
|
|
|
which runs the test suite and updates
|
|
|
|
[documentation](https://torsion.org/borgmatic/). These builds are also linked
|
|
|
|
from the [commits for the main
|
|
|
|
branch](https://projects.torsion.org/borgmatic-collective/borgmatic/commits/branch/main).
|
2019-02-04 07:20:59 +01:00
|
|
|
|
2019-10-30 18:54:42 +01:00
|
|
|
## Documentation development
|
|
|
|
|
|
|
|
Updates to borgmatic's documentation are welcome. It's formatted in Markdown
|
|
|
|
and located in the `docs/` directory in borgmatic's source, plus the
|
|
|
|
`README.md` file at the root.
|
|
|
|
|
2019-10-30 18:55:40 +01:00
|
|
|
To build and view a copy of the documentation with your local changes, run the
|
|
|
|
following from the root of borgmatic's source code:
|
2019-10-30 18:54:42 +01:00
|
|
|
|
|
|
|
```bash
|
2023-04-17 00:41:17 +02:00
|
|
|
scripts/dev-docs
|
2019-10-30 18:54:42 +01:00
|
|
|
```
|
|
|
|
|
2023-04-17 00:41:17 +02:00
|
|
|
This requires Docker (or Podman; see below) to be installed on your system.
|
|
|
|
This script assumes you have permission to run `docker`. If you don't, then
|
|
|
|
you may need to run with `sudo`.
|
2019-10-30 18:54:42 +01:00
|
|
|
|
|
|
|
After you run the script, you can point your web browser at
|
|
|
|
http://localhost:8080 to view the documentation with your changes.
|
|
|
|
|
|
|
|
To close the documentation server, ctrl-C the script. Note that it does not
|
|
|
|
currently auto-reload, so you'll need to stop it and re-run it for any
|
|
|
|
additional documentation changes to take effect.
|
2023-04-10 23:26:54 +02:00
|
|
|
|
|
|
|
|
|
|
|
#### Podman
|
|
|
|
|
|
|
|
<span class="minilink minilink-addedin">New in version 1.7.12</span>
|
|
|
|
borgmatic's developer build for documentation optionally supports using
|
|
|
|
[rootless](https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md)
|
|
|
|
[Podman](https://podman.io/) instead of Docker.
|
|
|
|
|
|
|
|
Setting up Podman is outside the scope of this documentation. But once you
|
2023-04-22 19:07:40 +02:00
|
|
|
install and configure Podman, then `scripts/dev-docs` should automatically use
|
2023-04-10 23:26:54 +02:00
|
|
|
Podman instead of Docker.
|