Dan Helfman
8cec7c74d8
Add "--strip-components all" on the "extract" action to remove leading path components ( #647 ).
2023-03-09 10:09:16 -08:00
Dan Helfman
b343363bb8
Change the default action order to: "create", "prune", "compact", "check" ( #304 ).
2023-03-08 14:05:06 -08:00
Dan Helfman
9db31bd1e9
Run any command-line actions in the order specified instead of using a fixed ordering ( #304 ).
2023-03-08 13:19:41 -08:00
Dan Helfman
d88bcc8be9
Add Healthchecks "log" state feature to NEWS.
2023-03-07 15:45:23 -08:00
Dan Helfman
332f7c4bb6
Add support for healthchecks "log" feature ( #628 ).
...
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/645
2023-03-07 22:21:30 +00:00
Dan Helfman
5d19d86e4a
Add flake8-quotes to complain about incorrect quoting so I don't have to!
2023-03-07 14:08:35 -08:00
Soumik Dutta
044ae7869a
fix tests
...
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-03-08 03:30:12 +05:30
Soumik Dutta
98e429594e
added tests to make sure unsupported log states are detected
...
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-03-06 20:31:00 +05:30
Soumik Dutta
f442aeae9c
fix logs_monitor_start_error()
...
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-03-06 05:21:56 +05:30
Soumik Dutta
e211863cba
update test_borgmatic.py
...
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-03-06 05:12:24 +05:30
Soumik Dutta
45256ae33f
add test for healthchecks
...
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-03-06 03:38:08 +05:30
Dan Helfman
9ec220c600
Add SQLite database dump/restore hook ( #295 ).
...
feat: add dump-restore support for sqlite databases
2023-03-04 09:47:21 -08:00
Divyansh Singh
c71eb60cd2
mock os.remove instead of actually removing a file
2023-03-04 13:08:30 +05:30
Divyansh Singh
675e54ba9f
use os.remove and improve tests
2023-03-04 12:43:07 +05:30
Divyansh Singh
767a7d900b
e2e tests schema update
2023-03-04 01:29:01 +05:30
Divyansh Singh
903507bd03
code review
2023-03-04 01:27:07 +05:30
Dan Helfman
a071e02d20
With the "create" action and the "--list" ("--files") flag, only show excluded files at verbosity 2 ( #620 ).
2023-03-02 15:33:42 -08:00
Divyansh Singh
3aa88085ed
formatting fix
2023-03-03 00:01:52 +05:30
Divyansh Singh
af1cc27988
feat: add dump-restore support for sqlite databases
2023-03-02 23:55:16 +05:30
Dan Helfman
dbf8301c19
Add "checkpoint_volume" configuration option to creates checkpoints every specified number of bytes.
2023-02-27 10:47:17 -08:00
Dan Helfman
2a306bef12
Fix tests.
2023-02-26 23:34:17 -08:00
Dan Helfman
d7a07f0428
Support status character changes in Borg 2.0.0b5 when filtering out special files that cause Borg to hang.
2023-02-26 22:36:13 -08:00
Dan Helfman
da321e180d
Fix the "create" action with the "--dry-run" flag querying for databases when a PostgreSQL/MySQL "all" database is configured.
2023-02-26 22:15:12 -08:00
Dan Helfman
c6582e1171
Internally support new Borg 2.0.0b5 "--filter" status characters / item flags for the "create" action.
2023-02-26 17:17:25 -08:00
Dan Helfman
9b83afe491
With the "create" action, only one of "--list" ("--files") and "--progress" flags can be used.
2023-02-26 17:05:56 -08:00
Tom Hubrecht
95575c3450
Add auth test for the ntfy hook
2023-02-25 20:04:39 +01:00
Dan Helfman
418ebc8843
Add MySQL database hook "add_drop_database" configuration option to control whether dumped MySQL databases get dropped right before restore ( #642 ).
2023-02-20 15:32:47 -08:00
Dan Helfman
f5a448c7c2
Fix for potential data loss (data not getting backed up) when dumping large "directory" format PostgreSQL/MongoDB databases ( #643 ).
2023-02-20 15:18:51 -08:00
Dan Helfman
c1eb210253
Fix code style flake issue.
2023-01-26 15:09:35 -08:00
Dan Helfman
30cca62d09
Add configuration options for database command customization ( #630 ).
2023-01-26 14:59:17 -08:00
Dan Helfman
0e6b2c6773
Optionally dump "all" PostgreSQL databases to separate files instead of one combined dump file ( #438 , #560 ).
2023-01-25 23:31:07 -08:00
palto42
fd7ad86daa
conditional warning for excluding special files
2023-01-03 21:53:51 +01:00
Dan Helfman
f67c544be6
Optionally dump "all" PostgreSQL databases to separate files instead of one combined dump file ( #438 , #560 ).
2022-12-15 22:59:42 -08:00
Dan Helfman
c657764367
Fix logs that interfere with JSON output by making warnings go to stderr instead of stdout ( #602 ).
2022-12-02 12:12:10 -08:00
Dan Helfman
ca7e18bb29
Override PostgreSQL dump/restore commands via configuration options ( #311 ).
...
Merge pull request #49 from jpaniagualaconich/specify-pg-dump-restore-commands
2022-11-18 08:33:14 -08:00
Dan Helfman
6975a5b155
Fix "data" consistency check to support "check_last" and consistency "prefix" options ( #611 ).
2022-11-17 10:19:48 -08:00
Javier Paniagua
faf682ca35
specify pg dump/restore commands ( #311 )
2022-11-06 11:12:53 +01:00
Dan Helfman
89500df429
Fix traceback when a configuration section is present but lacking any options ( #604 ).
2022-10-23 13:56:03 -07:00
Dan Helfman
9c42e7e817
Fix regression in which "check" action errored on certain systems ( #597 , #598 ).
2022-10-14 16:19:26 -07:00
Dan Helfman
953277a066
Fix special file detection when broken symlinks are encountered ( #596 ).
2022-10-14 09:41:08 -07:00
Dan Helfman
d7f1c10c8c
To prevent Borg hangs, unconditionally delete stale named pipes before dumping databases ( #360 ).
2022-10-12 10:26:09 -07:00
Dan Helfman
a31ce337e9
Skip auto-exclusion of special files when user explicitly sets read_special to true ( #587 ).
2022-10-06 11:07:43 -07:00
Dan Helfman
c31702d092
Fix for potential data loss with "patterns_from". Also, display excluded files ( #590 ).
2022-10-04 22:57:18 -07:00
Dan Helfman
ba8fbe7a44
Add "break-lock" action for removing any repository and cache locks leftover from Borg aborting ( #357 ).
2022-10-04 13:42:18 -07:00
Dan Helfman
2774c2e4c0
Add support for Borg 2's "--match-archives" flag (replaces "--glob-archives") ( #591 ).
2022-10-03 22:50:37 -07:00
Dan Helfman
ae036aebd7
When the "read_special" option is true or database hooks are enabled, auto-exclude special files for a "create" action to prevent Borg from hanging ( #587 ).
2022-10-03 12:58:13 -07:00
Dan Helfman
ac7c7d4036
Warn when ignoring a configured "read_special" value of false, as true is needed when database hooks are enabled ( #587 ).
2022-09-20 13:52:13 -07:00
Dan Helfman
679bb839d7
Fix hang when database hooks are enabled and "patterns" contains a parent directory of "~/.borgmatic" ( #582 ).
2022-09-08 10:16:42 -07:00
Dan Helfman
9e64d847ef
Fix regression in which "borgmatic info --archive ..." showed repository info instead of archive info with Borg 1 ( #577 ).
2022-08-30 20:42:42 -07:00
Dan Helfman
87c97b7568
Fixed spurious, intermittent test failures related to command execution and logging.
2022-08-28 09:06:06 -07:00
Dan Helfman
c46f2b8508
Fix conflict between "patterns" and "source_directories" ( #574 ), make "source_directories" optional ( #542 ).
2022-08-25 11:55:34 -07:00
Dan Helfman
ef7e95e22a
Fix end-to-end tests.
2022-08-21 23:29:13 -07:00
Dan Helfman
317dc7fbce
Add "before_actions" and "after_actions" command hooks that run before/after all the actions for each repository, update docs to cover per-repository configurations ( #463 ).
2022-08-21 21:48:37 -07:00
Dan Helfman
462326406e
Drop only-style actions like "--create", rename "prune --files" to "prune --list", and add "--list" alias to "create" and "export-tar" ( #571 ).
2022-08-21 14:25:16 -07:00
Dan Helfman
540f9f6b72
Add missing test for "transfer" action ( #557 ).
2022-08-19 09:40:29 -07:00
Dan Helfman
f9d7faf884
Fix mount action to work without archive again ( #557 ).
2022-08-18 23:33:05 -07:00
Dan Helfman
7dee6194a2
Add new "transfer" action for Borg 2 ( #557 ).
2022-08-18 23:06:51 -07:00
Dan Helfman
68f9c1b950
Add generate-borgmatic-config end-to-end test.
2022-08-18 14:28:46 -07:00
Dan Helfman
43d711463c
Add additional command-line flags to rcreate action ( #557 ).
2022-08-18 14:28:12 -07:00
Dan Helfman
b40e9b7da2
Ignore archive filter parameters passed to list action when --archive is given ( #557 ).
2022-08-18 09:59:48 -07:00
Dan Helfman
89d201c8ff
Fleshing out NEWS for the Borg 2 changes.
2022-08-17 21:54:00 -07:00
Dan Helfman
f47c98c4a5
Rename several configuration options to match Borg 2 ( #557 ).
2022-08-17 21:14:58 -07:00
Dan Helfman
3b6ed06686
Add --other-repo flag to rcreate action ( #557 ).
2022-08-17 17:33:09 -07:00
Dan Helfman
57009e22b5
Use flag-related utility functions in info action ( #557 ).
2022-08-17 17:11:02 -07:00
Dan Helfman
3ab7a3b64a
Replace use of --prefix with --glob-archives in info action ( #557 ).
2022-08-17 15:36:19 -07:00
Dan Helfman
596dd49cf5
Use --glob-archives instead of --prefix on rlist command ( #557 ).
2022-08-17 14:26:35 -07:00
Dan Helfman
28d847b8b1
Warn and tranform on non-ssh://-style repositories ( #557 ).
2022-08-17 10:13:11 -07:00
Dan Helfman
30abd0e3de
Update borg action for Borg 2 support ( #557 ).
2022-08-16 09:30:00 -07:00
Dan Helfman
f36e38ec20
Update mount action for Borg 2 support ( #557 ).
2022-08-15 19:32:37 -07:00
Dan Helfman
d807ce095e
Update export-tar action for Borg 2 support ( #557 ).
2022-08-15 17:34:12 -07:00
Dan Helfman
7626fe1189
Disallow borg list --json with --archive or --find ( #557 ).
2022-08-15 15:40:28 -07:00
Dan Helfman
cc04bf57df
Update list action for Borg 2 support, add rinfo action, and update extract consistency check for Borg 2.
2022-08-15 15:04:40 -07:00
Dan Helfman
cce6d56661
Update extract action for Borg 2 support ( #557 ).
2022-08-13 23:07:29 -07:00
Dan Helfman
a05d0f378e
Factor out repository/archive flags formatting code from create action ( #557 ).
2022-08-13 22:50:14 -07:00
Dan Helfman
94321aec7a
Update compact action for Borg 2 support ( #557 ).
2022-08-13 22:07:15 -07:00
Dan Helfman
4a55749bd2
Update prune action for Borg 2 support ( #557 ).
2022-08-13 17:26:51 -07:00
Dan Helfman
2898e63166
Update create action for Borg 2 support ( #557 ).
2022-08-12 23:54:13 -07:00
Dan Helfman
c7176bd00a
Add rinfo action for Borg 2 support ( #557 ).
2022-08-12 23:06:56 -07:00
Dan Helfman
647ecdac29
Borg 2 support in borgmatic check action ( #557 ).
2022-08-12 15:46:33 -07:00
Dan Helfman
e7a8acfb96
Add missing rinfo action source files ( #557 ).
2022-08-12 14:59:03 -07:00
Dan Helfman
622caa0c21
Support for Borg 2's rcreate and rinfo sub-commands ( #557 ).
2022-08-12 14:53:20 -07:00
Dan Helfman
9aece3936a
Modify "mount" and "extract" actions to require the "--repository" flag when multiple repositories are configured ( #566 ).
2022-07-25 11:30:02 -07:00
Dan Helfman
bcad0de1a4
Add verify_tls option for Healthchecks to optionally disable TLS verification.
2022-07-23 23:11:41 -07:00
Uli
5c6407047f
feat: add verify_tls flag for Healthchecks
2022-07-24 07:37:00 +02:00
Dan Helfman
6ddae20fa1
Fix handling of "repository" and "data" consistency checks to prevent invalid Borg flags ( #565 ).
2022-07-23 21:02:21 -07:00
Dan Helfman
16066942e3
Fix traceback with "create" action and "--json" flag when a database hook is configured ( #563 ).
2022-07-19 10:25:10 -07:00
Dan Helfman
e85d551eac
Fix all database hooks to error when the requested database to restore isn't present in the Borg archive ( #560 ).
2022-07-06 23:21:24 -07:00
Dan Helfman
2b23a63a08
Add end-to-end test for overrides.
2022-07-06 18:20:51 -07:00
Dan Helfman
457ed80744
Fix environment variable plumbing so options in one configuration file aren't used for others ( #555 ).
2022-06-30 13:42:17 -07:00
Dan Helfman
10723efc68
Fix all monitoring hooks to warn if the server returns an HTTP 4xx error ( #554 ).
2022-06-29 21:19:40 -07:00
Dan Helfman
bd4d109009
Fix logging to include the full traceback when Borg experiences an internal error ( #553 ).
2022-06-28 13:38:24 -07:00
Dan Helfman
d372a86fe6
Code formatting.
2022-06-23 10:41:04 -07:00
Sébastien MB
b63c854509
Fix escaped environment variable in configuration
...
- when an env variable is escaped in the configuration file, we expect
not to resolve it and remove the escape char `\`
2022-06-17 09:50:56 +02:00
Dan Helfman
aecb6fcd74
Code style, rename command-line flag, and move new code into its own file ( #546 )
2022-06-16 11:35:24 -07:00
Sébastien MB
97b5cd089d
Allow environment variable resolution in configuration file
...
- all string fields containing an environment variable like ${FOO} will
be resolved
- supported format ${FOO}, ${FOO:-bar} and ${FOO-bar} to allow default
values if variable is not present in environment
- add --no-env argument for CLI to disable the feature which is enabled
by default
Resolves : #546
2022-06-16 18:52:54 +02:00
Gavin Chappell
a8b8d507b6
add a hook for sending push notifications via ntfy.sh
2022-06-09 21:10:38 +01:00
Dan Helfman
3561c93d74
Fix Healthchecks tests that leak global state, breaking downstream tests (discovered in #543 ).
2022-06-09 11:05:44 -07:00
Dan Helfman
9aefb5179f
Fix None find paths ( #541 ).
2022-06-03 15:20:05 -07:00
Dan Helfman
d14f22e121
Add "borgmatic list --find" flag for searching for files across multiple archives ( #541 ).
2022-06-03 15:12:14 -07:00
Dan Helfman
80ec3e7d97
Deprecate "borgmatic list --successful" flag, as listing only non-checkpoint (successful) archives is now the default in newer versions of Borg.
2022-06-02 20:35:39 -07:00
Dan Helfman
2bc91ac3d2
Add "generate-borgmatic-config --overwrite" flag to replace an existing destination file ( #539 ).
2022-05-29 16:03:55 -07:00
Dan Helfman
5b615d51a4
Add support for "borgmatic borg debug" command ( #538 ).
2022-05-29 15:43:03 -07:00
Dan Helfman
8fa90053cf
Add "borgmatic check --force" flag to ignore configured check frequencies ( #523 ).
2022-05-28 19:29:33 -07:00
Dan Helfman
ad0e2e0d7c
Tweak default check frequency to 1 month ( #523 ).
2022-05-28 15:49:50 -07:00
Dan Helfman
e76bfa555f
Reduce the default consistency check frequency and support configuring the frequency independently for each check ( #523 ).
2022-05-28 14:42:19 -07:00
Dan Helfman
8ddb7268eb
Reuse "borg info" function.
2022-05-27 13:51:11 -07:00
Dan Helfman
cb5fe02ebd
Fix broken Bash completion end-to-end test.
2022-05-26 11:18:46 -07:00
Dan Helfman
77b84f8a48
Add Bash completion script so you can tab-complete the borgmatic command-line.
2022-05-26 10:27:53 -07:00
Dan Helfman
be59a3e574
Fix generate-borgmatic-config with "--source" flag to support more complex schema changes like the new Healthchecks configuration options ( #536 ).
2022-05-25 10:26:26 -07:00
Dan Helfman
54933ebef5
Change connection failures for monitoring hooks to be warnings instead of errors ( #439 ).
2022-05-24 15:50:04 -07:00
Dan Helfman
157e59ac88
Add Healthchecks monitoring hook "send_logs" option to enable/disable sending borgmatic logs to the Healthchecks server ( #460 ).
2022-05-24 14:44:33 -07:00
Dan Helfman
8b179e4647
Reverse logic of Healtchecks "skip_states" option to just "states" ( #525 ).
2022-05-24 14:09:42 -07:00
Dan Helfman
865eff7d98
Add Healthchecks monitoring hook "skip_states" option to disable pinging for particular monitoring states ( #525 ).
2022-05-24 13:59:28 -07:00
Dan Helfman
b9741f4d0b
Add Healthchecks monitoring hook "ping_body_limit" option to configure how many bytes of logs to send to the Healthchecks server ( #294 ).
2022-05-24 12:23:38 -07:00
Dan Helfman
02781662f8
Change monitoring hooks to specify the ping URL / integration key as a named option.
2022-05-23 20:02:10 -07:00
Dan Helfman
32a1043468
Remove the error when "archive_name_format" is specified but a retention prefix isn't ( #402 ).
2022-05-23 16:11:24 -07:00
Dan Helfman
3e4aeec649
Warn when an unsupported variable is used in a hook command ( #420 ).
2022-05-23 15:27:54 -07:00
Dan Helfman
255cc6ec23
When deep merging common configuration, merge colliding list values by appending them ( #531 ).
2022-05-20 15:28:28 -07:00
Dan Helfman
788281cfb9
When a configuration include is a relative path, load it from either the current working directory or from the directory containing the file doing the including ( #532 ).
2022-05-19 17:15:05 -07:00
Dan Helfman
525266ede6
Deep merging when including common configuration ( #381 ).
2022-04-25 21:18:37 -07:00
Dan Helfman
0e6b425ac5
Fix "borgmatic borg key ..." to pass parameters to Borg in correct order ( #515 ).
2022-04-23 14:03:15 -07:00
Dan Helfman
ed7fe5c6d0
Instead of executing "before" command hooks before all borgmatic actions run (and "after" hooks after), execute these hooks right before/after the corresponding action ( #473 ).
2022-04-21 22:08:25 -07:00
Dan Helfman
0c027a3050
Fix handling of TERM signal to exit borgmatic, not just forward the signal to Borg ( #516 ).
2022-04-03 13:12:48 -07:00
Dan Helfman
9f44bbad65
Fix borgmatic exit code (so it's zero) when initial Borg calls fail but later retries succeed ( #517 ).
2022-04-02 22:28:41 -07:00
Dan Helfman
976516d0e1
When loading a configuration file that is unreadable due to file permissions, warn instead of erroring ( #444 ).
2022-03-08 10:19:36 -08:00
Dan Helfman
574eb91921
Fix Borg usage error in the "compact" action when running "borgmatic --dry-run". Now, skip "compact" entirely during a dry run ( #507 ).
2022-03-07 21:46:12 -08:00
Dan Helfman
28fef3264b
Fix handling of "patterns_from" and "exclude_from" options to error instead of warning when referencing unreadable files and running "create" action ( #486 ).
2022-03-07 15:32:07 -08:00
Dan Helfman
9161dbcb7d
Removing unnecessary leading underscores from functions.
2022-03-07 11:58:29 -08:00
Dan Helfman
4b3027e4fc
Add test for new working_directory option ( #431 ).
2022-03-03 11:48:18 -08:00
Dan Helfman
0eb2634f9b
Working directory option to support source directories with relative paths ( #431 ).
...
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/477
2022-03-03 19:28:17 +00:00
Dan Helfman
9317cbaaf0
Code formatting.
2022-02-10 10:23:34 -08:00
Dan Helfman
1b5f04b79f
When using the "remote_rate_limit" option, tailor the flags passed to Borg depending on the Borg version ( #394 ).
2022-02-10 10:16:09 -08:00
Dan Helfman
948c86f62c
When using the "numeric_owner" option with the "extract" action, tailor the flags passed to Borg depending on the Borg version ( #394 ).
2022-02-10 10:09:18 -08:00
Dan Helfman
7e7209322a
When using the "numeric_owner" option, tailor the flags passed to Borg depending on the Borg version ( #394 ).
2022-02-10 09:51:13 -08:00
Dan Helfman
00a57fd947
Code formatting.
2022-02-09 21:20:28 -08:00
Dan Helfman
6bf6ac310b
When using the "bsd_flags" option, tailor the flags passed to Borg depending on the Borg version ( #394 ).
2022-02-09 21:11:00 -08:00
Dan Helfman
4b5af2770d
When the "atime" option is used, tailor the flags passed to Borg depending on version ( #394 ).
2022-02-09 16:54:35 -08:00
Dan Helfman
b525e70e1c
Run "compact" action by default when no actions are specified ( #394 ).
2022-02-09 14:33:12 -08:00
Dan Helfman
e3f4bf0293
Build fix for ruamel.yaml.clib error.
2022-02-08 12:52:45 -08:00
Dan Helfman
48f44d2f3d
Add tests for compact action ( #394 ).
2022-02-08 12:05:02 -08:00
Dan Helfman
bec73245e9
Fix traceback when a YAML validation error occurs ( #480 , #482 ).
2022-01-19 20:39:03 -08:00
fabianschilling
b39f08694d
Merge branch 'master' into pr-working-directory
2022-01-05 09:30:27 +00:00
Dan Helfman
2ee75546f5
Add MongoDB database hook documentation.
2022-01-04 16:26:38 -08:00
Andrea Ghensi
7c6ce9399c
fix integration tests and mongodb auth
2021-12-29 22:18:50 +01:00
Andrea Ghensi
6b7653484b
Add mongodb dump hook
2021-12-26 01:00:58 +01:00
Fabian Schilling
85e0334826
Add missing working_directory arg to pass tests
2021-12-10 18:24:41 +01:00
Dan Helfman
a1673d1fa1
Fix unicode error when restoring particular MySQL databases ( #476 ).
2021-12-08 16:40:25 -08:00
Dan Helfman
7a086d8430
Fix import ordering.
2021-11-29 14:00:14 -08:00
Dan Helfman
0e8e9ced64
When command-line configuration override produces a parse error, error cleanly ( #471 ).
2021-11-29 12:49:21 -08:00
nebulon42
c3e76585fc
move mysqldump options to the beginning of the command due to MySQL bug 30994.
2021-11-26 17:16:03 +01:00
nebulon42
3729ba5ca3
add list_options setting, fixes #306
2021-11-20 15:43:58 +01:00
Dan Helfman
38ebfd2969
Rename retry_timeout to retry_wait and standardize log formatting ( #28 ).
2021-11-15 11:51:17 -08:00
cadamswaite
976a877a25
Formatting
2021-11-14 22:37:42 +00:00
cadamswaite
b4117916b8
Add timeout and tests
2021-11-14 22:15:22 +00:00
cadamswaite
19cad89978
Add some tests for retry logic
2021-11-14 21:35:23 +00:00
Dan Helfman
449896f661
Fix error when configured source directories are not present on the filesystem at the time of backup ( #387 ).
2021-10-11 10:40:10 -07:00
Dan Helfman
acb2ca79d9
Fix traceback that can occur when dumping a database ( #440 ).
2021-08-06 08:58:11 -07:00
Dan Helfman
80b33fbf8a
Code style reformatting.
2021-07-27 09:39:48 -07:00
Marek Szuba
e8b8d86592
tests/integration/test_execute: use plain Python rather than xxd
...
Removes this test's dependencies on vim and /dev/urandom.
Signed-off-by: Marek Szuba <marek.szuba@cern.ch>
2021-07-27 13:50:16 +01:00
Dan Helfman
2a8692c64f
Fix integration test to hopefully work on Alpine ( #430 ).
2021-07-25 22:50:00 -07:00
Dan Helfman
1709f57ff0
Fix hang when restoring a PostgreSQL "tar" format database dump ( #430 ).
2021-07-25 22:30:15 -07:00
Dan Helfman
54d57e1349
Add test for #407 : Fix syslog logging on FreeBSD.
2021-06-23 10:21:45 -07:00
Dan Helfman
27d37b606b
Better error messages! Switch the library used for validating configuration files (from pykwalify to jsonschema).
2021-06-22 13:27:59 -07:00
Dan Helfman
cf8882f2bc
Run arbitrary Borg commands with new "borgmatic borg" action ( #425 ).
2021-06-17 20:41:44 -07:00
Dan Helfman
9fd28d2eed
Fix error handling to error loudly when Borg gets killed due to running out of memory ( #423 )!
2021-06-08 11:43:55 -07:00
Dan Helfman
d6e376d32d
Fix end-to-end test broken by change in source directory examples.
2021-04-18 17:54:54 -07:00
Dan Helfman
d543109ef4
"Fix" build failure with Alpine Edge by switching from Edge to Alpine 3.13.
2021-04-09 15:58:23 -07:00
Dan Helfman
4897a78fd3
Fix database tests broken by PostgreSQL upgrade in Alpine Edge.
2020-12-24 22:23:09 -08:00
Dan Helfman
ad7198ba66
Tweak to test failing on some machines.
2020-11-26 16:22:42 -08:00
Dan Helfman
c0ae01f5d5
Code formatting.
2020-11-25 17:46:57 -08:00
Dan Helfman
ccd1627175
Fix timing-related test error in Alpine Edge.
2020-11-25 15:48:33 -08:00
Dan Helfman
1f4f28b4dc
Drop support for Python 3.5. Only support black code formatter on Python 3.8+.
2020-11-22 17:27:21 -08:00
Dan Helfman
ea6cd53067
Update versions of test dependencies (test_requirements.txt and test containers).
2020-11-22 14:48:07 -08:00
Dan Helfman
9b819f32f8
Fix traceback when upgrading old INI-style configuration with upgrade-borgmatic-config ( #367 ).
2020-11-18 15:01:10 -08:00
Dan Helfman
b619bde037
Fix broken end-to-end test.
2020-11-18 14:20:07 -08:00
François Poulain
ebac02f118
Mention placeholders for repositories in documentation; fix #369
2020-11-18 20:22:41 +01:00
Dan Helfman
1ea04aedf0
Merge override values when specifying the "--override" flag multiple times ( #361 ).
2020-10-23 16:06:00 -07:00
Dan Helfman
0e978299cf
Fix traceback when a database hook value is null in a configuration file ( #355 ).
2020-10-07 15:41:45 -07:00
networkjanitor
3c723e8d99
Merge branch 'master' into master
2020-08-12 21:11:45 +00:00
Dan Helfman
72bd96c656
Fix traceback when a configuration directory is non-readable due to directory permissions ( #350 ).
2020-08-12 11:32:00 -07:00
networkjanitor
893fca2816
Add before_extract and after_extract hooks
2020-08-01 16:08:32 +02:00
Dan Helfman
b3fd1be5f6
Add "borgmatic export-tar" action to export an archive to a tar-formatted file or stream ( #300 ).
2020-07-22 21:53:55 -07:00
Dan Helfman
6f82c9979b
Add #339 to NEWS and add test.
2020-07-07 22:31:17 -07:00
Dan Helfman
37cc229749
Fix duplicate logging to Healthchecks and send "after_*" hooks output to Healthchecks ( #328 ).
2020-06-23 11:01:03 -07:00
Dan Helfman
17c2d109e5
Add tests for pass-through of BORG_* environment variables.
2020-06-21 14:41:22 -07:00
Dan Helfman
32e15dc905
Add a few more mocks to PostgreSQL SSL tests.
2020-06-20 14:39:16 -07:00
Dan Helfman
f5ebca4907
Add SSL support to PostgreSQL database configuration ( #331 ).
...
Reviewed-on: https://projects.torsion.org/witten/borgmatic/pulls/331
2020-06-20 21:24:14 +00:00
Edward Shornock
d2d92b1f1a
Add tests for the PostgreSQL SSL options
2020-06-20 23:32:24 +03:00
Dan Helfman
27cbe9dfc0
Fix for potential data loss (data not getting backed up) when borgmatic omitted configured source directories in certain situations ( #333 ).
2020-06-19 20:16:38 -07:00
Edward Shornock
abd47fc14e
Add SSL support to PostgreSQL hooks
2020-06-19 02:19:17 +03:00
Dan Helfman
7fb4061759
Improve configuration reference documentation readability via more aggressive word-wrapping in configuration schema descriptions.
2020-06-17 23:15:12 -07:00
Dan Helfman
aa14449857
Add "borgmatic extract --strip-components" flag to remove leading path components when extracting an archive ( #324 ).
2020-06-06 14:57:14 -07:00
Dan Helfman
a155eefa23
Fix for certain configuration options like ssh_command impacting Borg invocations for separate configuration files ( #323 ).
2020-06-06 14:30:04 -07:00
Dan Helfman
d7277893fb
Fix hang when a stale database dump named pipe from an aborted borgmatic run remains on disk ( #316 ).
2020-06-02 12:40:32 -07:00
Dan Helfman
097a09578a
Fix enabled database hooks to implicitly set one_file_system configuration option to true to prevent Borg hang. ( #315 ).
2020-05-26 09:20:14 -07:00
Dan Helfman
65472c8de2
Fix error message when there are no MySQL databases to dump for "all" databases ( #319 ).
2020-05-26 08:59:04 -07:00
Dan Helfman
96df52ec50
Fix hang when streaming a database dump to Borg with implicit duplicate source directories by deduplicating them first ( #316 ).
2020-05-20 13:33:53 -07:00
Dan Helfman
89cb5eb76d
Fix regression in support for PostgreSQL's "directory" dump format ( #314 ).
2020-05-18 11:31:29 -07:00
Dan Helfman
048a9ebb52
Add an additional end-to-end database test.
2020-05-15 10:12:49 -07:00
Dan Helfman
3e5a19d95a
Add missing test coverage.
2020-05-14 23:21:43 -07:00
Dan Helfman
d88f321cef
Fix legitimate database dump command errors (exit code 1) not being treated as errors by borgmatic ( #310 ).
2020-05-14 22:38:38 -07:00
Dan Helfman
15ea70a71b
Flesh out missing test coverage for execute.py.
2020-05-12 15:49:49 -07:00
Dan Helfman
8b91c01a4c
Add some missing test coverage.
2020-05-11 11:34:14 -07:00
Dan Helfman
3bcef72050
Add some missing test mocks that were masking lack of unit coverage.
2020-05-11 11:17:24 -07:00
Dan Helfman
695c764a01
Merge log output functions into one.
2020-05-11 10:55:50 -07:00
Dan Helfman
1ea047dd94
Remove "borgmatic restore --progress" flag, as it now conflicts with streaming database restores.
2020-05-09 21:53:16 -07:00
Dan Helfman
4b523f9e2c
Make database restore output only show at verbosity 2.
2020-05-08 19:38:33 -07:00
Dan Helfman
6a61070d85
Use shell redirection rather than the --file flag to sidestep synchronization issues when pg_dump/pg_dumpall tries to write to a named pipe.
2020-05-08 19:11:26 -07:00
Dan Helfman
f36082938e
Additional test coverage.
2020-05-08 09:48:04 -07:00
Dan Helfman
1ba996ad93
Additional test coverage.
2020-05-07 12:14:27 -07:00
Dan Helfman
a23fdf946d
Stream database dumps and restores directly to/from Borg without using any additional filesystem space ( #258 ).
2020-05-07 11:44:04 -07:00
Dan Helfman
12cf6913ef
Remove unused function parameter.
2020-05-03 14:07:34 -07:00
Dan Helfman
e4eff0e3dc
Bump version for release.
2020-04-24 15:56:56 -07:00
Dan Helfman
dce1928dc4
Fix PostgreSQL restore error on "all" database dump.
2020-04-24 15:50:33 -07:00
Nathan Beals
3c8dc4929f
Added test_restore_all_database_dump
unit test.
...
Updated the other unit tests, as I had to re-arrange argument order
Added an 'all' test for the postgres end-to-end test.
Ran black formatter on it all.
2020-04-24 18:32:53 -04:00
Dan Helfman
e511014a28
Fix MySQL restore error on "all" database dump by excluding system tables ( #301 ).
2020-04-22 12:17:22 -07:00
Dan Helfman
bae5f88824
Upgrade test database versions.
2020-04-21 10:01:26 -07:00
Dan Helfman
55141bda67
Specify "--archive latest" to all actions that accept an archive ( #289 ).
2020-01-29 16:59:02 -08:00
Dan Helfman
bc02c123e6
Monitor backups with PagerDuty hook integration ( #245 ).
2020-01-27 15:32:09 -08:00
Dan Helfman
e76d5ad988
Fix tests.
2020-01-27 12:56:12 -08:00
Dan Helfman
2405e97c38
Backup to a removable drive or intermittent server via "soft failure" feature ( #284 ).
2020-01-24 20:52:48 -08:00
Dan Helfman
fdbb2ee905
View consistency check progress via "--progress" flag for "check" action ( #287 ).
2020-01-24 11:27:16 -08:00
Dan Helfman
94b9ef56be
Change "exclude_if_present" option to support multiple filenames, rather than just a single filename ( #280 ).
2020-01-23 13:41:37 -08:00
Dan Helfman
53e6ff9524
No longer list files or show stats by default at verbosity 2.
2020-01-22 15:23:49 -08:00
Dan Helfman
f66fd1caaa
Customize Healthchecks log level via borgmatic "--monitoring-verbosity" flag ( #277 ).
2020-01-22 15:10:47 -08:00
Dan Helfman
d93fdbc5ad
Support "--files" and "--stats" flags at verbosity level 0.
2020-01-22 13:28:24 -08:00
palto42
75b5e7254e
changes as per comments in PR #283
2020-01-22 19:03:26 +01:00
palto42
5f0c084bee
Merge 'upstream/master' into list-files
2020-01-22 18:12:26 +01:00
Dan Helfman
7824a034ca
Add test for database dump directory removal.
2020-01-21 10:34:46 -08:00
palto42
e27ba0d08a
less detail at v1 + option "--files" for details
2020-01-11 16:38:07 +01:00
Dan Helfman
d0c533555e
In "borgmatic --help", don't expand $HOME in listing of default "--config" paths.
2020-01-02 10:37:31 -08:00
Dan Helfman
911668f0c8
Only trigger "on_error" hooks and monitoring failures for "prune", "create", and "check" actions, and not for other actions ( #270 ).
2019-12-20 13:58:02 -08:00
Dan Helfman
d64bcd5e83
When pruning with verbosity level 1, list pruned and kept archives.
2019-12-17 20:12:41 -08:00
Dan Helfman
f787dfe809
Override particular configuration options from the command-line via "--override" flag ( #268 ).
2019-12-17 11:46:27 -08:00
Dan Helfman
e009bfeaa2
Update Healthchecks/Cronitor/Cronhub monitoring integrations to fire for "check" and "prune" actions, not just "create" ( #249 ).
2019-12-12 22:54:45 -08:00
Dan Helfman
c6cb21a748
Switch to read-only container filesystem to avoid *.pyc getting created with busted permissions.
2019-12-11 21:24:37 -08:00
Dan Helfman
464ff2fe96
Run end-to-end tests on developer machines with Docker Compose for approximate parity with continuous integration tests.
2019-12-11 16:43:01 -08:00
Dan Helfman
0cc711173a
Merge branch 'master' into end-to-end-database-tests
2019-12-11 12:27:14 -08:00
Dan Helfman
14e5cfc8f8
Support piping "borgmatic list" output to grep. Retain colored output when piping/redirecting ( #271 ).
2019-12-11 12:12:25 -08:00
Dan Helfman
68281339b7
Black.
2019-12-10 16:57:12 -08:00
Dan Helfman
01e2cf08d1
Fix Drone CI services syntax.
2019-12-10 16:43:43 -08:00
Dan Helfman
9f821862b7
End-to-end tests for database dump and restore.
2019-12-10 16:41:01 -08:00
Dan Helfman
8660af745e
Optionally change the internal database dump path via "borgmatic_source_directory" option in location configuration section ( #259 ).
2019-12-10 16:04:34 -08:00
Dan Helfman
826e4352d1
Filter listed paths via "borgmatic list --path" flag ( #269 ).
2019-12-08 14:07:02 -08:00
Dan Helfman
b94999bba4
Fix "borgmatic umount" so it only runs Borg once instead of once per repository / configuration file.
2019-12-07 21:36:51 -08:00
Dan Helfman
65cc4c9429
Fix "--repository" flag to accept relative paths.
2019-12-06 16:29:41 -08:00
Dan Helfman
df2be9620b
Mount whole repositories via "borgmatic mount" without any "--archive" flag ( #253 ).
2019-12-06 15:58:54 -08:00
Dan Helfman
2ab9daaa0f
Attempt to repair any inconsistencies found during a consistency check via "borgmatic check --repair" flag ( #266 ).
2019-12-04 16:07:00 -08:00
Dan Helfman
0c6c61a272
Pass extra options directly to particular Borg commands, handy for Borg options that borgmatic does not yet support natively ( #235 ).
2019-12-04 15:48:10 -08:00
Dan Helfman
00f62ca023
Fix for "before_backup" hook not triggering an error when the command contains "borg" and has an exit code of 1 ( #256 ).
2019-11-30 16:55:05 -08:00
Dan Helfman
9b2ca15de6
Fix for garbled Borg file listing when using "borgmatic create --progress" with verbosity level 1 or 2 ( #257 ).
2019-11-30 15:31:36 -08:00
Dan Helfman
c4aa34bf5c
Fix for missing Healthchecks monitoring payload or HTTP 500 due to incorrect unicode encoding ( #260 ).
2019-11-30 14:51:32 -08:00
Dan Helfman
d978a2d190
Fix for database dump removal incorrectly skipping some database dumps.
2019-11-25 15:27:59 -08:00
Dan Helfman
375036e409
Support for mounting an archive as a FUSE filesystem via "borgmatic mount" action, and unmounting via "borgmatic umount" ( #123 ).
2019-11-25 14:56:20 -08:00
Dan Helfman
f4a231420f
Show summary log errors regardless of verbosity. Log the "summary:" header with level based on the contained logs.
2019-11-25 10:31:09 -08:00
Dan Helfman
55ebfdda39
Show full error logs at "--verbosity 0" so you can see command output without upping the verbosity level.
2019-11-19 10:09:25 -08:00
Dan Helfman
2cc743cf47
With "borgmatic check", consider Borg warnings as errors ( #247 ).
2019-11-17 19:06:28 -08:00
Dan Helfman
d99e6d1994
Remove obnoxious log message when Healthchecks hook sends failure for check action.
2019-11-17 18:55:40 -08:00
Dan Helfman
50f62d73b7
When using the Healthchecks monitoring hook, include borgmatic logs in the payloads for completion and failure pings ( #241 ).
2019-11-17 16:54:27 -08:00
Dan Helfman
1f166a47e9
Fix failing test due to dictionary order (or the lack thereof..) in Python 3.5.
2019-11-13 16:42:58 -08:00
Dan Helfman
6cdc92bd0c
Fix for "borgmatic restore" showing success and incorrectly extracting archive files, even when no databases are configured to restore ( #246 ).
2019-11-13 10:41:57 -08:00
Dan Helfman
0a9f4e8708
Reopen the file given by "--log-file" flag if an external program rotates the log file while borgmatic is running.
2019-11-12 16:13:25 -08:00
Dan Helfman
781fac3266
Refactor ping monitors (Healthchecks, Cronitor, Cronhub) to share a common invocation and function signature.
2019-11-12 15:31:07 -08:00
Dan Helfman
04e5b42606
Fix repository does not exist error with "borgmatic extract" when repository is remote ( #243 ).
2019-11-12 11:47:24 -08:00
Dan Helfman
ebeb5efe05
More accurately detecting restore of unconfigured database ( #228 ).
2019-11-12 11:10:47 -08:00
Dan Helfman
a3e939f34b
Factor out filtering of database configuration to function with tests ( #228 ).
2019-11-12 10:39:36 -08:00
Dan Helfman
2a771161e7
Finish test coverage for MySQL restore ( #228 ).
2019-11-12 09:50:26 -08:00
Dan Helfman
ded042d8cc
First crack at MySQL/MariaDB database restore ( #228 ).
2019-11-11 21:59:30 -08:00
Dan Helfman
9d29ecf304
Remove MySQL/MariaDB database dumps after backing them up ( #228 ).
2019-11-08 11:53:27 -08:00
Dan Helfman
427b57e2a9
Database dump hooks for MySQL/MariaDB ( #228 ).
2019-11-08 11:17:52 -08:00
Dan Helfman
17fda7281a
Monitor backups with Cronhub hook integration. Fix Healthchecks/Cronitor hooks to respect dry run.
2019-11-07 10:08:44 -08:00
Dan Helfman
ac777965d0
Fix regression of generate-borgmatic-config working without --source flag ( #239 ).
2019-11-06 09:52:21 -08:00
Dan Helfman
2115eeb6a2
Upgrade your borgmatic configuration to get new options and comments via "generate-borgmatic-config --source" ( #239 ).
2019-11-06 09:31:00 -08:00
palto42
18d3542fbc
new verbosity level "-1" for errors
2019-11-03 09:55:19 +01:00
Dan Helfman
a9104ed090
Handle log file error more consistently with other error. Add --log-file-verbosity flag. Add docs.
2019-11-02 11:23:18 -07:00
Dan Helfman
06f134cc71
Log to file instead of syslog via command-line "--log-file" flag ( #233 ).
2019-11-02 17:43:39 +00:00
palto42
26a1a3d1e0
test cases for logfile option
2019-11-02 14:34:16 +01:00
Dan Helfman
f48f52079d
Support for Borg --keep-exclude-tags and --exclude-nodump options ( #234 ).
2019-11-01 15:50:59 -07:00
Dan Helfman
8fd46b8c70
Monitor backups with Cronitor hook integration.
2019-11-01 11:33:15 -07:00
Dan Helfman
e3dd545345
Extract files to a particular directory via "borgmatic extract --destination" flag. Also rename "--restore-path" to "--path" to reduce confusion.
2019-11-01 10:00:26 -07:00
Dan Helfman
939f4d4e3d
Add additional tests for database restores.
2019-10-31 14:11:19 -07:00
Dan Helfman
3006db0cae
Restore backed up PostgreSQL databases via "borgmatic restore" sub-command ( #229 ).
2019-10-31 12:57:36 -07:00
Dan Helfman
37dc94bc79
Add test for removal of database dumps.
2019-10-23 13:36:03 -07:00
Dan Helfman
fc274b43f0
Rename "borgmatic list --pattern-from" flag to "--patterns-from" to match Borg ( #230 ).
2019-10-22 22:42:36 -07:00
Dan Helfman
9ab12e4312
Tests for database dumping ( #225 ).
2019-10-22 21:39:30 -07:00
Dan Helfman
458e7776c5
Database dump hooks for PostgreSQL, so you can easily dump your databases before backups run ( #225 ).
2019-10-22 16:28:42 -07:00
Dan Helfman
fa5fa1c11b
Move hooks into directory, so there can be one source file per type of hook ( #225 ).
2019-10-21 15:52:14 -07:00
Dan Helfman
f8bc67be8d
Config generation support for sequences of maps, needed for database dump hooks ( #225 ).
2019-10-21 15:17:47 -07:00
Dan Helfman
128ebf04ce
Dead man's switch via healthchecks.io integration ( #223 ) + new monitoring documentation.
2019-10-15 10:49:14 -07:00
Dan Helfman
7b3b28616d
Add "borgmatic list --successful" flag to only list successful (non-checkpoint) archives ( #86 ).
2019-10-13 15:58:11 -07:00
Dan Helfman
1d37b14356
More detailed error alerting via runtime context available in "on_error" hook ( #174 ).
2019-10-01 12:23:16 -07:00
Dan Helfman
6c617eddd5
When backups to one of several repositories fails, keep backing up to the other repositories ( #144 ).
2019-09-30 22:19:31 -07:00
Dan Helfman
e14ebee4e0
User-defined hooks for global setup or cleanup that run before/after all actions. ( #192 ).
2019-09-28 16:18:10 -07:00
Dan Helfman
a897ffd514
Fix "borgmatic create --progress" output so that it updates on the console in real-time ( #221 ).
2019-09-25 12:03:10 -07:00
Dan Helfman
07222cd984
Fix visibility of "borgmatic prune --stats" output ( #219 ).
2019-09-23 13:07:51 -07:00
Dan Helfman
340bd72176
Fix regression of argument parsing for default actions ( #220 ).
2019-09-22 11:30:58 -07:00
Dan Helfman
1a1bb71af1
Fix error with "borgmatic check --only" command-line flag with "extract" consistency check ( #217 ).
2019-09-20 11:43:27 -07:00
Dan Helfman
81739791e0
Override configured consistency checks via "borgmatic check --only" command-line flag ( #210 ).
2019-09-19 11:43:53 -07:00
Dan Helfman
4cdff74e9b
Support for Borg check --verify-data flag via borgmatic "data" consistency check ( #210 ).
2019-09-18 16:52:27 -07:00
Dan Helfman
11e830bb1d
Fix flake8 warning.
2019-09-18 14:11:56 -07:00