psw/borgmatic
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1364 commits 2 branches 0 tags 3.6 MiB
Commit graph

461 commits

Author SHA1 Message Date
root
7cad5a8608 Add temporary_directory as a configuration option defaulting to $TMPDIR 2020-08-26 15:29:02 +00:00
palto42
631c3068a9 systemd security settings 2020-08-22 15:41:25 +02:00
networkjanitor
3c723e8d99 Merge branch 'master' into master 2020-08-12 21:11:45 +00:00
networkjanitor
c5776447b9 fixed description strings for before_extract and after_extract 2020-08-12 23:07:57 +02:00
Dan Helfman
5356f487a5 Move before/after_everything hooks so they're easier to find. 2020-08-12 13:46:31 -07:00
Dan Helfman
72bd96c656 Fix traceback when a configuration directory is non-readable due to directory permissions (#350). 2020-08-12 11:32:00 -07:00
Jakub Duchateau
95c415f416
Change "before" to "after" in desc 2020-08-07 16:41:35 +02:00
Dan Helfman
06dc336481 Fix schema exclude patterns example path. 2020-08-06 09:59:48 -07:00
networkjanitor
893fca2816 Add before_extract and after_extract hooks 2020-08-01 16:08:32 +02:00
Dan Helfman
b3fd1be5f6 Add "borgmatic export-tar" action to export an archive to a tar-formatted file or stream (#300). 2020-07-22 21:53:55 -07:00
Dan Helfman
a23083f737 Clarify schema documentation about skipping pruning. 2020-07-22 10:35:16 -07:00
Dan Helfman
0a659a397f Attempt to fix no-repro test failure of log_outputs() (#339). 2020-07-07 22:05:53 -07:00
Dan Helfman
3aaa89fb08 Formatting. 2020-06-25 20:25:29 -07:00
Dan Helfman
35d542a676 Fix for traceback when running Cronitor, Cronhub, and PagerDuty monitor hooks (#336). 2020-06-25 20:23:25 -07:00
Dan Helfman
37cc229749 Fix duplicate logging to Healthchecks and send "after_*" hooks output to Healthchecks (#328). 2020-06-23 11:01:03 -07:00
Dan Helfman
17c2d109e5 Add tests for pass-through of BORG_* environment variables. 2020-06-21 14:41:22 -07:00
Dan Helfman
c8d5de2179 Fix broken pass-through of BORG_* environment variables to Borg (#327). 
Reviewed-on: https://projects.torsion.org/witten/borgmatic/pulls/327
 2020-06-21 21:29:59 +00:00
Dan Helfman
f5ebca4907 Add SSL support to PostgreSQL database configuration (#331). 
Reviewed-on: https://projects.torsion.org/witten/borgmatic/pulls/331
 2020-06-20 21:24:14 +00:00
Edward Shornock
01db676d68 Change the example for the ssl_mode parameter 2020-06-20 23:32:24 +03:00
Dan Helfman
27cbe9dfc0 Fix for potential data loss (data not getting backed up) when borgmatic omitted configured source directories in certain situations (#333). 2020-06-19 20:16:38 -07:00
Edward Shornock
8fb830099f Re-add the ilbpq-ssl documentation URL to the schema 
It's been moved from describing `ssl_mode` to the general
postgresql_database description key.
 2020-06-19 13:22:39 +03:00
Edward Shornock
463a133a63 Ensure schema lines are less than 80 characters in length 2020-06-19 13:22:39 +03:00
Edward Shornock
a16fed8887 Rename PostgreSQL SSL config variables 
e.g. s/sslmode/ssl_mode/g to conform with borgmatic naming conventions.
 2020-06-19 13:20:14 +03:00
Edward Shornock
33113890f5 Reduce duplication with a common function 2020-06-19 12:32:36 +03:00
Edward Shornock
abd47fc14e Add SSL support to PostgreSQL hooks 2020-06-19 02:19:17 +03:00
Dan Helfman
7fb4061759 Improve configuration reference documentation readability via more aggressive word-wrapping in configuration schema descriptions. 2020-06-17 23:15:12 -07:00
Ralph Heinkel
a12a1121b6 Use values from BORG_* env variables if they are not specified in config.yaml 2020-06-15 19:50:11 +02:00
Dan Helfman
aa14449857 Add "borgmatic extract --strip-components" flag to remove leading path components when extracting an archive (#324). 2020-06-06 14:57:14 -07:00
Dan Helfman
ed7b1cd3d7 Add some no-cover pragmas on functions that don't need tests. 2020-06-06 14:33:06 -07:00
Dan Helfman
a155eefa23 Fix for certain configuration options like ssh_command impacting Borg invocations for separate configuration files (#323). 2020-06-06 14:30:04 -07:00
Dan Helfman
398665be9e Allow before_backup and similiar hooks to exit with a soft failure without altering the monitoring status (#292). 2020-06-02 14:33:41 -07:00
Dan Helfman
d7277893fb Fix hang when a stale database dump named pipe from an aborted borgmatic run remains on disk (#316). 2020-06-02 12:40:32 -07:00
Dan Helfman
00033bf0a8 Tweak comment indentation in generated configuration file for clarity. 2020-06-02 11:37:13 -07:00
Dan Helfman
097a09578a Fix enabled database hooks to implicitly set one_file_system configuration option to true to prevent Borg hang. (#315). 2020-05-26 09:20:14 -07:00
Dan Helfman
65472c8de2 Fix error message when there are no MySQL databases to dump for "all" databases (#319). 2020-05-26 08:59:04 -07:00
Dan Helfman
96df52ec50 Fix hang when streaming a database dump to Borg with implicit duplicate source directories by deduplicating them first (#316). 2020-05-20 13:33:53 -07:00
Dan Helfman
89cb5eb76d Fix regression in support for PostgreSQL's "directory" dump format (#314). 2020-05-18 11:31:29 -07:00
Dan Helfman
2ddf38f99c Fix error handling when executing commands to handle more edge cases. 2020-05-14 23:04:01 -07:00
Dan Helfman
d88f321cef Fix legitimate database dump command errors (exit code 1) not being treated as errors by borgmatic (#310). 2020-05-14 22:38:38 -07:00
Dan Helfman
15ea70a71b Flesh out missing test coverage for execute.py. 2020-05-12 15:49:49 -07:00
Dan Helfman
3bcef72050 Add some missing test mocks that were masking lack of unit coverage. 2020-05-11 11:17:24 -07:00
Dan Helfman
695c764a01 Merge log output functions into one. 2020-05-11 10:55:50 -07:00
Dan Helfman
f7c93ea2e8 Wait for process to finish before trying to check exit status. 2020-05-09 23:09:48 -07:00
Dan Helfman
1ea047dd94 Remove "borgmatic restore --progress" flag, as it now conflicts with streaming database restores. 2020-05-09 21:53:16 -07:00
Dan Helfman
4b523f9e2c Make database restore output only show at verbosity 2. 2020-05-08 19:38:33 -07:00
Dan Helfman
6a61070d85 Use shell redirection rather than the --file flag to sidestep synchronization issues when pg_dump/pg_dumpall tries to write to a named pipe. 2020-05-08 19:11:26 -07:00
Dan Helfman
1ba996ad93 Additional test coverage. 2020-05-07 12:14:27 -07:00
Dan Helfman
a23fdf946d Stream database dumps and restores directly to/from Borg without using any additional filesystem space (#258). 2020-05-07 11:44:04 -07:00
Dan Helfman
12cf6913ef Remove unused function parameter. 2020-05-03 14:07:34 -07:00
Dan Helfman
dce1928dc4
Fix PostgreSQL restore error on "all" database dump. 2020-04-24 15:50:33 -07:00
Nathan Beals
3c8dc4929f Added test_restore_all_database_dump unit test. 
Updated the other unit tests, as I had to re-arrange argument order
Added an 'all' test for the postgres end-to-end test.

Ran black formatter on it all.
 2020-04-24 18:32:53 -04:00
Dan Helfman
e511014a28 Fix MySQL restore error on "all" database dump by excluding system tables (#301). 2020-04-22 12:17:22 -07:00
Nathan Beals
f6407bafcb Remove the --create flag, was causing an error 2020-04-10 11:24:13 -04:00
Nathan Beals
d5e9f67cec Finished. Now uses 'psql' to run the plain-text scripts that pg_dumpall creates 2020-04-10 10:55:53 -04:00
Nathan Beals
b14f371c05 First attempt at fixing this pg_dumpall/restoring issue 2020-04-10 09:20:00 -04:00
Dan Helfman
d6562c4b1e Fix the "--stats" and "--files" flags so that they yield output at verbosity 0 (#290). 2020-01-31 10:23:36 -08:00
Dan Helfman
55141bda67 Specify "--archive latest" to all actions that accept an archive (#289). 2020-01-29 16:59:02 -08:00
Dan Helfman
bc02c123e6 Monitor backups with PagerDuty hook integration (#245). 2020-01-27 15:32:09 -08:00
Dan Helfman
8ad8a9c422 Add per-action hooks: "before_prune", "after_prune", "before_check", and "after_check" (#255). 2020-01-27 11:07:07 -08:00
Dan Helfman
2405e97c38 Backup to a removable drive or intermittent server via "soft failure" feature (#284). 2020-01-24 20:52:48 -08:00
Dan Helfman
fdbb2ee905 View consistency check progress via "--progress" flag for "check" action (#287). 2020-01-24 11:27:16 -08:00
Dan Helfman
94b9ef56be Change "exclude_if_present" option to support multiple filenames, rather than just a single filename (#280). 2020-01-23 13:41:37 -08:00
Dan Helfman
952168ce25 Fix unwanted console log messages with "list --json" and "info --json". 2020-01-23 13:40:54 -08:00
Dan Helfman
5273037a94 For "list" and "info" actions, show repository names even at verbosity 0. 2020-01-23 11:17:39 -08:00
Dan Helfman
53e6ff9524 No longer list files or show stats by default at verbosity 2. 2020-01-22 15:23:49 -08:00
Dan Helfman
f66fd1caaa Customize Healthchecks log level via borgmatic "--monitoring-verbosity" flag (#277). 2020-01-22 15:10:47 -08:00
Dan Helfman
d93fdbc5ad Support "--files" and "--stats" flags at verbosity level 0. 2020-01-22 13:28:24 -08:00
palto42
75b5e7254e changes as per comments in PR #283 2020-01-22 19:03:26 +01:00
palto42
5f0c084bee Merge 'upstream/master' into list-files 2020-01-22 18:12:26 +01:00
Dan Helfman
39550a7fe9 Add ~/.config/borgmatic.d as another configuration directory default (#274). 2020-01-22 09:26:58 -08:00
Ronan Dunklau
8a91c79fb0 Support directory format dump cleanup. 
Previously, only deleting a dump in a single-file format was supported.
This led to errors when performing a PostgreSQL directory format backup.
 2020-01-19 15:15:47 +01:00
palto42
e108526bab disable --stats by default 2020-01-18 14:38:59 +01:00
palto42
e27ba0d08a less detail at v1 + option "--files" for details 2020-01-11 16:38:07 +01:00
Dan Helfman
5afe0e3d63 Disable colored output when "--json" flag is used, so as to produce valid JSON ouput (#276). 2020-01-04 15:50:41 -08:00
Dan Helfman
d0c533555e In "borgmatic --help", don't expand $HOME in listing of default "--config" paths. 2020-01-02 10:37:31 -08:00
Dan Helfman
911668f0c8 Only trigger "on_error" hooks and monitoring failures for "prune", "create", and "check" actions, and not for other actions (#270). 2019-12-20 13:58:02 -08:00
Dan Helfman
d64bcd5e83 When pruning with verbosity level 1, list pruned and kept archives. 2019-12-17 20:12:41 -08:00
Dan Helfman
f787dfe809 Override particular configuration options from the command-line via "--override" flag (#268). 2019-12-17 11:46:27 -08:00
Dan Helfman
e009bfeaa2 Update Healthchecks/Cronitor/Cronhub monitoring integrations to fire for "check" and "prune" actions, not just "create" (#249). 2019-12-12 22:54:45 -08:00
Matthew Daley
dd16504329 Use --remote-path, --debug and --info when checking for repo existence 
These are currently not being used in the call to `borg info` performed
as part of the borgmatic init command to check whether or not the repo
already exists.
 2019-12-13 15:45:12 +13:00
Dan Helfman
14e5cfc8f8 Support piping "borgmatic list" output to grep. Retain colored output when piping/redirecting (#271). 2019-12-11 12:12:25 -08:00
Dan Helfman
8660af745e Optionally change the internal database dump path via "borgmatic_source_directory" option in location configuration section (#259). 2019-12-10 16:04:34 -08:00
Dan Helfman
826e4352d1 Filter listed paths via "borgmatic list --path" flag (#269). 2019-12-08 14:07:02 -08:00
Dan Helfman
b94999bba4 Fix "borgmatic umount" so it only runs Borg once instead of once per repository / configuration file. 2019-12-07 21:36:51 -08:00
Dan Helfman
65cc4c9429 Fix "--repository" flag to accept relative paths. 2019-12-06 16:29:41 -08:00
Dan Helfman
df2be9620b Mount whole repositories via "borgmatic mount" without any "--archive" flag (#253). 2019-12-06 15:58:54 -08:00
Dan Helfman
2ab9daaa0f Attempt to repair any inconsistencies found during a consistency check via "borgmatic check --repair" flag (#266). 2019-12-04 16:07:00 -08:00
Dan Helfman
0c6c61a272 Pass extra options directly to particular Borg commands, handy for Borg options that borgmatic does not yet support natively (#235). 2019-12-04 15:48:10 -08:00
Dan Helfman
00f62ca023 Fix for "before_backup" hook not triggering an error when the command contains "borg" and has an exit code of 1 (#256). 2019-11-30 16:55:05 -08:00
Dan Helfman
9b2ca15de6 Fix for garbled Borg file listing when using "borgmatic create --progress" with verbosity level 1 or 2 (#257). 2019-11-30 15:31:36 -08:00
Dan Helfman
c4aa34bf5c Fix for missing Healthchecks monitoring payload or HTTP 500 due to incorrect unicode encoding (#260). 2019-11-30 14:51:32 -08:00
Dan Helfman
d978a2d190 Fix for database dump removal incorrectly skipping some database dumps. 2019-11-25 15:27:59 -08:00
Dan Helfman
375036e409 Support for mounting an archive as a FUSE filesystem via "borgmatic mount" action, and unmounting via "borgmatic umount" (#123). 2019-11-25 14:56:20 -08:00
Dan Helfman
f4a231420f Show summary log errors regardless of verbosity. Log the "summary:" header with level based on the contained logs. 2019-11-25 10:31:09 -08:00
Dan Helfman
55ebfdda39 Show full error logs at "--verbosity 0" so you can see command output without upping the verbosity level. 2019-11-19 10:09:25 -08:00
Dan Helfman
e63e2e0852 Rephrasing verbosity levels. 2019-11-17 22:52:26 -08:00
Dan Helfman
edc4b9e60e Update the command-line help description. 2019-11-17 22:49:16 -08:00
Dan Helfman
2cc743cf47 With "borgmatic check", consider Borg warnings as errors (#247). 2019-11-17 19:06:28 -08:00
Dan Helfman
d99e6d1994 Remove obnoxious log message when Healthchecks hook sends failure for check action. 2019-11-17 18:55:40 -08:00
Dan Helfman
50f62d73b7 When using the Healthchecks monitoring hook, include borgmatic logs in the payloads for completion and failure pings (#241). 2019-11-17 16:54:27 -08:00
Dan Helfman
6cdc92bd0c Fix for "borgmatic restore" showing success and incorrectly extracting archive files, even when no databases are configured to restore (#246). 2019-11-13 10:41:57 -08:00
Dan Helfman
0a9f4e8708 Reopen the file given by "--log-file" flag if an external program rotates the log file while borgmatic is running. 2019-11-12 16:13:25 -08:00
Dan Helfman
781fac3266 Refactor ping monitors (Healthchecks, Cronitor, Cronhub) to share a common invocation and function signature. 2019-11-12 15:31:07 -08:00
Dan Helfman
04e5b42606 Fix repository does not exist error with "borgmatic extract" when repository is remote (#243). 2019-11-12 11:47:24 -08:00
Dan Helfman
30525c43bf Another edge case: Don't error on non-matching restore path globs (#228). 2019-11-12 11:24:31 -08:00
Dan Helfman
ebeb5efe05 More accurately detecting restore of unconfigured database (#228). 2019-11-12 11:10:47 -08:00
Dan Helfman
a3e939f34b Factor out filtering of database configuration to function with tests (#228). 2019-11-12 10:39:36 -08:00
Dan Helfman
2a771161e7 Finish test coverage for MySQL restore (#228). 2019-11-12 09:50:26 -08:00
Dan Helfman
ded042d8cc First crack at MySQL/MariaDB database restore (#228). 2019-11-11 21:59:30 -08:00
Dan Helfman
9d29ecf304 Remove MySQL/MariaDB database dumps after backing them up (#228). 2019-11-08 11:53:27 -08:00
Dan Helfman
427b57e2a9 Database dump hooks for MySQL/MariaDB (#228). 2019-11-08 11:17:52 -08:00
Dan Helfman
17fda7281a Monitor backups with Cronhub hook integration. Fix Healthchecks/Cronitor hooks to respect dry run. 2019-11-07 10:08:44 -08:00
Dan Helfman
ac777965d0 Fix regression of generate-borgmatic-config working without --source flag (#239). 2019-11-06 09:52:21 -08:00
Dan Helfman
31d3bc9bd8 In documentation, clarify when Healthchecks/Cronitor hooks fire in relation to other hooks (#238). 2019-11-06 09:44:46 -08:00
Dan Helfman
2115eeb6a2 Upgrade your borgmatic configuration to get new options and comments via "generate-borgmatic-config --source" (#239). 2019-11-06 09:31:00 -08:00
palto42
18d3542fbc new verbosity level "-1" for errors 2019-11-03 09:55:19 +01:00
Dan Helfman
a9104ed090 Handle log file error more consistently with other error. Add --log-file-verbosity flag. Add docs. 2019-11-02 11:23:18 -07:00
Dan Helfman
06f134cc71 Log to file instead of syslog via command-line "--log-file" flag (#233). 2019-11-02 17:43:39 +00:00
palto42
6da05cbe2d Exception handling for logfile option 2019-11-02 14:33:57 +01:00
Dan Helfman
f48f52079d Support for Borg --keep-exclude-tags and --exclude-nodump options (#234). 2019-11-01 15:50:59 -07:00
palto42
b121290c0f Merge remote-tracking branch 'upstream/master' into logging 2019-11-01 19:44:04 +01:00
Dan Helfman
8fd46b8c70 Monitor backups with Cronitor hook integration. 2019-11-01 11:33:15 -07:00
Dan Helfman
603f525352 Clarify --archive help. 2019-11-01 10:50:20 -07:00
palto42
8c8640d0ab file-logger replaces syslog 2019-11-01 18:42:24 +01:00
Dan Helfman
e3dd545345 Extract files to a particular directory via "borgmatic extract --destination" flag. Also rename "--restore-path" to "--path" to reduce confusion. 2019-11-01 10:00:26 -07:00
Dan Helfman
939f4d4e3d Add additional tests for database restores. 2019-10-31 14:11:19 -07:00
Dan Helfman
3006db0cae Restore backed up PostgreSQL databases via "borgmatic restore" sub-command (#229). 2019-10-31 12:57:36 -07:00
palto42
22640a9ca0 new option for log-file 2019-10-31 10:44:22 +01:00
Dan Helfman
4dc4fe0b8d Reordering hooks a bit so that pre-backup ordering mirrors post-backup ordering. 2019-10-27 14:47:55 -07:00
Dan Helfman
fc274b43f0 Rename "borgmatic list --pattern-from" flag to "--patterns-from" to match Borg (#230). 2019-10-22 22:42:36 -07:00
Dan Helfman
458e7776c5 Database dump hooks for PostgreSQL, so you can easily dump your databases before backups run (#225). 2019-10-22 16:28:42 -07:00
Dan Helfman
fa5fa1c11b Move hooks into directory, so there can be one source file per type of hook (#225). 2019-10-21 15:52:14 -07:00
Dan Helfman
f8bc67be8d Config generation support for sequences of maps, needed for database dump hooks (#225). 2019-10-21 15:17:47 -07:00
Dan Helfman
c12c47cace Fix "borgmatic list --successful" with a slightly better heuristic for listing successful (non-checkpoint) archives. 2019-10-16 10:24:58 -07:00
Dan Helfman
128ebf04ce Dead man's switch via healthchecks.io integration (#223) + new monitoring documentation. 2019-10-15 10:49:14 -07:00
Dan Helfman
7b3b28616d Add "borgmatic list --successful" flag to only list successful (non-checkpoint) archives (#86). 2019-10-13 15:58:11 -07:00
Dan Helfman
f3910f49ca Fix incorrect help on borg list --last flag. 2019-10-13 14:46:28 -07:00
Dan Helfman
1d37b14356 More detailed error alerting via runtime context available in "on_error" hook (#174). 2019-10-01 12:23:16 -07:00
Dan Helfman
6c617eddd5 When backups to one of several repositories fails, keep backing up to the other repositories (#144). 2019-09-30 22:19:31 -07:00
Dan Helfman
e14ebee4e0 User-defined hooks for global setup or cleanup that run before/after all actions. (#192). 2019-09-28 16:18:10 -07:00
Dan Helfman
a897ffd514 Fix "borgmatic create --progress" output so that it updates on the console in real-time (#221). 2019-09-25 12:03:10 -07:00
Dan Helfman
07222cd984 Fix visibility of "borgmatic prune --stats" output (#219). 2019-09-23 13:07:51 -07:00
Dan Helfman
340bd72176 Fix regression of argument parsing for default actions (#220). 2019-09-22 11:30:58 -07:00
Dan Helfman
1a1bb71af1 Fix error with "borgmatic check --only" command-line flag with "extract" consistency check (#217). 2019-09-20 11:43:27 -07:00
Dan Helfman
ae45dfe63a Clarify command-like help for check --only. 2019-09-19 15:20:05 -07:00
Dan Helfman
81739791e0 Override configured consistency checks via "borgmatic check --only" command-line flag (#210). 2019-09-19 11:43:53 -07:00
Dan Helfman
4cdff74e9b Support for Borg check --verify-data flag via borgmatic "data" consistency check (#210). 2019-09-18 16:52:27 -07:00
Dan Helfman
f2198de151 Merge branch 'comments-white-space' of polyzen/borgmatic into master 2019-09-18 21:03:56 +00:00
Daniel M. Capella
0c439c0c02
Add space to separate comments from tokens 
https://yaml.org/spec/1.2/spec.html#id2780069
 2019-09-17 20:00:58 -04:00
Dan Helfman
f11a9bb4aa Revert "Fix for spurious Borg traceback when initializing a repository in an empty directory (#201)." 
This reverts commit 9585c8f908.
 2019-09-14 16:14:20 -07:00