Fix enabled database hooks to implicitly set one_file_system configuration option to true to prevent Borg hang. (#315).

This commit is contained in:
Dan Helfman 2020-05-26 09:20:14 -07:00
parent 65472c8de2
commit 097a09578a
5 changed files with 27 additions and 7 deletions

2
NEWS
View file

@ -1,6 +1,8 @@
1.5.5.dev0 1.5.5.dev0
* #314: Fix regression in support for PostgreSQL's "directory" dump format. Unlike other dump * #314: Fix regression in support for PostgreSQL's "directory" dump format. Unlike other dump
formats, the "directory" dump format does not stream directly to/from Borg. formats, the "directory" dump format does not stream directly to/from Borg.
* #315: Fix enabled database hooks to implicitly set one_file_system configuration option to true.
This prevents Borg from reading devices like /dev/zero and hanging.
* #316: Fix hang when streaming a database dump to Borg with implicit duplicate source directories * #316: Fix hang when streaming a database dump to Borg with implicit duplicate source directories
by deduplicating them first. by deduplicating them first.
* #319: Fix error message when there are no MySQL databases to dump for "all" databases. * #319: Fix error message when there are no MySQL databases to dump for "all" databases.

View file

@ -207,7 +207,11 @@ def create_archive(
+ (('--chunker-params', chunker_params) if chunker_params else ()) + (('--chunker-params', chunker_params) if chunker_params else ())
+ (('--compression', compression) if compression else ()) + (('--compression', compression) if compression else ())
+ (('--remote-ratelimit', str(remote_rate_limit)) if remote_rate_limit else ()) + (('--remote-ratelimit', str(remote_rate_limit)) if remote_rate_limit else ())
+ (('--one-file-system',) if location_config.get('one_file_system') else ()) + (
('--one-file-system',)
if location_config.get('one_file_system') or stream_processes
else ()
)
+ (('--numeric-owner',) if location_config.get('numeric_owner') else ()) + (('--numeric-owner',) if location_config.get('numeric_owner') else ())
+ (('--noatime',) if location_config.get('atime') is False else ()) + (('--noatime',) if location_config.get('atime') is False else ())
+ (('--noctime',) if location_config.get('ctime') is False else ()) + (('--noctime',) if location_config.get('ctime') is False else ())

View file

@ -30,7 +30,10 @@ map:
- user@backupserver:sourcehostname.borg - user@backupserver:sourcehostname.borg
one_file_system: one_file_system:
type: bool type: bool
desc: Stay in same file system (do not cross mount points). Defaults to false. desc: |
Stay in same file system (do not cross mount points). Defaults to false. But when
a database hook is used, the setting here is ignored and one_file_system is
considered true.
example: true example: true
numeric_owner: numeric_owner:
type: bool type: bool
@ -53,7 +56,9 @@ map:
desc: | desc: |
Use Borg's --read-special flag to allow backup of block and other special Use Borg's --read-special flag to allow backup of block and other special
devices. Use with caution, as it will lead to problems if used when devices. Use with caution, as it will lead to problems if used when
backing up special devices such as /dev/zero. Defaults to false. backing up special devices such as /dev/zero. Defaults to false. But when a
database hook is used, the setting here is ignored and read_special is
considered true.
example: false example: false
bsd_flags: bsd_flags:
type: bool type: bool
@ -452,7 +457,9 @@ map:
type: str type: str
desc: | desc: |
Database name (required if using this hook). Or "all" to dump all Database name (required if using this hook). Or "all" to dump all
databases on the host. databases on the host. Note that using this database hook implicitly
enables both read_special and one_file_system (see above) to support
dump and restore streaming.
example: users example: users
hostname: hostname:
type: str type: str
@ -508,7 +515,9 @@ map:
type: str type: str
desc: | desc: |
Database name (required if using this hook). Or "all" to dump all Database name (required if using this hook). Or "all" to dump all
databases on the host. databases on the host. Note that using this database hook implicitly
enables both read_special and one_file_system (see above) to support
dump and restore streaming.
example: users example: users
hostname: hostname:
type: str type: str

View file

@ -31,6 +31,10 @@ To support this, borgmatic creates temporary named pipes in `~/.borgmatic` by
default. To customize this path, set the `borgmatic_source_directory` option default. To customize this path, set the `borgmatic_source_directory` option
in the `location` section of borgmatic's configuration. in the `location` section of borgmatic's configuration.
Also note that using a database hook implicitly enables both the
`read_special` and `one_file_system` configuration settings (even if they're
disabled in your configuration) to support this dump and restore streaming.
Here's a more involved example that connects to remote databases: Here's a more involved example that connects to remote databases:
```yaml ```yaml

View file

@ -1064,7 +1064,8 @@ def test_create_archive_with_progress_and_stream_processes_calls_borg_with_progr
flexmock(module).should_receive('_make_pattern_flags').and_return(()) flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(()) flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command_with_processes').with_args( flexmock(module).should_receive('execute_command_with_processes').with_args(
('borg', 'create', '--read-special', '--progress') + ARCHIVE_WITH_PATHS, ('borg', 'create', '--one-file-system', '--read-special', '--progress')
+ ARCHIVE_WITH_PATHS,
processes=processes, processes=processes,
output_log_level=logging.INFO, output_log_level=logging.INFO,
output_file=module.DO_NOT_CAPTURE, output_file=module.DO_NOT_CAPTURE,
@ -1320,7 +1321,7 @@ def test_create_archive_with_stream_processes_calls_borg_with_processes():
flexmock(module).should_receive('_make_pattern_flags').and_return(()) flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(()) flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command_with_processes').with_args( flexmock(module).should_receive('execute_command_with_processes').with_args(
('borg', 'create', '--read-special') + ARCHIVE_WITH_PATHS, ('borg', 'create', '--one-file-system', '--read-special') + ARCHIVE_WITH_PATHS,
processes=processes, processes=processes,
output_log_level=logging.INFO, output_log_level=logging.INFO,
output_file=None, output_file=None,