Qubes/core-admin
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

qubes/tools: qvm-backup and qvm-backup-restore tools

Browse Source 
Fixes QubesOS/qubes-issues#1213
Fixes QubesOS/qubes-issues#1214
This commit is contained in:
Marek Marczykowski-Górecki 2016-09-21 16:02:50 +02:00
parent 7af3f4b19a
commit 96a4bb650b
No known key found for this signature in database
GPG Key ID: 063938BA42CFA724
5 changed files with 534 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
doc/manpages/qvm-backup-restore.rst
View File

@ -15,6 +15,15 @@ Options
Show this help message and exit
.. option:: --verbose, -v
Increase verbosity
.. option:: --quiet, -q
Decrease verbosity
.. option:: --verify-only
Do not restore the data, only verify backup integrity
@ -31,6 +40,10 @@ Options
Do not restore VMs that are already present on the host
.. option:: --rename-conflicting
Restore VMs that are already present on the host under different names
.. option:: --force-root
Force to run, even with root privileges
@ -56,17 +69,11 @@ Options
Restore from a backup located in a specific AppVM
.. option:: --encrypted, -e
.. option:: --passphrase-file, -p
The backup is encrypted
Read passphrase from file, or use '-' to read from stdin
.. option:: --compressed. -z
The backup is compressed
.. option:: --debug
Enable (a lot of) debug output
Authors
=======

80
doc/manpages/qvm-backup.rst
View File

@ -1,26 +1,86 @@
.. program:: qvm-backup
=======================================================
:program:`qvm-backup` -- Create backup of specified VMs
=======================================================
:program:`qvm-backup` -- None
=============================
Synopsis
========
:command:`qvm-backup` [*options*] <*backup-dir-path*>
--------
:command:`qvm-backup` skel-manpage.py [-h] [--verbose] [--quiet] [--force-root] [--exclude EXCLUDE_LIST] [--dest-vm *APPVM*] [--encrypt] [--no-encrypt] [--passphrase-file PASS_FILE] [--enc-algo CRYPTO_ALGORITHM] [--hmac-algo HMAC_ALGORITHM] [--compress] [--compress-filter COMPRESS_FILTER] [--tmpdir *TMPDIR*] backup_location [vms [vms ...]]
Options
=======
-------
.. option:: --help, -h
Show this help message and exit
show this help message and exit
.. option:: --exclude=EXCLUDE_LIST, -x EXCLUDE_LIST
.. option:: --verbose, -v
Exclude the specified VM from backup (might be repeated)
increase verbosity
.. option:: --quiet, -q
decrease verbosity
.. option:: --force-root
force to run as root
.. option:: --exclude, -x
Exclude the specified VM from the backup (may be repeated)
.. option:: --dest-vm, -d
Specify the destination VM to which the backup will be sent (implies -e)
.. option:: --encrypt, -e
Encrypt the backup
.. option:: --no-encrypt
Skip encryption even if sending the backup to a VM
.. option:: --passphrase-file, -p
Read passphrase from a file, or use '-' to read from stdin
.. option:: --enc-algo, -E
Specify a non-default encryption algorithm. For a list of supported algorithms, execute 'openssl list-cipher-algorithms' (implies -e)
.. option:: --hmac-algo, -H
Specify a non-default HMAC algorithm. For a list of supported algorithms, execute 'openssl list-message-digest-algorithms'
.. option:: --compress, -z
Compress the backup
.. option:: --compress-filter, -Z
Specify a non-default compression filter program (default: gzip)
.. option:: --tmpdir
Specify a temporary directory (if you have at least 1GB free RAM in dom0, use of /tmp is advised) (default: /var/tmp)
Arguments
---------
The first positional parameter is the backup location (directory path, or
command to pipe backup to). After that you may specify the qubes you'd like to
backup. If not specified, all qubes with `include_in_backups` property set are
included.
Authors
=======
-------
| Joanna Rutkowska <joanna at invisiblethingslab dot com>
| Rafal Wojtczuk <rafal at invisiblethingslab dot com>
| Marek Marczykowski <marmarek at invisiblethingslab dot com>
| Wojtek Porczyk <woju at invisiblethingslab dot com>
.. vim: ts=3 sw=3 et tw=80

186
qubes/tools/qvm_backup.py Normal file
View File

@ -0,0 +1,186 @@
#!/usr/bin/python2
# -*- encoding: utf8 -*-
#
# The Qubes OS Project, http://www.qubes-os.org
#
# Copyright (C) 2016 Marek Marczykowski-Górecki
# <marmarek@invisiblethingslab.com>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
from __future__ import print_function
import getpass
import locale
import os
import sys
import qubes.backup
import qubes.tools
import qubes.utils
parser = qubes.tools.QubesArgumentParser(want_force_root=True)
parser.add_argument("--exclude", "-x", action="append",
dest="exclude_list", default=[],
help="Exclude the specified VM from the backup (may be "
"repeated)")
parser.add_argument("--dest-vm", "-d", action="store",
dest="appvm", default=None,
help="Specify the destination VM to which the backup "
"will be sent (implies -e)")
parser.add_argument("--encrypt", "-e", action="store_true", dest="encrypted",
default=False,
help="Encrypt the backup")
parser.add_argument("--no-encrypt", action="store_true",
dest="no_encrypt", default=False,
help="Skip encryption even if sending the backup to a "
"VM")
parser.add_argument("--passphrase-file", "-p", action="store",
dest="pass_file", default=None,
help="Read passphrase from a file, or use '-' to read "
"from stdin")
parser.add_argument("--enc-algo", "-E", action="store",
dest="crypto_algorithm", default=None,
help="Specify a non-default encryption algorithm. For a "
"list of supported algorithms, execute 'openssl "
"list-cipher-algorithms' (implies -e)")
parser.add_argument("--hmac-algo", "-H", action="store",
dest="hmac_algorithm", default=None,
help="Specify a non-default HMAC algorithm. For a list "
"of supported algorithms, execute 'openssl "
"list-message-digest-algorithms'")
parser.add_argument("--compress", "-z", action="store_true", dest="compressed",
default=False,
help="Compress the backup")
parser.add_argument("--compress-filter", "-Z", action="store",
dest="compression_filter", default=False,
help="Specify a non-default compression filter program "
"(default: gzip)")
parser.add_argument("--tmpdir", action="store", dest="tmpdir", default=None,
help="Specify a temporary directory (if you have at least "
"1GB free RAM in dom0, use of /tmp is advised) ("
"default: /var/tmp)")
parser.add_argument("backup_location", action="store",
help="Backup location (directory path, or command to pipe backup to)")
parser.add_argument("vms", nargs="*", action=qubes.tools.VmNameAction,
help="Backup only those VMs")
def main(args=None):
args = parser.parse_args(args)
appvm = None
if args.appvm:
try:
appvm = args.app.domains[args.appvm]
except KeyError:
parser.error('no such domain: {!r}'.format(args.appvm))
args.app.log.info(("NOTE: VM {} will be excluded because it is "
"the backup destination.").format(args.appvm),
file=sys.stderr)
if appvm:
args.exclude_list.append(appvm.name)
if args.appvm or args.crypto_algorithm:
args.encrypted = True
if args.no_encrypt:
args.encrypted = False
try:
backup = qubes.backup.Backup(args.app,
args.domains if args.domains else None,
exclude_list=args.exclude_list)
except qubes.exc.QubesException as e:
parser.error_runtime(str(e))
# unreachable - error_runtime will raise SystemExit
return 1
backup.target_dir = args.backup_location
if not appvm:
if os.path.isdir(args.backup_location):
stat = os.statvfs(args.backup_location)
else:
stat = os.statvfs(os.path.dirname(args.backup_location))
backup_fs_free_sz = stat.f_bsize * stat.f_bavail
print()
if backup.total_backup_bytes > backup_fs_free_sz:
parser.error_runtime("Not enough space available on the "
"backup filesystem!")
args.app.log.info("Available space: {0}".format(
qubes.utils.size_to_human(backup_fs_free_sz)))
else:
stat = os.statvfs('/var/tmp')
backup_fs_free_sz = stat.f_bsize * stat.f_bavail
print()
if backup_fs_free_sz < 1000000000:
parser.error_runtime("Not enough space available "
"on the local filesystem (1GB required for temporary files)!")
if not appvm.is_running():
appvm.start()
if not args.encrypted:
args.app.log.info("WARNING: The backup will NOT be encrypted!", file=sys.stderr)
if args.pass_file is not None:
pass_f = open(args.pass_file) if args.pass_file != "-" else sys.stdin
passphrase = pass_f.readline().rstrip()
if pass_f is not sys.stdin:
pass_f.close()
else:
if raw_input("Do you want to proceed? [y/N] ").upper() != "Y":
return 0
prompt = ("Please enter the passphrase that will be used to {}verify "
"the backup: ").format('encrypt and ' if args.encrypted else '')
passphrase = getpass.getpass(prompt)
if getpass.getpass("Enter again for verification: ") != passphrase:
parser.error_runtime("Passphrase mismatch!")
backup.encrypted = args.encrypted
backup.compressed = args.compressed
if args.compression_filter:
backup.compression_filter = args.compression_filter
encoding = sys.stdin.encoding or locale.getpreferredencoding()
backup.passphrase = passphrase.decode(encoding)
if args.hmac_algorithm:
backup.hmac_algorithm = args.hmac_algorithm
if args.crypto_algorithm:
backup.crypto_algorithm = args.crypto_algorithm
if args.tmpdir:
backup.tmpdir = args.tmpdir
if appvm:
backup.target_vm = appvm
try:
backup.backup_do()
except qubes.exc.QubesException as e:
parser.error_runtime(str(e))
print()
args.app.log.info("Backup completed.")
return 0
if __name__ == '__main__':
main()

261
qubes/tools/qvm_backup_restore.py Normal file
View File

@ -0,0 +1,261 @@
#!/usr/bin/python2
# -*- encoding: utf8 -*-
#
# The Qubes OS Project, http://www.qubes-os.org
#
# Copyright (C) 2016 Marek Marczykowski-Górecki
# <marmarek@invisiblethingslab.com>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
from __future__ import print_function
import getpass
import locale
import sys
import qubes.backup
import qubes.tools
import qubes.utils
parser = qubes.tools.QubesArgumentParser(want_force_root=True)
parser.add_argument("--verify-only", action="store_true",
dest="verify_only", default=False,
help="Verify backup integrity without restoring any "
"data")
parser.add_argument("--skip-broken", action="store_true", dest="skip_broken",
default=False,
help="Do not restore VMs that have missing TemplateVMs "
"or NetVMs")
parser.add_argument("--ignore-missing", action="store_true",
dest="ignore_missing", default=False,
help="Restore VMs even if their associated TemplateVMs "
"and NetVMs are missing")
parser.add_argument("--skip-conflicting", action="store_true",
dest="skip_conflicting", default=False,
help="Do not restore VMs that are already present on "
"the host")
parser.add_argument("--rename-conflicting", action="store_true",
dest="rename_conflicting", default=False,
help="Restore VMs that are already present on the host "
"under different names")
parser.add_argument("--replace-template", action="append",
dest="replace_template", default=[],
help="Restore VMs using another TemplateVM; syntax: "
"old-template-name:new-template-name (may be "
"repeated)")
parser.add_argument("-x", "--exclude", action="append", dest="exclude",
default=[],
help="Skip restore of specified VM (may be repeated)")
parser.add_argument("--skip-dom0-home", action="store_false", dest="dom0_home",
default=True,
help="Do not restore dom0 user home directory")
parser.add_argument("--ignore-username-mismatch", action="store_true",
dest="ignore_username_mismatch", default=False,
help="Ignore dom0 username mismatch when restoring home "
"directory")
parser.add_argument("-d", "--dest-vm", action="store", dest="appvm",
help="Specify VM containing the backup to be restored")
parser.add_argument("-p", "--passphrase-file", action="store",
dest="pass_file", default=None,
help="Read passphrase from file, or use '-' to read from stdin")
parser.add_argument('backup_location', action='store',
help="Backup directory name, or command to pipe from")
parser.add_argument('vms', nargs='*', action='store', default='[]',
help='Restore only those VMs')
def main(args=None):
args = parser.parse_args(args)
appvm = None
if args.appvm:
try:
appvm = args.app.domains[args.appvm]
except KeyError:
parser.error('no such domain: {!r}'.format(args.appvm))
if args.pass_file is not None:
pass_f = open(args.pass_file) if args.pass_file != "-" else sys.stdin
passphrase = pass_f.readline().rstrip()
if pass_f is not sys.stdin:
pass_f.close()
else:
passphrase = getpass.getpass("Please enter the passphrase to verify "
"and (if encrypted) decrypt the backup: ")
encoding = sys.stdin.encoding or locale.getpreferredencoding()
passphrase = passphrase.decode(encoding)
args.app.log.info("Checking backup content...")
try:
backup = qubes.backup.BackupRestore(args.app, args.backup_location,
appvm, passphrase)
except qubes.exc.QubesException as e:
parser.error_runtime(str(e))
# unreachable - error_runtime will raise SystemExit
return 1
if args.ignore_missing:
backup.options.use_default_template = True
backup.options.use_default_netvm = True
if args.replace_template:
backup.options.replace_template = args.replace_template
if args.rename_conflicting:
backup.options.rename_conflicting = True
if not args.dom0_home:
backup.options.dom0_home = False
if args.ignore_username_mismatch:
backup.options.ignore_username_mismatch = True
if args.exclude:
backup.options.exclude = args.exclude
if args.verify_only:
backup.options.verify_only = True
restore_info = None
try:
restore_info = backup.get_restore_info()
except qubes.exc.QubesException as e:
parser.error_runtime(str(e))
print(backup.get_restore_summary(restore_info))
there_are_conflicting_vms = False
there_are_missing_templates = False
there_are_missing_netvms = False
dom0_username_mismatch = False
for vm_info in restore_info.values():
assert isinstance(vm_info, qubes.backup.BackupRestore.VMToRestore)
if qubes.backup.BackupRestore.VMToRestore.EXCLUDED in vm_info.problems:
continue
if qubes.backup.BackupRestore.VMToRestore.MISSING_TEMPLATE in \
vm_info.problems:
there_are_missing_templates = True
if qubes.backup.BackupRestore.VMToRestore.MISSING_NETVM in \
vm_info.problems:
there_are_missing_netvms = True
if qubes.backup.BackupRestore.VMToRestore.ALREADY_EXISTS in \
vm_info.problems:
there_are_conflicting_vms = True
if qubes.backup.BackupRestore.Dom0ToRestore.USERNAME_MISMATCH in \
vm_info.problems:
dom0_username_mismatch = True
if there_are_conflicting_vms:
args.app.log.error(
"*** There are VMs with conflicting names on the host! ***")
if args.skip_conflicting:
args.app.log.error(
"Those VMs will not be restored. "
"The host VMs will NOT be overwritten.")
else:
args.app.log.error(
"Remove VMs with conflicting names from the host "
"before proceeding.")
args.app.log.error(
"Or use --skip-conflicting to restore only those VMs that "
"do not exist on the host.")
args.app.log.error(
"Or use --rename-conflicting to restore those VMs under "
"modified names (with numbers at the end).")
return 1
args.app.log.info("The above VMs will be copied and added to your system.")
args.app.log.info("Exisiting VMs will NOT be removed.")
if there_are_missing_templates:
args.app.log.error("*** One or more TemplateVMs are missing on the "
"host! ***")
if not (args.skip_broken or args.ignore_missing):
args.app.log.error("Install them before proceeding with the "
"restore.")
args.app.log.error("Or pass: --skip-broken or --ignore-missing.")
return 1
elif args.skip_broken:
args.app.log.error("Skipping broken entries: VMs that depend on "
"missing TemplateVMs will NOT be restored.")
elif args.ignore_missing:
args.app.log.error("Ignoring missing entries: VMs that depend "
"on missing TemplateVMs will NOT be restored.")
else:
args.app.log.error("INTERNAL ERROR! Please report this to the "
"Qubes OS team!")
return 1
if there_are_missing_netvms:
args.app.log.error("*** One or more NetVMs are missing on the "
"host! ***")
if not (args.skip_broken or args.ignore_missing):
args.app.log.error("Install them before proceeding with the "
"restore.")
args.app.log.error("Or pass: --skip-broken or --ignore-missing.")
return 1
elif args.skip_broken:
args.app.log.error("Skipping broken entries: VMs that depend on "
"missing NetVMs will NOT be restored.")
elif args.ignore_missing:
args.app.log.error("Ignoring missing entries: VMs that depend "
"on missing NetVMs will NOT be restored.")
else:
args.app.log.error("INTERNAL ERROR! Please report this to the "
"Qubes OS team!")
return 1
if 'dom0' in restore_info.keys() and args.dom0_home:
if dom0_username_mismatch:
args.app.log.error("*** Dom0 username mismatch! This can break "
"some settings! ***")
if not args.ignore_username_mismatch:
args.app.log.error("Skip restoring the dom0 home directory "
"(--skip-dom0-home), or pass "
"--ignore-username-mismatch to continue "
"anyway.")
return 1
else:
args.app.log.error("Continuing as directed.")
args.app.log.error("NOTE: Before restoring the dom0 home directory, "
"a new directory named "
"'home-pre-restore-<current-time>' will be "
"created inside the dom0 home directory. If any "
"restored files conflict with existing files, "
"the existing files will be moved to this new "
"directory.")
if args.pass_file is None:
if raw_input("Do you want to proceed? [y/N] ").upper() != "Y":
exit(0)
try:
backup.restore_do(restore_info)
except qubes.exc.QubesException as e:
parser.error_runtime(str(e))
if __name__ == '__main__':
main()

2
rpm_spec/core-dom0.spec
View File

@ -250,6 +250,8 @@ fi
%{python_sitelib}/qubes/tools/qubes_prefs.py*
%{python_sitelib}/qubes/tools/qvm_block.py*
%{python_sitelib}/qubes/tools/qubes_lvm.py*
%{python_sitelib}/qubes/tools/qvm_backup.py*
%{python_sitelib}/qubes/tools/qvm_backup_restore.py*
%{python_sitelib}/qubes/tools/qvm_create.py*
%{python_sitelib}/qubes/tools/qvm_device.py*
%{python_sitelib}/qubes/tools/qvm_features.py*