Commit Graph

76 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
2164a8d7b8
Change license to LGPL v2.1+
See this thread for reasoning and acceptance from contributors:
https://groups.google.com/d/topic/qubes-devel/G7KzrfU0lWY/discussion
"Changing qubes-core-admin license to LGPL v2.1+"
2017-10-12 00:11:50 +02:00
Marek Marczykowski-Górecki
bde8d45cca
backup: update last backup timestamp only when backup succeed
QubesOS/qubes-issues#2931
2017-07-29 04:47:44 +02:00
Marek Marczykowski-Górecki
1556814f59
backup: fix handling already encoded passphrase
When passphrase is retrieved from VM, it is already encoded. Do not try
to encode it again.

QubesOS/qubes-issues#2931
2017-07-29 04:47:43 +02:00
Marek Marczykowski-Górecki
abdad8c2b2
backup: allow backup running VMs
Since we have LVM by default, it is possible to backup VMs while they
are running. For now it will include its state from before startup, but
later we may implement some other logic (a snapshot of running VM).
2017-07-21 03:12:23 +02:00
Marek Marczykowski-Górecki
1ec0060577
backup: check inner archiver exit code
If inner tar returns with non-zero code, mostly like the backup is
incomplete, so it is very important to report it.
2017-07-21 03:11:02 +02:00
Marek Marczykowski-Górecki
81246cac64
backup: fix get_backup_summary
QubesOS/qubes-issues#2931
2017-07-20 12:09:17 +02:00
Marek Marczykowski-Górecki
579d90e174
backup: use proper API for getting volumes list and size
Do not assume static list of volume (although it is true for now), and
also use proper API for getting volume size, instead of assuming it's a
normal file.
2017-07-20 12:09:17 +02:00
Marek Marczykowski-Górecki
9dd0cfe6c5
backup: drop BackupHeader loading code
It is useless here since backup restore is moved to
qubes-core-admin-client.
2017-07-20 12:09:16 +02:00
Marek Marczykowski-Górecki
d4e9120903
backup: convert from multiprocessing to asyncio
QubesOS/qubes-issues#2931
2017-07-20 02:48:44 +02:00
Marek Marczykowski-Górecki
2ace5901df
backup: make pylint happy, remove dead settings
After removing backup-restore code the remaining part of backup.py is
not that bad - lets fix it and no longer exclude it from pylint.
2017-07-18 01:42:36 +02:00
Marek Marczykowski-Górecki
61519014cb
Remove qvm-backup-restore and its backend code
This functionality is moved to qubes-core-admin-client, which does it
over Admin API

QubesOS/qubes-issues#1214
2017-07-18 01:30:39 +02:00
Marek Marczykowski-Górecki
21940bef90
Change vm.hvm into vm.virt_mode
Fixes QubesOS/qubes-issues#2912
2017-07-17 12:26:10 +02:00
Marek Marczykowski-Górecki
f393ce62dc
Fix docstrings formatting
Fix warnings/errors reported by Sphinx
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
d75fc6c28b
backup: add a little more usage info, include docs in main index 2017-04-21 19:58:32 +02:00
Marek Marczykowski-Górecki
751415434c
backup: make hmac verification more defensive
Check HMAC file size, read it as binary or with 'ascii' encoding only.
2017-02-27 02:37:52 +01:00
Marek Marczykowski-Górecki
45709b510a
backup: minor fixes after bringing back scrypt support 2017-02-27 02:37:51 +01:00
Marek Marczykowski-Górecki
1363251438
Revert "Revert "backup: use 'scrypt' tool for backup encryption and integrity protection""
This reverts commit 0f1672dc63.

Bring it back. Lets not revert the whole feature just because required
package exists only in qubes-builder, not in some online repository.
Also, this revert didn't go as planned - there was a reference to a
'passphrase' local variable, but it wasn't assigned any value.

Cc: @woju
2017-02-27 02:37:50 +01:00
Marek Marczykowski-Górecki
2c3e112951
backup: one more py3k related fix 2017-02-27 02:37:48 +01:00
Marek Marczykowski-Górecki
9c5c70fe25
qubes/backup: py3k related fixes
- str/bytes usage
- functools.reduce
- dict.items instead of dict.iteritems etc
2017-02-27 02:35:50 +01:00
Wojtek Porczyk
d74567d65f qubes: port core to python3
fixes QubesOS/qubes-issues#2074
2017-01-20 16:42:51 +01:00
Wojtek Porczyk
0f1672dc63 Revert "backup: use 'scrypt' tool for backup encryption and integrity protection"
This reverts commit 418d749680.

Package `scrypt` is currently not installable (not present in any repo).

Cc: @marmarek
2016-12-05 18:36:13 +01:00
Marek Marczykowski-Górecki
af182c4fd1
backup: fixup restore options just before restoring VMs
When user included/excluded some VMs for restoration, it may be
neceesarry to fix dependencies between them (for example when default
template is no longer going to be restored).
Also fix handling conflicting names.
2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
043d20c05d
backup: fix handling non-ascii characters in backup passphrase
Fixes QubesOS/qubes-issues#2398
2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki
49e718cf57
backup: mark 'encryption' option as deprecated - all backups are encrypted
QubesOS/qubes-issues#971
2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki
51b66208f3
backup: verify if archive chunks are not reordered
Now, when file name is also integrity protected (prefixed to the
passphrase), we can make sure that input files are given in the same
order. And are parts of the same VM.

QubesOS/qubes-issues#971
2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki
4ad15c082b
backup: add 'backup_id' to integrity protection
This prevent switching parts of backup of the same VM between different
backups made by the same user (or actually: with the same passphrase).

QubesOS/qubes-issues#971
2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki
418d749680
backup: use 'scrypt' tool for backup encryption and integrity protection
`openssl dgst` and `openssl enc` used previously poorly handle key
stretching - in case of `openssl enc` encryption key is derived using
single MD5 iteration, without even any salt. This hardly prevent
brute force or even rainbow tables attacks. To make things worse, the
same key is used for encryption and integrity protection which ease
brute force even further.
All this is still about brute force attacks, so when using long, high
entropy passphrase, it should be still relatively safe. But lets do
better.
According to discussion in QubesOS/qubes-issues#971, scrypt algorithm is
a good choice for key stretching (it isn't the best of all existing, but
a good one and widely adopted). At the same time, lets switch away from
`openssl` tool, as it is very limited and apparently not designed for
production use. Use `scrypt` tool, which is very simple and does exactly
what we need - encrypt the data and integrity protect it. Its archive
format have own (simple) header with data required by the `scrypt`
algorithm, including salt. Internally data is encrypted with AES256-CTR
and integrity protected with HMAC-SHA256. For details see:
https://github.com/tarsnap/scrypt/blob/master/FORMAT

This means change of backup format. Mainly:

1. HMAC is stored in scrypt header, so don't use separate file for it.
Instead have data in files with `.enc` extension.
2. For compatibility leave `backup-header` and `backup-header.hmac`. But
`backup-header.hmac` is really scrypt-encrypted version of `backup-header`.
3. For each file, prepend its identifier to the passphrase, to
authenticate filename itself too. Having this we can guard against
reordering archive files within a single backup and across backups. This
identifier is built as:

        backup ID (from backup-header)!filename!

For backup-header itself, there is no backup ID (just 'backup-header!').

Fixes QubesOS/qubes-issues#971
2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki
d7c355eadb
backup: make wait_backup_feedback/handle_streams less ugly
Have a generic function `handle_streams`, instead of
`wait_backup_feedback` with open coded process names and manual
iteration over them.

No functional change, besides minor logging change.
2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki
33fecd90c1
qubes/backup: misc fixes
Fix restoring ProxyVM and NetVM from core2. Use correct VM class.
2016-10-05 01:58:11 +02:00
Marek Marczykowski-Górecki
339c47480e
qubes/backup: include LVM volumes content in backup
Use just introduced tar writer to archive content of LVM volumes (or
more generally: block devices). Place them as 'private.img' and
'root.img' files in the backup - just like in old format. This require
support for replacing file name in tar header - another thing trivially
supported with tar writer.
2016-10-05 01:55:30 +02:00
Marek Marczykowski-Górecki
278a5340dc
qubes/backup: fix relative path calculation
os.path.relpath strip trailing '/.' from the path, but it is important
to distinguish whole-directory archive (which is tar of '.').
2016-10-04 21:38:59 +02:00
Marek Marczykowski-Górecki
ab69fdd7f4
qubes/backup: reduce code duplication
Move inner tar process cleanup to a separate function
2016-10-03 13:43:36 +02:00
Marek Marczykowski-Górecki
20590bff57
backup: adjust LVM volume size when restoring its content.
Old backup metadata (old qubes.xml) does not contain info about
individual volume sizes. So, extract it from tar header (using verbose
output during restore) and resize volume accordingly.
Without this, restoring volumes larger than default would be impossible.
2016-09-29 02:08:29 +02:00
Marek Marczykowski-Górecki
0a35bd06aa
backup: support relocating files to different storage pool
To ease all this, rework restore workflow: first create QubesVM objects,
and all their files (as for fresh VM), then override them with data
from backup - possibly redirecting some files to new location. This
allows generic code to create LVM volumes and then only restore its
content.
2016-09-29 02:08:29 +02:00
Marek Marczykowski-Górecki
bbbe045b62
qubes/backup: better handle restoring kernel=None case
It makes sense for HVM domains, so allow such configuration.
2016-08-17 01:18:11 +02:00
Marek Marczykowski-Górecki
2f977ff4f5
backup: open backup collection in offline mode
Otherwise it may try to manipulate live libvirt objects with the same
names (and probably different UUIDs, which would fail).
2016-08-16 00:00:14 +02:00
Marek Marczykowski-Górecki
4e022382a5
Merge remote-tracking branch 'origin/master' into core3-devel 2016-08-08 00:11:46 +02:00
Bahtiar `kalkin-` Gadimov
61feb0ced7
Migrate backup to new storage api 2016-07-13 22:34:37 +02:00
Bahtiar `kalkin-` Gadimov
bb8b58b04c
qubes.backup fix verify_files 2016-07-13 22:34:36 +02:00
Bahtiar `kalkin-` Gadimov
b584d1a4bb
Remove QubesVM.is_netvm() and is_disposablevm() 2016-06-23 14:23:33 +02:00
Bahtiar `kalkin-` Gadimov
4a0b7585d1
Remove QubesVM.is_proxyvm() 2016-06-23 14:23:32 +02:00
Bahtiar `kalkin-` Gadimov
e08ca1ff57
Remove QubesVM.is_template() 2016-06-23 14:23:31 +02:00
Bahtiar `kalkin-` Gadimov
0a471e719b
qubes.backup fix access to volume paths 2016-06-21 14:58:09 +02:00
Marek Marczykowski-Górecki
98effef606
backup: add option to use default kernel for restored VMs 2016-06-16 17:06:42 +02:00
Marek Marczykowski-Górecki
91404cc647
backup: collect files to backup once 2016-06-16 17:06:41 +02:00
Marek Marczykowski-Górecki
2dacb3a542
backup: drop/resolve minor "TODO" comments 2016-06-16 17:06:41 +02:00
Marek Marczykowski-Górecki
2909d252d6 backup: get_fisk_usage is now in storage.file 2016-06-02 12:41:59 +02:00
Marek Marczykowski-Górecki
8c6fe7ed90
Merge remote-tracking branch 'origin/master' into core3-devel-mm 2016-04-11 13:03:12 +02:00
Marek Marczykowski-Górecki
d9a8cd5d68
backup: minor code style fixes 2016-04-11 00:13:13 +02:00
Marek Marczykowski-Górecki
c28f50f6fe backup: minor fixes 2016-04-07 13:21:00 +02:00