Commit Graph

2957 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
0fcceb324d backups: fix appvm handling in qvm-backup-restore 2013-11-25 06:33:31 +01:00
Marek Marczykowski-Górecki
0743531244 backups: fix VM exclude logic (restore) 2013-11-25 06:33:06 +01:00
Marek Marczykowski-Górecki
bf6bf8ed8f backups: fix backup header extraction
Pass only 'qubes.xml.000' to tar2qfile - this way it will stop reading
the source after requested file(s).
2013-11-25 06:31:38 +01:00
Marek Marczykowski-Górecki
cc37927080 backups: fix backup cleanup 2013-11-25 05:46:57 +01:00
Marek Marczykowski-Górecki
d7e3f3cb0a backups: qvm-backup: check if /var/tmp have enough space 2013-11-25 05:43:15 +01:00
Marek Marczykowski-Górecki
3a898db663 backups: Prompt for password twice for verification 2013-11-25 05:42:47 +01:00
Marek Marczykowski-Górecki
c781a522d8 backups: move backup code to separate file
Also some major cleanups: Reduce some more code duplication
(verify_hmac, simplify backup_restore_prepare). Rename
backup_dir/backup_tmpdir variables to better match its purpose. Rename
backup_do_copy back to backup_do.  Require QubesVm object (instead of VM
name) as appvm param.
2013-11-25 05:41:13 +01:00
Marek Marczykowski-Górecki
657beaf655 backups: move extracted dom0 home from /var/tmp instead of copy 2013-11-25 01:11:29 +01:00
Marek Marczykowski-Górecki
e31c3ae8e7 backup: reduce volume size to 100M and limit queue length
This way backup process won't need more than 1GB for temporary files and
also will give more precise progress information. For now it looks like
the slowest element is qrexec, so without such limit, all the data would
be prepared (basically making second copy of it in dom0) while only
first few files would be transfered to the VM.
Also backup progress is calculated based on preparation thread, so when
it finishes there is some other time needed to flush all the data to the
VM. Limiting this amount makes progress somehow more accurate (but still
off by 1GB...).
2013-11-25 00:55:59 +01:00
Marek Marczykowski-Górecki
07ae02915f backups: add missing import in qvm-backup 2013-11-25 00:55:10 +01:00
Marek Marczykowski-Górecki
10a01010bb backups: fix handling multi-volume archive during restore
We can't wait for tar next volume prompt using stderr.readline(),
because tar don't output EOL marker after this prompt. The other way
would be switching file descriptor to non-blocking mode and using lower
level os.read(), but this looks like more error-prone way (races...).
So change idea of handling such archives: after switching to next
archive volume, simply send '\n' to tar (which will receive when
needed). When getting "*.000" file, assume that previous archive was
over and wait for previous tar process. Then start the new one.

Also don't give explicit tape length, only turn multi-volume mode on. So
will correctly handle all multi-volume archives, regardless of its size.
2013-11-25 00:48:54 +01:00
Marek Marczykowski-Górecki
7229b78bbf backups: minor reduce code duplication 2013-11-25 00:48:00 +01:00
Marek Marczykowski-Górecki
bc59d7e054 backups: include file path in internal archive, implement dom0 home restore
This is mostly revert of "3d1b40f backups: keep file without path in
inner tar archive" in terms of archive format, but the code is more
robust than old one. Especially reuse already computed dir paths. Also
restore only requested files (based on selected VMs and its qubes.xml
data). Change the restore workflow to restore files first to temporary
directory, then move to final dirs. This approach:
 - will be compatible with hashed vm name in the archive path
 - is required to handle dom0 home backup (directory outside of
   /var/lib/qubes)
 - it should be also more defensive - make any changes in /var/lib/qubes
 only after successful extraction of files and creating Qubes*Vm object

Second change in this commit is implement of dom0 home backup/restore.
As qubes.xml now contains data about dom0, we have information whether
it is included in the backup (before getting actual files).
2013-11-25 00:36:40 +01:00
Marek Marczykowski-Górecki
dc6fd3c8f3 core: store dom0 info in qubes.xml
At least to have there info about its backup.
2013-11-24 23:50:39 +01:00
Marek Marczykowski-Górecki
a64f7c12ad backups: desperate try to improve readability
Especially kill long lines.
2013-11-24 23:49:53 +01:00
Marek Marczykowski-Górecki
c306b9c00a backups: increase readability of long function calls 2013-11-24 23:49:53 +01:00
Marek Marczykowski-Górecki
5477aea877 backups: increase buffer size for better performance
After this change the bottleneck is qrexec throughput.
2013-11-24 23:49:53 +01:00
Marek Marczykowski-Górecki
c64b6c04ce backups: make all the debug easy to disable 2013-11-24 23:49:39 +01:00
Marek Marczykowski-Górecki
005db6a5ab backups: fix race condition in "tape" change event during backup
Ensure that outer tar/encryptor gets all the data *and EOF* before
signalling inner tar to continue. Previously it could happen that inner
tar begins to write next data chunk, while qvm-backup still holds
previous data chunk open.
2013-11-24 03:28:21 +01:00
Marek Marczykowski-Górecki
3d1b40f25c backups: keep file without path in inner tar archive
It is senseless to have full file path in multiple locations:
 - external archive
 - qubes.xml
 - internal archive
Also it is more logical to have only "private.img" file in archive
placed in "appvms/untrusted/private.img.000". Although this is rather
cosmetic change for VMs data, it is required to backup arbitrary
directory, like dom0 user home.

Also use os.path.* instead of manual string operations (split,
partition). It is more foolproof.
2013-11-24 03:23:27 +01:00
Marek Marczykowski-Górecki
6c61e79ebf backups: don't echo entered passwords 2013-11-24 03:19:11 +01:00
Marek Marczykowski-Górecki
61b3a81e82 backup: remove unused argument from backup_prepare 2013-11-24 03:17:15 +01:00
Marek Marczykowski-Górecki
e7701d9c5d backup: check for disk space if target is local directory 2013-11-24 03:15:44 +01:00
Olivier MEDOC
51f119326b backup: improved error handling during restore 2013-11-23 02:44:05 +01:00
Olivier MEDOC
e875ae9d06 backups: use tar2qfile filtering to enable partial backup restore 2013-11-23 02:44:05 +01:00
Joanna Rutkowska
2891a15302 version 2.1.28 2013-11-22 14:42:45 +01:00
Marek Marczykowski-Górecki
a3d02db170 doc: update qvm-prefs doc 2013-11-21 17:22:01 +01:00
Marek Marczykowski-Górecki
167b412e54 qvm-tools: unify the qvm-prefs labels (#756) 2013-11-21 14:54:50 +01:00
Marek Marczykowski-Górecki
fe834bcb9c qvm-tools: fix set_* return code
Use return True/False to report success/failure instead of exit(1). This
fixes regression introduced by "92b479b qvm-tools: exit with code 1 on
error", which results in some setting not saved.
2013-11-21 14:51:14 +01:00
Marek Marczykowski-Górecki
1b83e5c687 hvm: default to template's MAC in MAC auto mode (#755) 2013-11-21 14:49:42 +01:00
Marek Marczykowski-Górecki
aeb83d1a45 hvm: do not reset root.img to template state when debug mode enabled 2013-11-21 04:36:53 +01:00
Marek Marczykowski-Górecki
a457b62728 core: more flexible mechanism for template compatibility check
Using class method allow the users (Qubes Manager at least) to check
for compatibility without having any particular VM instance - useful
while creating the VM.
2013-11-21 03:42:31 +01:00
Marek Marczykowski-Górecki
efeb284ab1 core: do not call resize2fs on private.img in dom0
Do not parse VM data (filesystem metadata in this case) in dom0, as this
expose dom0 for potential attack.
2013-11-21 03:38:12 +01:00
Joanna Rutkowska
b91aa4133d version 2.1.27 2013-11-20 15:55:36 +01:00
Marek Marczykowski-Górecki
f85c2ffa5a Merge remote-tracking branch 'joanna/master' 2013-11-20 02:58:00 +01:00
Marek Marczykowski-Górecki
6fddae3b9b Support for autostart VMs (#724) 2013-11-20 02:57:17 +01:00
Marek Marczykowski-Górecki
2005207462 Template support for HVM (#719)
Any HVM (which isn't already template-based) can be a template for
another HVM. For now do not allow simultaneous run of template and its
VM (this assumption simplify the implementation, as no root-cow.img is
needed).
2013-11-19 18:42:59 +01:00
Marek Marczykowski-Górecki
92b479bf49 qvm-tools: exit with code 1 on error
Not only print error message.
2013-11-19 18:40:16 +01:00
Marek Marczykowski-Górecki
25fd41aa2f qvm-tools: do not assume that every template VM must have root-cow.img
Especially HVM templates do not have (at least for now).
2013-11-19 18:39:22 +01:00
Marek Marczykowski-Górecki
1756ab33e9 qvm-tools: make qvm-ls code more defensive
Do not assume only predefined VMs types, do not assume only one type of
template etc.
2013-11-19 18:36:12 +01:00
Marek Marczykowski-Górecki
4090fdf758 QubesHVm: restore private.img support 2013-11-19 18:35:10 +01:00
Marek Marczykowski-Górecki
1315bdec87 QubesHVm: fix copy&paste error 2013-11-19 18:33:35 +01:00
Marek Marczykowski-Górecki
f0e24c358e qvm-tools: clarify help message/options error checking (#741) 2013-11-18 01:15:17 +01:00
Joanna Rutkowska
07f011ebec version 2.1.26 2013-11-16 15:26:53 +01:00
Marek Marczykowski-Górecki
dfa9e7c0df Add device-mapper devices to name->(major,minor) mapping
qvm-block will still not list device-mapper devices, but it would be
much easier to modify it.
2013-11-16 01:29:50 +01:00
Marek Marczykowski-Górecki
b3c127091d hvm: check for HVM capability on host when failed to start the VM 2013-11-09 23:48:49 +01:00
Marek Marczykowski-Górecki
8cffdea41f backup: move qubes.{Backup,Restore} services to core-agent-linux repo 2013-11-09 18:57:26 +01:00
Marek Marczykowski-Górecki
3666d6ced9 backup: wait for process termination in restore header phase
One more race condition, which could cause qvm-backup-restore hang.
2013-11-09 18:56:08 +01:00
Marek Marczykowski-Górecki
319158d5b1 backup: restore: Ignore qubes.xml in the second restore pass
Already processed in backup prepare phase). This is only because
qfile-dom0-unpacker doesn't support selective unpack (like tar do).
This should be extended to skip also VMs not selected for restore.
2013-11-09 18:54:52 +01:00
Marek Marczykowski-Górecki
1880f61c2d backup: restore: process files until EOF received
Not only until unpacker process is running. This is another race
condition, which would cause some data left in the pipe buffer not
processed.
2013-11-09 18:53:57 +01:00