Instead of failing the restore process, pause data extracting, until
already extracted data gets processed.
This commit implements this for local tar extraction. Extracting data
received from a VM is handled in qfile-(dom0-)unpacker, which also has
similar feature, turned on in 6338b936e2.
FixesQubesOS/qubes-issues#4791
Avoid failing restore after potentially lengthy processing - check
available space first.
This will be even more important after adding waiting for free space -
then if there won't be enough free space, the process will deadlock.
QubesOS/qubes-issues#4791
Xterm doesn't preserve exit code of the process running inside. This
means, the whole xterm always exits with 0, even if qvm-backup-restore
failed.
Fix this by printing the exit code at the end to the log and then extract
that last line from the log on the calling side. This way we can also
distinguish whether qvm-backup-restore or xterm failed.
Allow running unattended, with qvm-backup-restore --passphrase-file.
This require few modifications:
- copy the passphrase file into the DisposableVM (that VM knows the
passphrase anyway, so there is no extra data leak)
- close the terminal when operation finishes
Closing the terminal would eliminate almost all the feedback (operation
log, errors, warnings etc), so write it into a file in DisposableVM and
later extract it and show on the stdout. Similar to qvm-run, color it
red as a content coming from a VM.
QubesOS/qubes-issues#5310
Having Admin API, it is possible to do this properly now:
- create DisposableVM
- assign it proper permissions to create VMs and control those created
VMs
- run restore process inside
- cleanup DisposableVM afterwards
Since the RestoreInDisposableVM class contains de facto reverse parser
for qvm-backup-restore command line, add a test that will spot when it
gets out of sync.
This feature depends on modifications in various other components,
including:
- linux-utils and core-agent-linux for update qfile-unpacker
- core-admin for qrexec policy modification
QubesOS/qubes-issues#5310
Make it possible to use qvm-backup-restore in a VM. This commit is about
accessing backup itself, when stored in another VM. This involve using
qfile-unpacker instead of qfile-dom0-unpacker and also requesting disk
space monitoring, as a VM probably won't have enough space to hold the
whole backup at once.
QubesOS/qubes-issues#4791QubesOS/qubes-issues#5310
Allow setting alternative qrexec service to retrieve backup content. The
service API is slightly different than the default one: it will get only
list of files/directories to extract on its stdin, but not backup
location. The latter could be provided as a service argument, or using
other out-of-band mechanism.
This will be useful for paranoid backup restore mode, to take away
control over location/command from sandboxed qvm-backup-restore process.
QubesOS/qubes-issues#5310
Before reporting a tag as not restored, verify if it really wasn't
restored. Generally created-by-* tags cannot be created manually. But
when restoring a backup in dom0, created-by-dom0 tag is added, which in
many cases will match what want to be restored.
Adjust tests to check this too.
Previous commit introduced protection against uncommon (potentially
malicious) compression filters. This breaks restoring backups made with
a custom compression filter. Add an option to override this check, by
naming compression filter to use explicitly.
Compression filter named in a backup header is executed in restore
environment (commonly dom0). While this field is properly authenticated,
there may be cases where backup archive comes from less
trusted source, like migrating from potentially compromised
system.
Modify backup header parsing code to add field specific validators.
Whitelist only know crypto, hmac and compression algorithms.
Based on a patch by Jean-Philippe Ouellet <jpo@vt.edu>
Reported-by: Jean-Philippe Ouellet <jpo@vt.edu>
For various reasons, decryption may be slower than initial encryption
(different hardware, different system load etc). Do not fail the restore
operation if scrypt anticipate it will take considerably more time or
memory, than while creating the backup (where the scrypt parameters were
originally set).
Thanks to @jharveyb for the report and suggested solution.
FixesQubesOS/qubes-issues#4683
When both threads and processes are used, logging module can deadlock.
Workaround this by re-initializating locks in logging module.
See https://bugs.python.org/issue6721 for details.
Revert this commit when the python issue is fixed (in the python version
used for backup restore).
This is specifically the case on Travis-CI. But since dom0 backup is
restored into separate directory now, instead of directly overriding
user home, this check is much less relevant now.
Make use of better security of Qubes 4.x by using HVM by default. If
some VMs are incompatible with it (like MirageOS based), user can always
switch it to PV manually later.
