Commit Graph

46 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
55c9a82c0a
Merge branch 'backup-logging-deadlock'
* backup-logging-deadlock:
  backup: avoid deadlock on logging during restore
2018-07-17 15:46:17 +02:00
Marek Marczykowski-Górecki
a7bfc03509
backup: avoid deadlock on logging during restore
When both threads and processes are used, logging module can deadlock.
Workaround this by re-initializating locks in logging module.
See https://bugs.python.org/issue6721 for details.
Revert this commit when the python issue is fixed (in the python version
used for backup restore).
2018-07-16 22:30:35 +02:00
Marek Marczykowski-Górecki
16064f6fb4
Fixes for pylint 2.0
Ignore most of them - we still support python 2.7 here.
Fix no-else-return.
2018-07-16 02:25:25 +02:00
Rusty Bird
0171269138
restore.py: add template-related debug log messages 2018-02-24 12:43:42 +00:00
Rusty Bird
ab65338cfb
restory.py: restore template_for_dispvms property early 2018-02-24 12:43:42 +00:00
Rusty Bird
c86b731d3e
restore.py: factor out _restore_property() helper 2018-02-24 12:43:42 +00:00
Rusty Bird
2ace32bb73
restore.py: deal with DispVM templates 2018-02-24 12:43:42 +00:00
Rusty Bird
68c8b7fafc
restore.py: sort into three tiers in _templates_first() 2018-02-24 12:43:42 +00:00
Rusty Bird
01ce5387d3
restore.py: don't try to restore dispid property 2018-02-23 17:15:20 +00:00
Marek Marczykowski-Górecki
56dc984507
backup: don't crash when no 'qubes' group is present
This is specifically the case on Travis-CI. But since dom0 backup is
restored into separate directory now, instead of directly overriding
user home, this check is much less relevant now.
2018-02-14 04:28:23 +01:00
Marek Marczykowski-Górecki
a821034653
backup: add support for openssl 1.1.0 options
The command for listing available algorithms have changed:
old: openssl list-message-digest-algorithms
new: openssl list -digest-algorithms
2018-02-14 04:28:23 +01:00
Marek Marczykowski-Górecki
04635e7e80
backup: skip dom0's properties while restoring core2 backup
Fixes QubesOS/qubes-issues#3467
2018-02-14 04:28:22 +01:00
Christopher Laprise
e81c2f3cf6
unused variable 2018-02-01 23:20:31 -05:00
Christopher Laprise
e3d12fb96d
style issues 2018-02-01 23:00:06 -05:00
Christopher Laprise
30dd7acaa9
Fix dom0 restore 2018-02-01 22:21:24 -05:00
Christopher Laprise
60bfd63a25
Fix dom0 handling 2018-02-01 19:40:42 -05:00
Christopher Laprise
4b46d59179
Fix AdminVm class name 2018-01-24 20:42:15 -05:00
Marek Marczykowski-Górecki
2861d46dfa
backup/restore: do not shrink volumes on restore
This operation is blocked in current storage pool implementations.
2018-01-18 19:09:38 +01:00
Marek Marczykowski-Górecki
0e033645ad
backup: restore Qubes 3.x VMs as HVMs
Make use of better security of Qubes 4.x by using HVM by default. If
some VMs are incompatible with it (like MirageOS based), user can always
switch it to PV manually later.
2018-01-04 23:39:31 +01:00
Marek Marczykowski-Górecki
c82d5b0a73
Make pylint happy
Updated pylint on Travis-CI is slightly stricter.
2017-12-22 18:38:24 +01:00
Marek Marczykowski-Górecki
796a3d43ae
backup: make pylint happy
Use 'vm_info' instead of restore_info['dom0'], as it is available inside
the loop. And break some lines to not exceed 80 chars..
2017-12-22 17:23:27 +01:00
Marek Marczykowski-Górecki
c2281fefc8
Merge remote-tracking branch 'qubesos/pr/43'
* qubesos/pr/43:
  Fix dom0-related issues
2017-12-22 17:19:59 +01:00
Christopher Laprise
c6bc4f05cb
Fix dom0-related issues 2017-12-11 10:16:53 -05:00
Marek Marczykowski-Górecki
5944de1eed
Make pylint happy 2017-12-10 22:42:48 +01:00
Christopher Laprise
d84423b0b2
verify-only avoid dom0 handler 2017-12-07 22:36:58 -05:00
Christopher Laprise
f86c640d65
Fix verify-only 2017-12-06 21:31:30 -05:00
Christopher Laprise
6fe5f8c0e7
Issue #3303 2017-12-01 18:21:59 -05:00
Marek Marczykowski-Górecki
7f728e2dad
backup: fix handling labels in Qubes 4.0 backups
Labels can be referred as either id or name. Support both ways.

Fixes QubesOS/qubes-issues#3211
2017-10-29 02:58:37 +02:00
Marek Marczykowski-Górecki
f89d67c3e3
backup: improve migrating templates from Qubes 3.x
In Qubes 3.x any PV template had qrexec-agent and gui-agent installed.
Adjust features appropriately while restoring them.
2017-10-19 14:54:30 +02:00
Marek Marczykowski-Górecki
513163276f
backup: do not try to read tar's stderr past EOF
If EOF is reached on tar's stderr, stop reading it, even if didn't found
expected data. Log this event.

This may happen when tar output some fatal error, instead of filelist.
2017-10-16 04:11:04 +02:00
Marek Marczykowski-Górecki
7a6e8d04b8
backup: improve restoring HVMs from Qubes 3.2
Set kernel to '' - boot from VM's own bootloader.
Keep it on MiniOS-based stubdomain - otherwise Windows would not boot.
2017-10-16 04:11:04 +02:00
Marek Marczykowski-Górecki
d1eab2d4f5
backup: update DispVM template property: template_for_dispvms
It was renamed from 'dispvm_allowed'.

QubesOS/qubes-issues#3047
2017-10-16 04:11:03 +02:00
Marek Marczykowski-Górecki
001b42f7d8
backup: fix name of meminfo-writer feature
It's 'service.meminfo-writer'.
2017-10-16 04:11:02 +02:00
Marek Marczykowski-Górecki
9cdb2a8152
backup: fix restoring template VMs
os.path.splitext fails on path without proper file base name, like
'/something/..000'. Use plain string methods (rsplit).

Fixes QubesOS/qubes-issues#3167
2017-10-16 03:04:59 +02:00
Marek Marczykowski-Górecki
edcaed537a
Always use QubesVM objects, instead of AppVM/TemplateVM etc
Very few calls at client side really needs VM class name. So, even in
non-blind mode use just QubesVM class, to avoid strange cases depending
on blind mode being enabled or not. Then, have VM class name in 'klass'
property. If known at object creation time, cache it, otherwise query
qubesd at first access.
2017-10-02 21:12:16 +02:00
Marek Marczykowski-Górecki
f98ea4a520
backup/restore: do not fail the whole restore when qvm-appmenus is missing
Fixes QubesOS/qubes-issues#2991
2017-08-30 16:00:00 +02:00
Marek Marczykowski-Górecki
6568d624cc
backup/restore: make backup header extraction faster
Abort tar process after extracting requested files - do not parse the
archive until the end (possibly tens of GB later).

Fixes QubesOS/qubes-issues#2986
2017-08-30 15:51:44 +02:00
Marek Marczykowski-Górecki
9210048673
backup/restore: add option to ignore size limit
Allow to restore backup which have miscalculated VMs size, but otherwise
is good.
2017-07-29 05:13:46 +02:00
Marek Marczykowski-Górecki
213760f263
backup/restore: improve error reporting 2017-07-29 04:48:46 +02:00
Marek Marczykowski-Górecki
d8af76ed60
backup: move BackupRestore class and helpers to 'restore' submodule
This breaks cyclic imports and also allow cleaner separation between
backup make and restore code.

No functional change.
2017-07-21 03:54:04 +02:00
Marek Marczykowski-Górecki
ce2215c603
backup: improve error logging
Include VM and volume name in data-related error mesages.

QubesOS/qubes-issues#1214
2017-07-17 23:32:31 +02:00
Marek Marczykowski-Górecki
f2fa613dce
backup: use 'cat' instead of read-write loop in python
The most important part is fixing resize handling - call size_func
before data_func, but after tar gets initial data (and output file
size).
But other than that, it makes the process a little faster.

QubesOS/qubes-issues#1214
2017-07-17 23:30:37 +02:00
Marek Marczykowski-Górecki
96d4a2f066
backup: change 'hvm' property to 'virt_mode'
QubesOS/qubes-issues#2912
2017-07-17 20:40:08 +02:00
Marek Marczykowski-Górecki
ced735b476
backup: do not show full stacktrace to the user
Opt for a simple one-liner error messages, instead of meaningless stack
trace (it's most of the time about qubesd responding with error, so the
stack trace of actual problem is elsewhere).
2017-07-17 20:28:23 +02:00
Marek Marczykowski-Górecki
f1036c27a7
backup: add firewall and appmenus list handling 2017-07-17 20:28:21 +02:00
Marek Marczykowski-Górecki
268a3453a9
backup: initial support for backup restore over Admin API
The code is mostly copied from core-admin.

QubesOS/qubes-issues#1214
2017-07-17 20:24:34 +02:00