Commit Graph

287 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
4e022382a5
Merge remote-tracking branch 'origin/master' into core3-devel 2016-08-08 00:11:46 +02:00
Marek Marczykowski-Górecki
c028df3e1e
QubesWatch: fix handling just removed domains
Really do not throw exception in such a case.
Reported by HW42.
2016-07-18 15:07:02 +02:00
HW42
b467dd6218
QubesWatch: do not create multiple dom0 QubesDB connections
When calling _register_watches() multiple times for dom0 (by passing
None or since 7e9c816b by passing the corresponding libvirt domain) the
check was missing if there is already a QubesDB in _qdb. Therefore a new
QubesDB was created and the old one is destroyed by the GC. As a
consequence the watch_fd is closed but the libvirt event handle for this
fd is still registered. So when libvirt calls poll() it returns
immediately POLLNVAL with the closed fd. This is not caught in libvirt
and the callback is called as if an event happened. _qdb_handler() now
calls read_watch() on the new fd for dom0 and thereby hangs the thread.
This leads (at leads) to qubes-manager to miss VM status updates and
block device events.

Fixes QubesOS/qubes-issues#2178
2016-07-18 15:03:14 +02:00
Marek Marczykowski-Górecki
6068c7bc50
core: fix handling uses_default_netvm property
- for netvm it doesn't make sense, but instead of removing it (which
        surely will break some code), make it always False
- when settings VM connections, uses_default_netvm is already loaded
- handle it properly during backup restore (really use default netvm,
        istead of assuming it's the same as during backup)
2016-07-17 03:56:11 +02:00
Marek Marczykowski-Górecki
8222324146
backup: restore 'services' property
Fixes QubesOS/qubes-issues#2106
2016-07-17 03:56:10 +02:00
Marek Marczykowski-Górecki
c5d03cc32e
backup: ask user to update templates after restoring backup
To install all the fixes and to update applications + icons.

Fixes QubesOS/qubes-issues#2150
2016-07-17 03:56:10 +02:00
Marek Marczykowski-Górecki
f15ddf7801
backup: fix restoring on system without default netvm
Fixes QubesOS/qubes-issues#2168
2016-07-14 02:06:08 +02:00
Marek Marczykowski-Górecki
69e792d82c
usb_attach: improve error reporting
- specifically report when qubes-usb-proxy package is not installed
- include sanitized stderr output in error message
2016-07-12 06:27:53 +02:00
Marek Marczykowski-Górecki
7e9c816b7b
qubeswatch: use always "dom0" name for dom0
Libvirt reports dom0 as "Domain-0". Which is incompatible with how Qubes
and libxl toolstack names it ("dom0"). So handle this as a special case.
Otherwise reconnection retries leaks event object every iteration.

Fixes QubesOS/qubes-issues#860
Thanks @alex-mazzariol for help with debugging!
2016-07-01 19:16:07 +02:00
Wojtek Porczyk
6a10daf7be Merge branch 'master' into core3-devel
Remains to be fixed:
  88cb62fc
  d2640b51
  958c2926
2016-06-13 19:03:46 +02:00
Marek Marczykowski-Górecki
767d1f0074
qvm-usb: implement usb_detach_all
QubesOS/qubes-issues#531
2016-06-02 02:49:22 +02:00
Marek Marczykowski-Górecki
d5e06bfb83
qvm-usb: issue detach call to backend domain
Make sure that even compromised frontend will be cut of (possibly
sensitive - like a webcam) device. On the other hand, if backend domain
is already compromised, it may already compromise frontend domain too,
so none of them would be better to call detach to.

QubesOS/qubes-issues#531
2016-06-02 02:45:26 +02:00
Marek Marczykowski-Górecki
52fb410deb
qvm-usb: always pass VM as object reference not a name
Make the API consistent.

QubesOS/qubes-issues#531
2016-06-02 02:44:38 +02:00
Marek Marczykowski-Górecki
d67636308f
qvm-usb: modify for USBIP-over-qrexec implementation
QubesOS/qubes-issues#531
2016-05-26 01:38:08 +02:00
Marek Marczykowski-Górecki
7e76342919
backup: default tmpdir to /tmp, respect TMPDIR
This requires having at least 1GB free on /tmp, but it is fair
assumption - it's tmpfs in dom0 and while performing the backup most of
the VMs aren't running, so shouldn't be a problem. Anyway it is always
possible to set TMPDIR variable or pass --tmpdir cmdline option.
Using tmpfs based temporary directory should speedup the backup.

QubesOS/qubes-issues#1652
2016-05-05 00:00:25 +02:00
Marek Marczykowski-Górecki
a8ea1d4b39
Merge remote-tracking branch 'origin/pr/10'
* origin/pr/10:
  backup: Allow to specify custom temporary directory
2016-05-04 23:54:34 +02:00
Marek Marczykowski-Górecki
e0686e1e02 backup: initial conversion to core3 API 2016-04-07 13:05:25 +02:00
o
958c292607 fix qvm-ls display of cpu_time
qhost.measure_cpu_usage expects the qvm_collection as parameter. Also
the number of vcpus of dom0 seems to be 0, leading to a div by 0. A more
complete fix would probably involve e.g. a new num_cores property which
would contain number of vcpu for vhosts and number of actual cores for
dom0.

For now this is a partial solution.
2016-04-04 15:30:22 +02:00
Axon
01b667427f Nicely print list of excluded VMs
Prints a sorted list of VMs not selected for backup, one name per line.
(Fixes QubesOS/qubes-issues#1023)
2016-03-14 12:13:04 +00:00
Wojtek Porczyk
9eafa57539 Merge remote-tracking branch 'marmarek/master' into core3-devel
The following list is bollocks. There were many, many more.

Conflicts:
    core-modules/003QubesTemplateVm.py
    core-modules/005QubesNetVm.py
    core/qubes.py
    core/storage/__init__.py
    core/storage/xen.py
    doc/qvm-tools/qvm-pci.rst
    doc/qvm-tools/qvm-prefs.rst
    qubes/tools/qmemmand.py
    qvm-tools/qvm-create
    qvm-tools/qvm-prefs
    qvm-tools/qvm-start
    tests/__init__.py
    vm-config/xen-vm-template-hvm.xml

This commit took 2 days (26-27.01.2016) and put our friendship to test.
    --Wojtek and Marek
2016-03-03 01:13:51 +01:00
Marek Marczykowski-Górecki
d2640b517f
backup: Allow to specify custom temporary directory
Using tmpfs-backed directory may speed up the backup process.

QubesOS/qubes-issues#1652
2016-01-20 03:35:35 +01:00
Marek Marczykowski-Górecki
8b9049d8c6
core/hvm: create root-cow.img if it's missing
It should be created at VM creation time (or template changes commit).
But for example for HVM templates created before implementing
QubesOS/qubes-issues#1573, there would be no such image. So create it
when needed, just before VM startup

Fixes QubesOS/qubes-issues#1602
2016-01-11 21:53:33 +01:00
Marek Marczykowski-Górecki
ba3b908b73
backup: do not dereference symlinks in dom0 user home backup
Symlinks there should be preserved. Additionally broken symlink would
cause the backup to fail.

Fixes QubesOS/qubes-issues#1515
2016-01-07 05:48:33 +01:00
Marek Marczykowski-Górecki
35cb82cd2b
storage: use root-cow.img for HVM template, to support reverting changes
We still can't support running HVM template and its VMs simultaneously
(easily), but still, have root-cow.img handled for HVM template, to
allow qvm-revert-template-changes.

Fixes QubesOS/qubes-issues#1573
2016-01-07 05:48:32 +01:00
Marek Marczykowski-Górecki
fa196f1350
storage: actually use storage.root_img and such properties
Actual VM root disk used hardcoded image names, instead of properties
set for that purpose...
2016-01-07 05:48:32 +01:00
Bahtiar `kalkin-` Gadimov
c8d26c253e Storage refactor device config generation
- QubesVmStorage provides now a default get_config_params() method which should
  be enough for all possible Storage implementations.
- When writing a custom Storage implementation, one has just to reimplement the
  following methods:
  * root_dev_config()
  * private_dev_config()
  * volatile_dev_config()
- QubesVmStorage provides a default implementation of other_dev_config(),
  because it can be shared by all storage implementations
2015-12-22 11:24:28 +01:00
Marek Marczykowski-Górecki
b114d291ca
Merge remote-tracking branch 'qubesos/pr/12'
* qubesos/pr/12:
  Fix circular deps workaround in Pool.vmdir_path()
  Move device names from XenStorage to QubesVmStorage
  Provide method format_disk_dev() to all storages
  Move the vmdir logic from XenPool to Pool
2015-11-27 20:00:37 +01:00
Marek Marczykowski-Górecki
24d660d61e
backup: add option to on-the-fly renaming conflicting VMs
Fixes QubesOS/qubes-issues#869
2015-11-27 20:00:34 +01:00
Marek Marczykowski-Górecki
2fdbf51ccc
core: have QubesVmCollection.unlock_db no-op if not locked 2015-11-27 20:00:19 +01:00
Bahtiar `kalkin-` Gadimov
f5cef35cdf Fix circular deps workaround in Pool.vmdir_path() 2015-11-25 16:01:28 +01:00
Bahtiar `kalkin-` Gadimov
85421e3f48 Move device names from XenStorage to QubesVmStorage 2015-11-25 12:49:56 +01:00
Bahtiar `kalkin-` Gadimov
9b23576ff6 Provide method format_disk_dev() to all storages
The method XenStorage._format_disk_dev() generates the xml config for a device.
It is not specific to the Xen file storage implementation. It can and must be
reused by other storage implementations
2015-11-25 12:48:47 +01:00
Bahtiar `kalkin-` Gadimov
feaaaa75fa Move the vmdir logic from XenPool to Pool
Any storage implementation needs this logic for saving the vm config and
`*.desktop` files.
2015-11-25 12:47:25 +01:00
Marek Marczykowski-Górecki
0c476f014d
backup: avoid deadlock when VM process fails
If SendWorker queue is full, check if that thread is still alive.
Otherwise it would deadlock on putting an entry to that queue.
This also requires that SendWorker must ensure that the main thread
isn't currently waiting for queue space when it fails. We can do this by
simply removing an entry from a queue - so on the next iteration
SendWorker would be already dead and main thread would notice it.
Getting an entry from queue in such (error) situation is harmless,
because other checks will notice it's an error condition.

Fixes QubesOS/qubes-issues#1359
2015-11-25 01:35:04 +01:00
Bahtiar `kalkin-` Gadimov
885dc5cd81 Rename storage dir to dir_path, type to driver
Bad idea to use attributes which are named after built in functions.
2015-11-21 14:46:23 +01:00
Bahtiar `kalkin-` Gadimov
9eee00c6d7 QubesNetVm and subclasses use servicevm/ as vmdir 2015-11-17 21:21:02 +01:00
Bahtiar `kalkin-` Gadimov
1934f06869 XenStorage add DisposableVm handling 2015-11-17 21:15:13 +01:00
Bahtiar `kalkin-` Gadimov
76224dac86 Path to the Vm images is set by the storage
- This moves the logic for setting the path to the storage specific class like
  XenStore
2015-11-17 19:37:20 +01:00
Bahtiar `kalkin-` Gadimov
0bc0bc9045 XenStorage make sure subdirs exist in pool dir 2015-11-17 19:37:20 +01:00
Bahtiar `kalkin-` Gadimov
58f23ca392 Add configurable pool_dir to XenPool 2015-11-17 19:37:20 +01:00
Bahtiar `kalkin-` Gadimov
16d480cf4c Add storage add_pool & remove_pool 2015-11-17 19:37:19 +01:00
Bahtiar `kalkin-` Gadimov
26711e7e9a Use default pool_config if pool name is 'default' 2015-11-17 19:37:19 +01:00
Bahtiar `kalkin-` Gadimov
78891dd70f QubesVm save the name of the storage pool used 2015-11-17 19:37:19 +01:00
Bahtiar `kalkin-` Gadimov
d1685a13df Add storage.pool_exists 2015-11-17 19:37:19 +01:00
Bahtiar `kalkin-` Gadimov
8e0207a199 Rename QubesXenVmStorage to XenStorage 2015-11-17 19:37:19 +01:00
Bahtiar `kalkin-` Gadimov
bfaf37dae5 Add pool config parsing 2015-11-17 19:37:18 +01:00
Bahtiar `kalkin-` Gadimov
15d5e6edbb Add Pool & XenPool 2015-11-17 19:37:18 +01:00
Bahtiar `kalkin-` Gadimov
7550fccf94 Removed WNI 2015-11-17 19:37:18 +01:00
Rusty Bird
3e3ac36f58
cp --reflink=auto to speed up clone/backup/restore on btrfs 2015-11-10 17:10:03 +00:00
Marek Marczykowski-Górecki
f714a8224b
backup: follow symbolic links
For a long time Qubes backup did not include symlinked files, which
apparently is quite common practice for users with multiple disks (for
example HDD + SSD). It is covered in documentation
(https://www.qubes-os.org/doc/secondary-storage/), but better solution
would be to simply include symlinked files.

Restore of such files would (of course) not preserve the symlinks -
normal files will be restored instead. But that's fine. If the user want
to move the data to another location, he/she can do that and restore the
symlink.

The only possible breakage from this change is having a copy (instead of
symlink) to a VM icon. But storing that symlink in a backup was broken
for some time (because of --xform usage) and it is handled during
restore, so not a real problem.

This doesn't cover all the problems with symlinked VM images - the other
one is qvm-block behaviour, which would treat such images as non-system
disks, so easily detachable (which would break VM operation). But that's
another story.

Fixes QubesOS/qubes-issues#1384
2015-11-07 05:03:54 +01:00