Commit Graph

5773 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
41f463f2fc
Adjust test for fixed typo 2018-12-07 14:19:11 +01:00
AJ Jordan
067f86ba3f
Fix typo 2018-12-07 02:19:36 -05:00
AJ Jordan
d4e567cb10
Fix typo 2018-12-06 20:43:39 -05:00
Marek Marczykowski-Górecki
7a6e4b284f
vm/mix/net: prevent setting provides_network=false if qube is still used
Add symmetric check to the one in 'netvm' property.

Fixes QubesOS/qubes-issues#4552
2018-12-06 20:37:50 +01:00
Marek Marczykowski-Górecki
9f6c97596d
tests: updates-available notification
First install test-pkg-1.0, then add test-pkg-1.1 to repo and check if
updates-available flag is set. Then install updates and check if the
flag is cleared.

QubesOS/qubes-issues#2009
2018-12-05 06:19:34 +01:00
Marek Marczykowski-Górecki
cea9f66bc3
tests/network: reduce code duplication 2018-12-05 05:42:15 +01:00
Marek Marczykowski-Górecki
2edafacbe4
tests: listen on 'misc' socket too 2018-12-05 05:42:15 +01:00
Marek Marczykowski-Górecki
ccd7e285a1
Remove unused core2migration.py
It is part of backup-restore code, but it was moved to
core-admin-client repo.
2018-12-04 16:12:33 +01:00
Marek Marczykowski-Górecki
162196a1f8
tests: fix lookup domain by name 2018-12-04 16:12:09 +01:00
Marek Marczykowski-Górecki
91c9e49612
Drop unnecessary 'pass' to make pylint happy 2018-12-04 16:06:33 +01:00
Marek Marczykowski-Górecki
eee9964b11
Merge remote-tracking branch 'origin/release4.0' 2018-12-04 16:03:33 +01:00
Simon Gaiser
10f4329db7 version 4.0.36 2018-12-03 19:20:20 +01:00
Marek Marczykowski-Górecki
7d1bcaf64c Introduce management_dispvm property
The new property is meant for management stack (Salt) to set which DVM
template should be used to maintain given VM. Since the DispVM based on
it will be given ultimate control over target VM (qubes.VMShell
service), it should be trusted. The one pointed to by default_dispvm
not necessary is one.

The property defaults to the value from the template (if any), and then
to a global management_dispvm property. By default it is set to None.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-12-03 19:18:26 +01:00
Marek Marczykowski-Górecki
79d48ab4c0
rpm: drop uneeded dependencies:
gnome-packagekit: no longer needed, especially not in this package;
qubes-dom0-update tool is in core-dom0-admin-linux, but preffer a `dnf
update` in a terminal emulator

bsdtar: was used by 3.x DispVM implementation, now isn't used anywhere
2018-11-30 00:29:14 +01:00
Marek Marczykowski-Górecki
d1e20e5178
Merge remote-tracking branch 'origin/pr/242'
* origin/pr/242:
  Drop async future check with python3 version
  tracis-ci: add support for fc29 dom0
2018-11-24 14:43:43 +01:00
fepitre
313a22d4d6 Drop async future check with python3 version
QubesOS/qubes-issues#4225
2018-11-22 19:34:58 +01:00
Frédéric Pierret (fepitre)
2d89ff479b tracis-ci: add support for fc29 dom0 2018-11-22 19:34:58 +01:00
Marek Marczykowski-Górecki
0407dc1ac5
version 4.0.35 2018-11-21 03:05:10 +01:00
Marek Marczykowski-Górecki
087a02c7f4
ext/services: add automatic migration meminfo-writer=False -> maxmem=0
Migrate meminfo-writer=False service setting to maxmem=0 as a method to
disable dynamic memory management. Remove the service from vm.features
dict in the process.

Additionally, translate any attempt to set the service.meminfo-writer
feature to either setting maxmem=0 or resetting it to the default (which
is memory balancing enabled if supported by given domain). This is to at
least partially not break existing tools using service.meminfo-writer as
a way to control dynamic memory management. This code does _not_ support
reading service.meminfo-writer feature state to get the current state of
dynamic memory management, as it would require synchronizing with all
the factors affecting its value. One of main reasons for migrating to
maxmem=0 approach is to avoid the need of such synchronization.

QubesOS/qubes-issues#4480
2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
62bc462a23
tests: default maxmem 2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
b8052f864a
tests: more cases for libvirt xml generation
Related to automatic mem balance enabling/disabling. Check how it behave
in presence of PCI devices, or explicit disabling it.
2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
4dc8631010
Use maxmem=0 to disable qmemman, add more automation to it
Use maxmem=0 for disabling dynamic memory balance, instead of cryptic
service.meminfo-writer feature. Under the hood, meminfo-writer service
is also set based on maxmem property (directly in qubesdb, not
vm.features dict).
Having this as a property (not "feature"), allow to have sensible
handling of default value. Specifically, disable it automatically if
otherwise it would crash a VM. This is the case for:
 - domain with PCI devices (PoD is not supported by Xen then)
 - domain without balloon driver and/or meminfo-writer service

The check for the latter is heuristic (assume presence of 'qrexec' also
can indicate balloon driver support), but it is true for currently
supported systems.

This also allows more reliable control of libvirt config: do not set
memory != maxmem, unless qmemman is enabled.

memory != maxmem only makes sense if qmemman for given domain is
enabled.  Besides wasting some domain resources for extra page tables
etc, for HVM domains this is harmful, because maxmem-memory difference
is made of Popupate-on-Demand pool, which - when depleted - will kill
the domain. This means domain without balloon driver will die as soon
as will try to use more than initial memory - but without balloon driver
it sees maxmem memory and doesn't know about the lower limit.

Fixes QubesOS/qubes-issues#4135
2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
35a53840f1
vm: send domain-start-failed event also if some device is missing
Checking device presence wasn't covered with try/except that send the
event.
2018-11-15 18:25:29 +01:00
Marek Marczykowski-Górecki
0eab082d85
ext/core-features: make 'template-postinstall' event async
It makes a lot of sense to call long-running operations in that event
handler, including calling back into the VM. Allow that by using
fire_event_async, not just fire_event.

Also, document the event.
2018-11-15 18:25:29 +01:00
Marek Marczykowski-Górecki
d2585aa871
tests/lvm: fix checking lvm pool existence cont.
Commit 15cf593bc5 "tests/lvm: fix checking
lvm pool existence" attempted to fix handling '-' in pool name by using
/dev/VG/LV symlink. But those are not created for thin pools. Change
back to /dev/mapper, but include '-' mangling.

Related QubesOS/qubes-issues#4332
2018-11-15 18:25:29 +01:00
Marek Marczykowski-Górecki
f023b3dd6e
backup: fix naming qubes.xml.000 in the archive
Restore old code for calculating subdir within the archive. The new one
had two problems:
 - set '/' for empty input subdir - which caused qubes.xml.000 to be
 named '/qubes.xml.000' (and then converted to '../../qubes.xml.000');
 among other things, this results in the wrong path used for encryption
 passphrase
 - resolved symlinks, which breaks calculating path for any symlinks
 within VM's directory (symlinks there should be treated as normal files
 to be sure that actual content is included in the backup)

This partially reverts 4e49b951ce.

Fixes QubesOS/qubes-issues#4493
2018-11-15 18:25:29 +01:00
Marek Marczykowski-Górecki
85a20428a6
libvirt: allow skipping hardcoded kernelopts
Add 'no-default-kernelopts' feature to skip default hardcoded
Linux-specific kernelopts.
This is especially useful for non-Linux VMs (including Mirage OS).

Fixes QubesOS/qubes-issues#4468
2018-11-15 17:54:26 +01:00
Marek Marczykowski-Górecki
328697730b
vm: fix deadlock on qrexec timeout handling
vm.kill() will try to get vm.startup_lock, so it can't be called while
holding it already.
Fix this by extracting vm._kill_locked(), which expect the lock to be
already taken by the caller.
2018-11-04 17:05:55 +01:00
Marek Marczykowski-Górecki
68dffb6895
api/admin: fix error message when refusing to create template on template
Fixes QubesOS/qubes-issues#4463
2018-11-04 17:05:55 +01:00
Marek Marczykowski-Górecki
e8201eebba
version 4.0.34 2018-11-01 22:31:18 +01:00
Marek Marczykowski-Górecki
64f290c9ba
ext/pci: fix error message about missing device
Print human readable device name, instead of "<PCIDevice at ...".

QubesOS/qubes-issues#4461
2018-11-01 22:28:50 +01:00
Marek Marczykowski-Górecki
00ca0459d9
ext/pci: use correct backend domain for getting PCIDevice instance
In practice backend_domain is ignored (all PCI devices belongs to dom0),
but lets fix this anyway.
2018-11-01 22:21:50 +01:00
Marek Marczykowski-Górecki
15cf593bc5
tests/lvm: fix checking lvm pool existence
If pool or group name have '-', it will be mangled as '--' in
/dev/mapper. Use /dev/VG_NAME/LV_NAME symlink instead.

Related QubesOS/qubes-issues#4332
2018-10-30 01:17:00 +01:00
Marek Marczykowski-Górecki
1ae6abdff5
exc: fix QubesMemoryError constructor
QubesVMError require 'vm' argument.
Fixes 2f3a9847 "exc: Make QubesMemoryError inherit from QubesVMError"
2018-10-30 01:14:58 +01:00
Marek Marczykowski-Górecki
b9a18a819c
Merge remote-tracking branch 'origin/pr/239'
* origin/pr/239:
  storage: fix NotImplementedError message for import_data()
  storage/reflink: make resize()/import_volume() more readable
  storage/reflink: unblock import_data() and import_data_end()
2018-10-29 23:00:04 +01:00
Marek Marczykowski-Górecki
fa2429aae4
Merge remote-tracking branch 'origin/pr/237'
* origin/pr/237:
  progress thresold removed as Marek suggested
  Avoid progress events flooding

Fixes QubesOS/qubes-issues#4406
Fixes QubesOS/qubes-issues#3035
2018-10-29 22:54:05 +01:00
Marek Marczykowski-Górecki
f30963fde1
tests/qubespolicy: adjust for removing 'assert' usage 2018-10-29 22:37:15 +01:00
Rusty Bird
7e4812a525
storage: fix NotImplementedError message for import_data() 2018-10-29 20:21:42 +00:00
Rusty Bird
73db2751b8
storage/reflink: make resize()/import_volume() more readable 2018-10-29 20:21:41 +00:00
Rusty Bird
425d993769
storage/reflink: unblock import_data() and import_data_end() 2018-10-29 20:21:39 +00:00
Marek Marczykowski-Górecki
f621e8792c
Merge branch 'master' into devel-no-assert 2018-10-29 20:29:53 +01:00
Marek Marczykowski-Górecki
3740e2d48b
api: make enforce() static
It doesn't use 'self'. And pylint complains.
2018-10-29 20:22:35 +01:00
Marek Marczykowski-Górecki
db6094f397
tests/api: adjust for proper exceptions instead of AssertionError 2018-10-29 20:22:10 +01:00
Marek Marczykowski-Górecki
2b5fc6299e
tests/api: do not test non-existing methods
Remove methods not included in specification (or with different
constraints). Keep commented out methods included in spec but not
implemented.
2018-10-29 20:21:36 +01:00
Marek Marczykowski-Górecki
c1c6dc2acd
Use ValueError in PropertyHolder.property_require()
Specifically do not use AssertionError, but also be consistent with
other value verification.
2018-10-29 20:16:41 +01:00
Marek Marczykowski-Górecki
114a9db09a
version 4.0.33 2018-10-29 05:45:56 +01:00
Marek Marczykowski-Górecki
26a553737f
storage/lvm: minor fix for lvs command building
Do not prepend 'sudo' each time - do a copy of array if that's
necessary.
2018-10-29 05:16:23 +01:00
Marek Marczykowski-Górecki
42061cb194
tests: try to collect qvm-open-in-dvm output if no editor window is shown
Try to collect more details about why the test failed. This will help
only if qvm-open-in-dvm exist early. On the other hand, if it hang, or
remote side fails to find the right editor (which results in GUI error
message), this change will not provide any more details.
2018-10-29 01:20:57 +01:00
Marek Marczykowski-Górecki
84c321b923
tests: increase session startup timeout for whonix-ws based VMs
First boot of whonix-ws based VM take extended period of time, because
a lot of files needs to be copied to private volume. This takes even
more time, when verbose logging through console is enabled. Extend the
timeout for that.
2018-10-27 16:44:53 +02:00
Marek Marczykowski-Górecki
84d3547f09
tests: adjust extra tests loader to work with nose2
Nose loader do not provide loader.loadTestsFromTestCase(), use
loader.loadTestsFromNames() instead.
2018-10-27 16:44:53 +02:00