Commit Graph

1268 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
ed3001da99
tests: fix asyncio usage cont...
- Add missing loop.run_until_complete() calls.
- Convert subprocess.Popen to asyncio.create_subprocess_exec where
needed (when called process needs to communicate with qubesd).
- Cleanup processes (call .wait()).
2017-09-29 11:53:58 +02:00
Marek Marczykowski-Górecki
f0da5b21df
tests: improve "revert template changes" test
There is no more significant difference between PV and HVM. VMs are HVM
by default anyway. More important for this test is difference between
Linux (with Qubes packages installed) and other OS-es. Rename tests
accordingly. The later one is still incomplete.
2017-09-29 11:53:58 +02:00
Marek Marczykowski-Górecki
4b51c9da07
tests/storage: don't fail the test when directory already exists
/var/tmp/test-pool should normally be removed by previous test cleanup,
but do not fail next test if it wasn't.
2017-09-29 11:53:58 +02:00
Marek Marczykowski-Górecki
344bb16ae2
tests: fix various object leaks at tests framework level
The most important change is doing vm.close() when removing domain -
this means it wouldn't be cleaned later by iterating over app.domains.

Other changes include removing VMs in the right order, regarding netvm
dependency (otherwise killing or removing may fail). And one more
missing coroutine handling (in shutdown_and_wait).
2017-09-29 11:53:58 +02:00
Marek Marczykowski-Górecki
a929916ce8
tests/storage: do not depend on system default storage pool
Use just created storage pool for tests, instead of system default
(which depends on system configuration).
2017-09-29 11:53:57 +02:00
Marek Marczykowski-Górecki
d8ff85eee6
tests: fix cleanup function
Don't crash when directory to cleanup do not exists.
2017-09-29 11:53:57 +02:00
Marek Marczykowski-Górecki
3075767bb8
tests: add more complex test for netvm cyclic connection
Something more complex than VM connecting to itself.
2017-09-29 11:53:57 +02:00
Marek Marczykowski-Górecki
e0cadfdd67
Merge remote-tracking branch 'qubesos/pr/150'
* qubesos/pr/150:
  qubes/tests: moar fixes
  test-packages: add missing libvirt classes
  qubes/tests: do not deadlock on .drain()
  qubes/vm: put name= first in __repr__
  tests: fix some memory leaks
  tests: complain about memory leaks
  tests: use one event loop and one libvirtaio impl
2017-09-22 20:54:11 +02:00
Wojtek Porczyk
589d567ebc qubes/tests: moar fixes 2017-09-21 14:18:35 +02:00
Wojtek Porczyk
b78ad1993c qubes/tests: do not deadlock on .drain() 2017-09-21 14:17:36 +02:00
Wojtek Porczyk
71eefbea14 qubes/vm: put name= first in __repr__
This is to have it in case the repr is truncated.
2017-09-19 17:02:19 +02:00
Wojtek Porczyk
bd123a3812 tests: fix some memory leaks
This is work in progress. There are still more leaks.
2017-09-19 17:01:29 +02:00
Wojtek Porczyk
5aa9fa2db4 tests: complain about memory leaks
Currently this detects leaking VM (and all subclasses), Qubes and
libvirt-related objects. Usable only with --failfast.
2017-09-19 16:59:09 +02:00
Wojtek Porczyk
7df8f51011 tests: use one event loop and one libvirtaio impl
Recently libvirt removed support for changing event implementation.
Therefore we have to use a single, global one and we check if it is
empty between tests.
2017-09-19 16:47:14 +02:00
HW42
c6c6f681ae tests: Fix expected HVM xml
The last two commits (intentionally) changed the libxl xml. So adapt the
expected value in the test.
2017-09-15 16:19:10 +02:00
Marek Marczykowski-Górecki
86681ba996
Merge remote-tracking branch 'qubesos/pr/149'
* qubesos/pr/149:
  lvm->init_cache: fix incorrect max splits
  Fix service qubesd not working with some locales
2017-09-14 00:15:33 +02:00
Marek Marczykowski-Górecki
55208a0c7c
Merge branch 'dispvm-policy' 2017-09-14 00:15:29 +02:00
Nicolai Søborg
9d797cc864
lvm->init_cache: fix incorrect max splits 2017-09-13 18:50:02 +02:00
Nicolai Søborg
c30131633b Fix service qubesd not working with some locales
On certain locales (e.g. danish) `usage_percent` will output a comma-separated number, which will make `attr` point the last two decimal points, s.t. `return vol_info['attr'][4] == 'a'` (in the `verify` func) will fail and `qubesd` wont run.
2017-09-12 01:57:49 +02:00
Marek Marczykowski-Górecki
0d59965a7b
Merge branch 'cdrom-boot'
* cdrom-boot:
  devices: fix error reporting
  api/admin: implement admin.vm.device....Set.persistent
  devices: implement DeviceCollection.update_persistent()
  devices: move DeviceInfo definition earlier
  api: do not fail events when listener is cancelled in the meantime
2017-09-07 13:02:04 +02:00
Marek Marczykowski-Górecki
99bd193688
Rename 'dispvm_allowed' to 'template_for_dispvms'
'dispvm_allowed' name was confusing, because it suggested being able to
spawn new DispVMs, not being a template for DispVM.

Fixes QubesOS/qubes-issues#3047
2017-09-05 02:07:26 +02:00
Marek Marczykowski-Górecki
c247ddff72
dispvm: fix setting up new DispVM
Clone properties from DispVM template after setting base properties
(qid, name, uuid). This means we can use standard clone_properties()
function. Otherwise various setters may fail - for example
netvm setter require uuid property initialized (for VM lookup in VM
collection).
Also, make dispvm_allowed check more robust - include direct creation of
DispVM, and also check just before VM startup (if property was changed
in the meantime).

Fixes QubesOS/qubes-issues#3057
2017-09-05 02:07:26 +02:00
Marek Marczykowski-Górecki
687ca4092c
tests: various fixes to integration tests
Mostly add missing self.loop.run_until_complete().
2017-09-05 02:01:58 +02:00
Marek Marczykowski-Górecki
d676576600
tests: fix qvm-sync-clock test for asyncio and new time sync API
qvm-sync-clock in dom0 now synchronize only dom0 time. For VM time,
qvm-sync-clock needs to be called in VM. Also, both will communicate
with qubesd, so must be called asynchronously from tests.
2017-09-05 02:01:58 +02:00
Marek Marczykowski-Górecki
32b57deb3a
devices: fix error reporting 2017-09-05 01:33:47 +02:00
Marek Marczykowski-Górecki
cea70748a6
api/admin: implement admin.vm.device....Set.persistent
This will allow converting persistent device (used to boot VM for
example) to non-persistent.

QubesOS/qubes-issues#3055
2017-09-05 01:33:47 +02:00
Marek Marczykowski-Górecki
9d062c4c66
devices: implement DeviceCollection.update_persistent()
Allow attached device to be converted from persistent to non-persistent
and the other way around.
This is to allow starting a VM with some device attached temporarily.
When VM is not running, it is possible to attach device only
persistently, so this change will allow to do that, then, after starting
the VM, change it to non-persistent - so it will not be attached again
at further startups.

QubesOS/qubes-issues#3055
2017-09-05 01:33:47 +02:00
Marek Marczykowski-Górecki
1f5d43a094
devices: move DeviceInfo definition earlier
Make it in logical order. And allow DeviceAssignment and
DeviceCollection use DeviceClass type, for example in annotations.
2017-09-05 01:33:46 +02:00
Marek Marczykowski-Górecki
8c1ba905a0
api: do not fail events when listener is cancelled in the meantime
Raising any error from event handler will abort further handlers of this
event.
2017-09-05 01:33:46 +02:00
Wojtek Porczyk
6a93137bbe qubes/app: remove .app attr from VMMConnection 2017-09-01 19:46:28 +02:00
Wojtek Porczyk
029ec9cb38 fix some more fd leaks 2017-08-31 20:30:24 +02:00
Wojtek Porczyk
6ff1bfdc16 qubes/tests: convert some tearDown into addCleanup
This is because .tearDown() is not executed if the exception occurs in
setUp() [for example self.skipTest() raises an exception]. The lower
levels of .tearDown() being executed are critical to not leaking file
descriptors.
2017-08-31 20:30:24 +02:00
Wojtek Porczyk
d0f2fdba55 qubes/tests: fix some of the fd leaks
This is WIP, there are more fds leaking.
2017-08-31 20:30:24 +02:00
Wojtek Porczyk
f1334e5095 qubes/tests/run: --break-to-repl 2017-08-31 20:30:24 +02:00
Wojtek Porczyk
25670fbd61 qubes.tests.list_templates(), and .close() the app 2017-08-31 20:30:23 +02:00
Wojtek Porczyk
de8ff20976 qubes: Allow for explicit closing of objects 2017-08-31 20:30:23 +02:00
Wojtek Porczyk
8547d06cc9 qubes/tests: skip env-dependent tests using decorator
Gtk segfaults without X11 display. Some tests need dedicated PCI device.
2017-08-31 20:29:34 +02:00
Wojtek Porczyk
044e10a6ec Merge remote-tracking branches 'origin/pull/142/head', 'origin/pull/143/head' and 'origin/pull/144/head' 2017-08-14 16:32:56 +02:00
Marek Marczykowski-Górecki
3f33a7bc2c
fix and enable ServicesExtension
This extension is responsible for communicating service.* features to
VMs - in other words, qvm-service framework

Fixes QubesOS/qubes-issues#3019
2017-08-14 02:30:52 +02:00
Marek Marczykowski-Górecki
7a9e5c3650
tests: qrexec-policy cli tool tests 2017-08-14 02:24:32 +02:00
Marek Marczykowski-Górecki
9f7915d998
Forbid creating VM with name 'none' or 'default'
Those were special names in Qubes 3.x, don't allow such VMs to avoid
(potentially fatal) confusion.

Fixes QubesOS/qubes-issues#3002
2017-08-14 02:24:32 +02:00
Marek Marczykowski-Górecki
6e5fe58128
storage/lvm: fix Volume() instance init when physical volume is unavailable
First, cache objects created with init_volume - this is the only place
where we have full volume configuration (including snap_on_start and
save_on_stop properties).
But also implement get_volume method, to get a volume instance for given
volume id. Such volume instance may be incomplete (other attributes are
available only in owning domain configuration), but it will be enough
for basic operations - like cheching and changing its size, cloning
etc.
Listing volumes still use list of physically present volumes.

This makes it possible to start qubesd service, without physical
presence of some storage devices. Starting VMs using such storage would
still fail, of course.

Fixes QubesOS/qubes-issues#2960
2017-08-12 22:44:03 +02:00
Marek Marczykowski-Górecki
9ad85a3dff
storage: fix getting list of pool driver parameters
function.func_code existed in python2. Use inspect module (instead of
__code__ attribute).
2017-08-12 22:42:30 +02:00
Marek Marczykowski-Górecki
3338854b1c
storage: fix storage.import_data_end when given Volume instance
Leftover from '0f12870 storage: use direct object references, not only
identifiers'
2017-08-12 22:39:41 +02:00
Marek Marczykowski-Górecki
0cd25b5350
firewall: fix Firewall.clone()
New rule require action to be defined, even if will be overwritten a
moment later.
2017-08-06 20:54:10 +02:00
Marek Marczykowski-Górecki
a833692c63
storage/lvm: extend volume snapshot of running VM too
Extend both backing volume as well as current snapshot, so the increased
size is visible immediately to the user.

Fixes QubesOS/qubes-issues#2982
2017-08-06 20:54:10 +02:00
Marek Marczykowski-Górecki
8488d5dc72
vm/adminvm: add default_dispvm property to AdminVM
This is useful to select default DispVM template for VMs started
directly by the user. This makes sense as long as AdminVM == GUIVM.

QubesOS/qubes-issues#2974
2017-08-06 20:54:10 +02:00
Marek Marczykowski-Górecki
971c7d4ac9
api/admin: add admin.vm.CreateDisposable in place of internal.vm.Create.DispVM
Add public Admin API call to create Disposable VM that would be
automatically destroyed after shutdown. Do not keep this functionality
for qrexec-policy tool only.
Also, use admin.vm.Start there, instead of internal.vm.Start and
admin.vm.Kill instead of internal.vm.CleanupDispVM (this is enough,
because DispVM now have auto_cleanup property).

QubesOS/qubes-issues#2974
2017-08-06 20:54:10 +02:00
Marek Marczykowski-Górecki
691a6f4d8c
vm/dispvm: add auto_cleanup property, unify creating new DispVM
Add auto_cleanup property, which remove DispVM after its shutdown
- this is to unify DispVM handling - less places needing special
handling after DispVM shutdown.

New DispVM inherit all settings from respective AppVM. Move this from
classmethod `DispVM.from_appvm()`, to DispVM constructor. This unify
creating new DispVM with any other VM class.
Notable exception are attached devices - because only one running VM can
have a device attached, this would prevent second DispVM started from
the same AppVM. If one need DispVM with some device attached, one can
create DispVM with auto_cleanup=False. Such DispVM will still not have
persistent storage (as any other DispVM).

Tests included.

QubesOS/qubes-issues#2974
2017-08-06 20:54:09 +02:00
Marek Marczykowski-Górecki
22f2fe6d69
block: add support for devtype option
QubesOS/qubes-issues#2951
2017-08-01 15:20:36 +02:00