Commit Graph

662 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
b6f77ebfa1
tests: do not use lazy unmount
If unmount is going to fail, let it do so explicitly, instead of hiding
the failure now, and observing it later at rmdir.
And if it fails, lets report what process is using that mount point.
2019-03-16 03:06:24 +01:00
Marek Marczykowski-Górecki
65bdff1fdf
tests: re-enable policy prompt tests in travis
Xenial environment has much newer GTK/Glib. For those test to run, few
more changes are needed:
 - relevant GTK packages installed
 - X server running (otherwise GTK terminate the process on module
   import...)
 - enable system side packages in virtualenv set by travis
2019-03-16 03:03:49 +01:00
Marek Marczykowski-Górecki
60bbbdd702
Merge branch 'kernelopts-files'
* kernelopts-files:
  vm: allow files in kernels_dir override built-in default kernelopts
2019-03-08 18:08:12 +01:00
Marek Marczykowski-Górecki
c093e14077
Merge branch 'libvirt-template-r41'
* libvirt-template-r41:
  libvirt: adjust domain xml template for upstream PVH format
2019-03-08 18:07:07 +01:00
Marek Marczykowski-Górecki
1ba8d7971f
tests: unit tests for internal.SuspendPre and internal.SuspendPost API 2019-02-28 00:43:33 +01:00
Marek Marczykowski-Górecki
933882ee9b
tests: fix race condition in firewall formatting test
Use fixed timestamp.
2019-02-27 15:38:22 +01:00
Marek Marczykowski-Górecki
2de5a8e894
vm,templates: allow to obtain common kernelopts from a kernel package
If kernel package ships default-kernelopts-common.txt file, use that
instead of hardcoded Linux-specific options.
For Linux kernel it may include xen_scrub_pages=0 option, but only if
initrd shipped with this kernel re-enable this option later.

QubesOS/qubes-issues#4839
QubesOS/qubes-issues#4736
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
357231ca8f
tests: network reconnect after network provider shutdown/crash
QubesOS/qubes-issues#3642
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
e69bd3c572
tests: fix race condition between mkfs and udev
Just after LVM volume is created, udev tries to analyze it. This
prevents mkfs from creating filesystem there.
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
50a89aac3f
tests: check if page scrubbing is still enabled when switching kernel
QubesOS/qubes-issues#4736
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
7433f83581
tests: add in-use storage pool removal test
QubesOS/qubes-issues#4454
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
5aa49168e5
tests: make storage.TC_00_Pool usable outside of dom0 2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
2bd709501b
tests: adjust cleanup to added in-use storage pool detection on removal 2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
d8b6d3efde
Make add_pool/remove_pool coroutines, allow Pool.{setup,destroy} as coroutines
Pool setup/destroy may be a time consuming operation, allow them to be
asynchronous. Fortunately add_pool and remove_pool are used only through
Admin API, so the change does not require modification of other
components.
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
f9593ce3e6
vm: allow files in kernels_dir override built-in default kernelopts
If default-kernelopts-pci.txt is present, it will override default
built-in kernelopts for the VMs with PCI device assigned.
Similarly if default-kernelopts-nopci.txt is present, it will override
default kernelopts for VMs without PCI devices.
For template-based VMs, kernelopts of the template takes precedence over
default-kernelopts-nopci.txt but not default-kernelopts-pci.txt.

Fixes QubesOS/qubes-issues#4839
2019-02-23 12:53:49 +01:00
Marek Marczykowski-Górecki
adcc9d7e77
libvirt: adjust domain xml template for upstream PVH format
The final PVH support upstream use different machine type name.
2019-02-19 00:59:05 +01:00
Marek Marczykowski-Górecki
bfb09f567f
Fix saving 'updates-available' flag reported by TemplateBasedVM
Look for the first updateable template up in the template chain, instead
of going just one level up. Especially this applies to
DispVM -> AppVM -> TemplateVM case.
If DispVM reports available updates, 'updates-available'
flag should be set on relevant TemplateVM, not AppVM (*-dvm).

Include test for the new case.

Fixes QubesOS/qubes-issues#3736
2019-01-19 03:25:21 +01:00
Marek Marczykowski-Górecki
18b0fa9db7
tests: improve mock setup in api_misc
Make self.src.template separate mock instance for better control over
results.
2019-01-19 03:25:20 +01:00
Marek Marczykowski-Górecki
186702646e
tests: drop expectedFailure from qubes_desktop_run test 2019-01-12 02:41:22 +01:00
Marek Marczykowski-Górecki
08621e7225
tests: grub in HVM qubes
Test grub in both PV and HVM. This change will also ease adding grub
test for PVH, when supported.
2019-01-12 02:41:22 +01:00
Marek Marczykowski-Górecki
adc3c4daa4
tests: update dom0_update for new updates available flag
Pending updates for dom0 are now signaled the same way as for templates
- with 'updates-available' feature, not a flag file.
2019-01-12 02:41:22 +01:00
Marek Marczykowski-Górecki
4c4e008368
tests: regression test LVM listing code
The revisions listing code mistakes VM name suffix with volume name.

QubesOS/qubes-issues#4680
2019-01-06 04:33:16 +01:00
Marek Marczykowski-Górecki
b08804e7c8
tests/extra: wrap ProcessWrapper.wait() to be asyncio-aware
The user of ExtraTestCase don't need to know anything about asyncio.
vm.run().wait() normally is a coroutine, but provide a wrapper that
handle asyncio.

This fixes FD leak in input proxy tests.
2019-01-06 04:33:16 +01:00
Marek Marczykowski-Górecki
0099aa1037
tests: adjust backupcompat for new maxmem handling
Since 4dc86310 "Use maxmem=0 to disable qmemman, add more automation to
it" meminfo-writer service is not accessible directly. maxmem property
is used to encode memory management instead.
2019-01-06 04:33:16 +01:00
Wojtek Porczyk
ff612a870b qubes/features: check_with_(template_and_)adminvm
- Two new methods: .features.check_with_adminvm() and
  .check_with_template_and_adminvm(). Common code refactored.

- Two new AdminAPI calls to take advantage of the methods:
  - admin.vm.feature.CheckWithAdminVM
  - admin.vm.feature.CheckWithTemplateAndAdminVM

- Features manager moved to separate module in anticipation of features
  on app object in R5.0. The attribute Features.vm renamed to
  Features.subject.

- Documentation, tests.
2018-12-20 18:28:33 +01:00
Marek Marczykowski-Górecki
3728230e3c
Merge branch 'maxmem' 2018-12-09 18:38:21 +01:00
Marek Marczykowski-Górecki
7a6e4b284f
vm/mix/net: prevent setting provides_network=false if qube is still used
Add symmetric check to the one in 'netvm' property.

Fixes QubesOS/qubes-issues#4552
2018-12-06 20:37:50 +01:00
Marek Marczykowski-Górecki
9f6c97596d
tests: updates-available notification
First install test-pkg-1.0, then add test-pkg-1.1 to repo and check if
updates-available flag is set. Then install updates and check if the
flag is cleared.

QubesOS/qubes-issues#2009
2018-12-05 06:19:34 +01:00
Marek Marczykowski-Górecki
cea9f66bc3
tests/network: reduce code duplication 2018-12-05 05:42:15 +01:00
Marek Marczykowski-Górecki
2edafacbe4
tests: listen on 'misc' socket too 2018-12-05 05:42:15 +01:00
Marek Marczykowski-Górecki
162196a1f8
tests: fix lookup domain by name 2018-12-04 16:12:09 +01:00
Marek Marczykowski-Górecki
7d1bcaf64c Introduce management_dispvm property
The new property is meant for management stack (Salt) to set which DVM
template should be used to maintain given VM. Since the DispVM based on
it will be given ultimate control over target VM (qubes.VMShell
service), it should be trusted. The one pointed to by default_dispvm
not necessary is one.

The property defaults to the value from the template (if any), and then
to a global management_dispvm property. By default it is set to None.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-12-03 19:18:26 +01:00
Marek Marczykowski-Górecki
62bc462a23
tests: default maxmem 2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
b8052f864a
tests: more cases for libvirt xml generation
Related to automatic mem balance enabling/disabling. Check how it behave
in presence of PCI devices, or explicit disabling it.
2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
4dc8631010
Use maxmem=0 to disable qmemman, add more automation to it
Use maxmem=0 for disabling dynamic memory balance, instead of cryptic
service.meminfo-writer feature. Under the hood, meminfo-writer service
is also set based on maxmem property (directly in qubesdb, not
vm.features dict).
Having this as a property (not "feature"), allow to have sensible
handling of default value. Specifically, disable it automatically if
otherwise it would crash a VM. This is the case for:
 - domain with PCI devices (PoD is not supported by Xen then)
 - domain without balloon driver and/or meminfo-writer service

The check for the latter is heuristic (assume presence of 'qrexec' also
can indicate balloon driver support), but it is true for currently
supported systems.

This also allows more reliable control of libvirt config: do not set
memory != maxmem, unless qmemman is enabled.

memory != maxmem only makes sense if qmemman for given domain is
enabled.  Besides wasting some domain resources for extra page tables
etc, for HVM domains this is harmful, because maxmem-memory difference
is made of Popupate-on-Demand pool, which - when depleted - will kill
the domain. This means domain without balloon driver will die as soon
as will try to use more than initial memory - but without balloon driver
it sees maxmem memory and doesn't know about the lower limit.

Fixes QubesOS/qubes-issues#4135
2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
0eab082d85
ext/core-features: make 'template-postinstall' event async
It makes a lot of sense to call long-running operations in that event
handler, including calling back into the VM. Allow that by using
fire_event_async, not just fire_event.

Also, document the event.
2018-11-15 18:25:29 +01:00
Marek Marczykowski-Górecki
d2585aa871
tests/lvm: fix checking lvm pool existence cont.
Commit 15cf593bc5 "tests/lvm: fix checking
lvm pool existence" attempted to fix handling '-' in pool name by using
/dev/VG/LV symlink. But those are not created for thin pools. Change
back to /dev/mapper, but include '-' mangling.

Related QubesOS/qubes-issues#4332
2018-11-15 18:25:29 +01:00
Marek Marczykowski-Górecki
68dffb6895
api/admin: fix error message when refusing to create template on template
Fixes QubesOS/qubes-issues#4463
2018-11-04 17:05:55 +01:00
Marek Marczykowski-Górecki
15cf593bc5
tests/lvm: fix checking lvm pool existence
If pool or group name have '-', it will be mangled as '--' in
/dev/mapper. Use /dev/VG_NAME/LV_NAME symlink instead.

Related QubesOS/qubes-issues#4332
2018-10-30 01:17:00 +01:00
Marek Marczykowski-Górecki
f621e8792c
Merge branch 'master' into devel-no-assert 2018-10-29 20:29:53 +01:00
Marek Marczykowski-Górecki
db6094f397
tests/api: adjust for proper exceptions instead of AssertionError 2018-10-29 20:22:10 +01:00
Marek Marczykowski-Górecki
2b5fc6299e
tests/api: do not test non-existing methods
Remove methods not included in specification (or with different
constraints). Keep commented out methods included in spec but not
implemented.
2018-10-29 20:21:36 +01:00
Marek Marczykowski-Górecki
42061cb194
tests: try to collect qvm-open-in-dvm output if no editor window is shown
Try to collect more details about why the test failed. This will help
only if qvm-open-in-dvm exist early. On the other hand, if it hang, or
remote side fails to find the right editor (which results in GUI error
message), this change will not provide any more details.
2018-10-29 01:20:57 +01:00
Marek Marczykowski-Górecki
84c321b923
tests: increase session startup timeout for whonix-ws based VMs
First boot of whonix-ws based VM take extended period of time, because
a lot of files needs to be copied to private volume. This takes even
more time, when verbose logging through console is enabled. Extend the
timeout for that.
2018-10-27 16:44:53 +02:00
Marek Marczykowski-Górecki
84d3547f09
tests: adjust extra tests loader to work with nose2
Nose loader do not provide loader.loadTestsFromTestCase(), use
loader.loadTestsFromNames() instead.
2018-10-27 16:44:53 +02:00
Marek Marczykowski-Górecki
fb14f589cb
tests: wait for full user session before doing rest of the test
Clean VM shutdown may timeout if its initiated before full startup, so
make sure the full startup is completed first.
2018-10-27 16:44:53 +02:00
Marek Marczykowski-Górecki
4742a630f2
tests: use iptables --wait
QubesOS/qubes-issues#3665 affects also tests...
2018-10-27 16:44:53 +02:00
Marek Marczykowski-Górecki
8be70c9e4d
ext/services: allow for os=Linux feature request from VM
It's weird to set it for Windows, but not Linux.
2018-10-27 16:44:53 +02:00
Marek Marczykowski-Górecki
e244c192ae
tests: use /bin/uname instead of /bin/hostname as dummy output generator
Use something included in coreutils installed everywhere.
2018-10-27 16:44:53 +02:00
Marek Marczykowski-Górecki
0b7aa546c6
tests: remove VM reference from QubesVMError
Yet another place wheren object references are leaked.
2018-10-27 16:44:53 +02:00