Commit Graph

6098 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
39ef189a93
Merge remote-tracking branch 'origin/pr/354'
* origin/pr/354: (35 commits)
  tests/lvm: re-introduce POOL_CONF
  tests/lvm & callback: remove explicit class references
  storage/callback: remove the "word of caution"
  storage/callback: comment fixes
  storage/callback: add the config ID as callback argument
  storage/callback: some callbacks added & removed
  tests/lvm & callback: Refactoring
  Revert "storage/callback: do not run sync code async"
  tests/callback: ensure missing conf causes errors
  storage/callback: do not run sync code async
  tests/callback: added callback-specific tests
  storage/callback: async Volume.export() & added Volume.export_end()
  storage/lvm: make the "hack" work with CallbackPool instances
  storage/callback: add the backend_class property
  tests/callback: add them to the rpm build
  storage/callback: various fixes
  tests/callback: add rudimentary tests for the callback driver
  tests/lvm: make the tests re-usable for other drivers
  storage/callback: fix issues detected by pylint
  storage/callback: volume callbacks now also rceive the source volume as argument (if there's any)
  ...
2020-11-27 00:26:18 +01:00
Marek Marczykowski-Górecki
7ab1703a06
Merge remote-tracking branch 'origin/pr/375'
* origin/pr/375:
  Fix line lengths
  Return better error messages from file pool
  Fix bugs found by Rusty Bird
  Fix export locking
  Re-add dirty check in case qubesd is restarted
  File volumes are started NAND exported
  file pool: snapshotting dirty volume not supported
  Always snapshot in the FILE pool
2020-11-27 00:19:51 +01:00
Marek Marczykowski-Górecki
b3bb65d3b0
Merge branch 'tests20201030'
* tests20201030:
  tests: fix test_550_ipv6_spoof_ip
  tests: wait for full VM startup before testing network and few other
  tests: fix logging network info on failed test
  tests: use dup-ed stdin in wait_on_fail
  tests: add PVH grub2 tests
  tests: adjust for applications list stored in features dict
  tests: remove test_030_clone
  tests: make sure dnsmasq is stopped before starting it again
  tests: fail the test early if VM fails to start
2020-11-26 22:44:39 +01:00
Demi Marie Obenour
542fee173d
Fix line lengths 2020-11-26 13:40:07 -05:00
Demi Marie Obenour
09785449ed
Return better error messages from file pool
A `qubes.storage.StoragePoolException` will be returned as a useful
error from `qvm-backup`.
2020-11-26 11:17:33 -05:00
Marek Marczykowski-Górecki
86ccbcc206
tests: fix test_550_ipv6_spoof_ip
Use 'ip route replace' instead of 'ip route add' to not fail on already
existing route entry. This worked before only because of a race
condition with a network setup.
2020-11-26 05:20:49 +01:00
Marek Marczykowski-Górecki
7d8d0c10d1
tests: wait for full VM startup before testing network and few other
Network may be configured (by qubes-misc-post service in some cases)
after qrexec is started. Delay any test after VM is fully started (as
the user would do).
Adjust wait_for_session function, to re-use VM startup timeout, which
make it adjustable for slower machines (like nested virt on openQA).
2020-11-26 05:19:53 +01:00
Marek Marczykowski-Górecki
12bc523b98
Make run-tests script try to find qrexec
Parts of core-admin (specifically some extensions) require qrexec python
module to work, and to be tested. Try to find it in a standard place (as
qubes-builder place it).
2020-11-26 05:13:56 +01:00
Marek Marczykowski-Górecki
77323cd30d
Start using 'async def' syntax
The Python version (and pylint) are new enough finally.
Adjust QubesVM.run* functions for now.
Python 3.8 brings also AsyncMock() which makes tests slightly less ugly.
2020-11-26 05:13:48 +01:00
Marek Marczykowski-Górecki
8b321e8fb0
gitlab-ci: install dependencies via dnf
Make them available system-wide, not only in user home - needed to call
some of the code via sudo in tests.
2020-11-26 05:13:38 +01:00
Demi Marie Obenour
7275939000
Fix bugs found by Rusty Bird 2020-11-25 16:42:35 -05:00
Demi Marie Obenour
ec51673f21
Fix export locking
If qubesd has restarted then _export_lock will be None
2020-11-25 15:08:13 -05:00
Demi Marie Obenour
e53d040051
Re-add dirty check in case qubesd is restarted 2020-11-25 13:33:32 -05:00
Demi Marie Obenour
e4854df42f
File volumes are started NAND exported
So add a lock to ensure this.
2020-11-25 13:25:57 -05:00
Demi Marie Obenour
14e9154e4e
file pool: snapshotting dirty volume not supported
Raise a NotImplementedError rather than risking corruption.
2020-11-24 19:15:25 -05:00
Marek Marczykowski-Górecki
5e4b495682
gitlab-ci: install pip before using it, adjust pylint call 2020-11-22 00:54:44 +01:00
Demi Marie Obenour
cee8201989
Always snapshot in the FILE pool
We must snapshot a VM’s disk before exporting it.  Otherwise, we will
likely corrupt the VM’s filesystem.

Fixes https://github.com/QubesOS/qubes-issues/issues/4324
2020-11-20 16:50:56 -05:00
Marek Marczykowski-Górecki
2c8057c43a
gitlab-ci: add tests job
- run tests in a vm runner to have LVM and loop devices access
- upload coverage data via wrapper to extract PR number
2020-11-18 05:47:50 +01:00
Marek Marczykowski-Górecki
a40344252a
rpm: fix %if syntax
rpm in fedora-33 complains about not quoted strings:

    error: bare words are no longer supported, please use "...":  xxen == xxen
2020-11-17 23:06:46 +01:00
3hhh
c717dacd8d
tests/lvm: re-introduce POOL_CONF
It's still needed by the lvm integration tests. If possible, it should
be avoided inside lvm unit tests though as using it will break
compatibility with callback unit tests.
2020-11-17 17:43:37 +01:00
Frédéric Pierret (fepitre)
a21783e5f6
Add .gitlab-ci.yml 2020-11-15 23:31:36 +01:00
Marek Marczykowski-Górecki
8233009e2a
tests: fix logging network info on failed test
- log from requested VM
- use 'critical' log level to reach journalctl
- log /var/log/xen/xen-hotplug.log too
2020-11-13 04:41:03 +01:00
Marek Marczykowski-Górecki
79fb0cade1
tests: use dup-ed stdin in wait_on_fail
transport.close() (necessary to detach reader from the loop) will close
the FD. In case of stdin, it will prevent another call to this waiting
function. Use dup(2) to register cloned FD, which is safe to close
without side effects.
2020-11-13 03:21:37 +01:00
Marek Marczykowski-Górecki
6db24d3eaf
tests: add PVH grub2 tests 2020-11-12 06:00:21 +01:00
Marek Marczykowski-Górecki
8f38753bdb
Re-enable SMAP for VMs
Buggy Linux version is no longer present in any supported template (the
last one was Debian jessie).

QubesOS/qubes-issues#2881
2020-11-05 05:28:39 +01:00
Marek Marczykowski-Górecki
73e55eb99a
tests: adjust for applications list stored in features dict 2020-11-02 01:44:32 +01:00
Marek Marczykowski-Górecki
2b49979c2a
tests: remove test_030_clone
Since qubesd-side clone_vm function is gone, it doesn't make sense to
test it. For some time already this tested only if manual step-by-step
clone implemented _in the test itself_ was done correctly.

Actual cloning is part of qubes-core-admin-client and is tested there.
2020-11-02 01:44:17 +01:00
Marek Marczykowski-Górecki
8b4a4a72b4
drop old workaround kernelopts
- nopat - added when PAT in Linux+Xen was buggy, no longer the case for a
long time.
- iommu=soft swiotlb=8192 - necessary to limit required memory on PV with
PCI devices; since we use HVM now (with proper IOMMU translation), this
is no longer needed. Furthermore, it will free some memory in sys-* vms.
2020-11-01 18:10:52 +01:00
Marek Marczykowski-Górecki
14c636469f
tests: make sure dnsmasq is stopped before starting it again
Avoid conflict on listening port ("Address already in use" error).
Send SIGTERM until all instances of dnsmasq exit.
2020-10-30 21:09:47 +01:00
Marek Marczykowski-Górecki
79d4b7162a
tests: fail the test early if VM fails to start
Make an exception in vm.start() actually interrupt the test. The
asyncio.wait() returns list of completed tasks, where exception may be
stored - but is not raised directly. Change to asyncio.gather() that will
propagate the exception by default.
As a side effect, avoid deprecated direct coroutine passing to
asyncio.wait(). This functionality in asyncio.gather() is not
deprecated.
2020-10-30 15:39:57 +01:00
Marek Marczykowski-Górecki
e4a8caaaa2
version 4.1.16 2020-10-10 05:09:34 +02:00
Marek Marczykowski-Górecki
7ffa7564cf
Merge remote-tracking branch 'origin/pr/369'
* origin/pr/369:
  ext: support for non-service feature advertisement
2020-10-10 03:33:09 +02:00
Marek Marczykowski-Górecki
bdd1184a3a
Merge remote-tracking branch 'origin/pr/367'
* origin/pr/367:
  gui: add --all and --dom0 to qubes-input-trigger script
  gui: adapt trigger for dom0 input devices
  gui: trigger services start for ps2 devices
2020-09-30 02:10:39 +02:00
Frédéric Pierret (fepitre)
7ee877caa0 gui: add --all and --dom0 to qubes-input-trigger script 2020-09-26 14:36:21 +02:00
Marek Marczykowski-Górecki
bc26e74339
ext: support for non-service feature advertisement
Add an API for VMs to announce support for non-service features. This is
very similar to supported-service.* features, but applies to non-service
features. This may be also used for announcing support for features that
do not use qvm-features framework itself - for example some VM kernel
features, installed drivers, packages etc.

QubesOS/qubes-issues#6030
2020-09-23 01:16:38 +02:00
Frédéric Pierret (fepitre)
f744d89803
gui: adapt trigger for dom0 input devices 2020-09-21 14:10:14 +02:00
herypt
67faa7c1f9
Add apparmor=1 security=apparmor to kernelopts if apparmor feature is set 2020-09-02 15:09:13 +02:00
herypt
80ecee51db
Set apparmor feature when template advertises support for it 2020-09-02 15:07:30 +02:00
Frédéric Pierret (fepitre)
480aeee9de
gui: trigger services start for ps2 devices 2020-08-30 23:27:17 +02:00
Marek Marczykowski-Górecki
1500ed8fcb
Make pylint happy
- ignore raise-missing-from
- fix super-with-arguments
2020-08-23 02:55:40 +02:00
Marta Marczykowska-Górecka
6b9528316f
Replaced error on nonexisting label name with a more descriptive one
Instead of unintuitive Value Error now we have dedicated QubesLabelNotFoundError.
Goal: to make qvm-prefs be less strange when one mixes up gray and grey again.
2020-08-23 01:58:43 +02:00
Marta Marczykowska-Górecka
b506586089
Fixed grey label color value causing accidental green icons
fixes QubesOS/qubes-issues#3471
2020-08-23 01:58:43 +02:00
Marek Marczykowski-Górecki
66d8a78481
version 4.1.15 2020-08-12 02:43:42 +02:00
Marek Marczykowski-Górecki
ed5b908371
tests: fix duplicated entries in TestVMsCollection
Deduplicate entries when iterating over TestVMsCollection values. Some
tests add given VM multiple times, to have it available under different
kind of keys (name, uuid etc) - similar to the real VMsCollection.
2020-08-12 02:01:31 +02:00
Marek Marczykowski-Górecki
24e0ddd7ab
Merge remote-tracking branch 'origin/pr/365'
* origin/pr/365:
  Added icon property to AdminVM
2020-08-12 01:26:29 +02:00
Marek Marczykowski-Górecki
f2b047c47e
Merge remote-tracking branch 'origin/pr/254'
* origin/pr/254:
  vm: allow StandaloneVM to be a DVM template
  vm: do not allow setting template_for_dispvms=False if there are any DispVMs
  vm: move DVM template specific code into separate mixin
2020-08-12 01:25:57 +02:00
Marek Marczykowski-Górecki
74725b584a
Merge branch 'test-fixes20200806'
* test-fixes20200806:
  tests/extra: add vm.run(..., gui=) argument
  tests: collect detailed diagnostics on failure
  tests: workaround a race in qrexec test
  tests: fix audio recording test
  tests: make qvm-sync-clock test more reliable
2020-08-12 01:25:27 +02:00
Marek Marczykowski-Górecki
c425df6c57
tests/extra: add vm.run(..., gui=) argument
A convenient (and compatible) option to wait for user session before
starting the command.
2020-08-11 02:04:32 +02:00
Marek Marczykowski-Górecki
46cc4ca910
tests: collect detailed diagnostics on failure
Help debugging test failures by collecting detailed information on
failure. It will be logger to the standard logger, which will end up
either on stderr or in journalctl.
2020-08-08 20:36:42 +02:00
Marek Marczykowski-Górecki
8b076dfe5f
tests: workaround a race in qrexec test
qrexec-client-vm may return earlier than it's child process (it exits
right away, without waiting for its child). Add a small wait before
reading exit code from a file.
2020-08-08 19:22:26 +02:00