Commit Graph

826 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
0bccddf1f5
Adjust code for possibly coroutine Volume.export() and Volume.export_end()
Now Volume.export() may be a coroutine and also may be accompanied by
Volume.export_end() cleaning up after it.

See previous commits for building blocks for this.

This commit adjusts usage of Volume.export() and adds matching
Volume.export_end() throughout the code base.

Fixes QubesOS/qubes-issues#5935
2020-07-08 12:50:10 +02:00
Marek Marczykowski-Górecki
8b760451a6
vm/dispvm: place all volumes in the same pool as DispVM's template
Make all volume's pool controlled by DisposableVM Template. This
specifically makes DispVM's volatile volume to be placed directly in the
same pool as its template.

Fixes QubesOS/qubes-issues#5933
2020-07-08 06:05:01 +02:00
Marek Marczykowski-Górecki
410a0728cc
tests: extend TestPool storage driver to make create_on_disk working
Add dummy TestVolume with empty create() method. Other core code
requires also TestPool.get_volume implemented, so add that too (naive
version remembering instances returned from TestPool.init_volume).
2020-07-08 06:05:01 +02:00
Marek Marczykowski-Górecki
e38265501c
tests: cleanup properly in wait_on_fail decorator
Close transport used to wait for user input, otherwise all further tests
would fail on cleanup (FD leak detected). This in practice is only
useful when using wait_on_fail decorator without --failfast option.
2020-07-08 06:05:01 +02:00
Marek Marczykowski-Górecki
d0f619d3c6
Merge branch 'tests-extra-fix' 2020-06-24 18:02:35 +02:00
Marek Marczykowski-Górecki
1ebbfa8dbc
tests: fix extra tests loader
Fix load error reporting - make sure 'err' variable is transferred into
'runTest' function scope.
Then, relax test loading requirements - use 'resolve' instead of 'load',
to bypass dependencies check (defined in setup.py of the package). The
required dependencies should be handled by RPM already, and in some
cases may not match those in python package. An example is PDF
converter, where dependencies at python level are set for the actual
converter, which is irrelevant for running tests from dom0 (tests will
interact with PDF converter inside a VM).
2020-06-24 01:27:57 +02:00
Rusty Bird
5ed762d152
storage: clean up __init__() of Pool() and subclasses
- Remove **kwargs
- Make all arguments keyword-only
- Use required arguments instead of asserts

QubesOS/qubes-issues#5895
2020-06-22 14:03:21 +00:00
Rusty Bird
c2ce28936e
storage/reflink: prefer canonical values for setup_check 2020-06-22 14:03:19 +00:00
Rusty Bird
8f1d59e9af
api/admin: report unexpected driver options in admin.pool.Add
Fixes QubesOS/qubes-issues#5895
2020-06-22 14:03:16 +00:00
Marek Marczykowski-Górecki
4f04a1db91
Revert "libvirt xen: Add gfx_passthru and device_video"
Libxl now automatically enables the gfx_passthru when IGD is attached.

This reverts commit 3e8f5a7a57.
2020-06-10 15:40:28 +02:00
Marek Marczykowski-Górecki
dc1b3b4d86
Do not announce RDRAND instruction on Ivy Bridge
XSA-320 / CVE-2020-0543 affects Ivy Bridge and later platforms, but a
fix (microcode update) won't be available for Ivy Bridge. Disable
affected instruction (do not announce it in CPUID - complying software
should not use it then).
2020-06-10 06:12:46 +02:00
Marek Marczykowski-Górecki
47d4030bdb
tests: move pure-qrexec tests into separate module
Move qrexec (as a mechianism) tests into qubes.tests.integ.qrexec.
Yet anoter split for better openQA jobs scheduling.
2020-06-06 04:07:55 +02:00
Marek Marczykowski-Górecki
60b8b40317
tests: move IPv6 tests into separate file
Move to qubes.tests.integ.network_ipv6.
Again, to allow scheduling as a separate openQA job.
2020-06-06 04:07:55 +02:00
Marek Marczykowski-Górecki
586526c072
tests: move vm update test to a seprate file
This allows scheduling them in a separate job on openQA
2020-06-06 04:07:54 +02:00
Marek Marczykowski-Górecki
4e6472b018
tests: switch simple http server to python3
Python2 is phased out and not available anymore in some templates.
2020-05-27 13:41:02 +02:00
Marek Marczykowski-Górecki
096b7d611a
tests: adjust top.enable salt test
Don't assume any other enabled top. If there is none, 'base'
(environment name) also isn't listed.
2020-05-27 13:36:30 +02:00
Marek Marczykowski-Górecki
aebb1594d7
Merge branch 'qubesd-socket'
* qubesd-socket:
  rpm: update dependency on qrexec
  Remove qubesd-query-fast
  Make qubesd connected directly as an socket qrexec service
2020-05-24 03:26:04 +02:00
Marek Marczykowski-Górecki
eba628e90f
Merge branch 'rename-property-del-reset'
* rename-property-del-reset:
  Fire property-reset event when default value might change
  Convert handler to use property-reset instead of property-del
  Remove leftovers of default_fw_netvm
  Deprecate property-del:name events and introduce property-reset:name instead
2020-05-24 02:17:28 +02:00
Marek Marczykowski-Górecki
b06f831528
Deprecate property-del:name events and introduce property-reset:name instead
And the same for -pre- events.

The property-del name is really confusing (it makes sense only for those
with deep knowledge of the implementation), because the property isn't
really deleted - it is only reverted to the "default" state (which most
properties have). So, name the event property-reset, intentionally
similar to property-set, as it is also kind of a value change.

Additionally the property-reset event is meant to be called when the
(dynamic) default value changes. Due to the current implementation, it
is a manual process so it can't be guaranteed to be called in all those
cases, but lets try to cover as much as possible.

Fixes QubesOS/qubes-issues#5834
2020-05-23 03:57:42 +02:00
Marek Marczykowski-Górecki
88c7c66bb9
tests/backup: ignore automatic 'servicevm' feature...
...when verifying old backup restored. It wasn't present in the backup,
but its presence is expected in some cases. Properly setting 'servicevm'
feature is tested elsewhere.
2020-05-16 13:00:40 +02:00
Marek Marczykowski-Górecki
c7d3635972
Merge remote-tracking branch 'origin/pr/342'
* origin/pr/342:
  Added a servicevm feature extension
  Removed unused Qubes Manager extension
  Add vm.icon property
2020-05-11 01:51:52 +02:00
Marek Marczykowski-Górecki
48ae89fe62
Make qubesd connected directly as an socket qrexec service
Remove intermediate qubesd-query-fast proxy process.
This requires changing socket protocol to match what qrexec is sending
in the header.

Fixes QubesOS/qubes-issues#3293
2020-05-11 01:13:15 +02:00
Marta Marczykowska-Górecka
419b4d5968
Added a servicevm feature extension
Used by vm.icon method, at the moment features['servicevm'] is set
when a VM provides_network.
2020-05-10 23:44:48 +02:00
Marta Marczykowska-Górecka
c34b53d83a
Add vm.icon property
This is a property for handling vm icons that change depending on
vm type.
Depends on QubesOS/qubes-artwork#17

references QubesOS/qubes-issues#5767
2020-05-05 16:14:10 +02:00
Artur Puzio
3e8f5a7a57
libvirt xen: Add gfx_passthru and device_video
Add gfx_passthru and device_video options to libvirt Xen domain template

Signed-off-by: Artur Puzio <contact@puzio.waw.pl>
2020-04-30 16:33:10 +02:00
Marek Marczykowski-Górecki
e5edbc53fd
Merge remote-tracking branch 'origin/pr/332'
* origin/pr/332:
  tests: improve audio tests
  tests: fix qvm-copy-to-vm test
  tests: ensure proper cleanup of auxiliary process
2020-04-10 05:34:56 +02:00
Marek Marczykowski-Górecki
46f09f897c
Merge remote-tracking branch 'origin/pr/326'
* origin/pr/326:
  ext/admin: workaround for extension's __init__() called multiple times
  tests: teardown fixes
  travis: include core-qrexec in tests
  api/admin: (ext/admin) limit listing VMs based on qrexec policy
  api/internal: extract get_system_info() function
2020-04-10 05:32:58 +02:00
Marek Marczykowski-Górecki
8420adf973
tests: improve audio tests
- wait for the client be listed in dom0
- report parecord stderr
- allow up to 20ms to be missing, to account for potentially suspended
  device initially
2020-04-09 05:56:12 +02:00
Rusty Bird
f9538a578d
tests/file: run_until_complete(); coro_maybe() for single arg 2020-04-07 21:04:04 +00:00
Rusty Bird
9122a14f94
tests/file: use self.loop 2020-04-07 21:04:02 +00:00
Marek Marczykowski-Górecki
3066190283
tests: fix qvm-copy-to-vm test
Make the check if remote file wasn't removed meaningful. Previously the
user didn't have permission to remote the source file, so even if the
tool would try, it would fail.
2020-04-01 17:57:12 +02:00
Marek Marczykowski-Górecki
1b7e2a5cbf
tests: ensure proper cleanup of auxiliary process
Various qrexec tests create auxiliary process (service_proc) as a local
variable. In case of test failure, process cleanup isn't called and may
lead to FD leaks and breaking subsequent tests.

Fix this by always saving such process instance in self.service_proc and
cleaning it up in self.tearDown() (this code is already there).

Add also waiting (and in case of timeout - killing) of a service call
process too.
2020-04-01 17:35:57 +02:00
Marek Marczykowski-Górecki
0341cc5258
tests: teardown fixes
Add few missing app.close() calls on test teardown.
Fix socket cleanup in TC_00_QubesDaemonProtocol() - not only close the
FD, but also unregister it from asyncio event loop.
2020-03-28 03:23:29 +01:00
Marek Marczykowski-Górecki
3f96c72ee3
api/admin: (ext/admin) limit listing VMs based on qrexec policy
Various Admin API calls, when directed at dom0, retrieve global system
view instead of a specific VM. This applies to admin.vm.List (called at
dom0 retrieve full VM list) and admin.Events (called at dom0 listen for
events of all the VMs). This makes it tricky to configure a management
VM with access to limited set of VMs only, because many tools require
ability to list VMs, and that would return full list.

Fix this issue by adding a filter to admin.vm.List and admin.Events
calls (using event handlers in AdminExtension) that filters the output
using qrexec policy. This version evaluates policy for each VM or event
(but loads only once). If the performance will be an issue, it can be
optimized later.

Fixes QubesOS/qubes-issues#5509
2020-03-28 03:23:28 +01:00
Marek Marczykowski-Górecki
8f0ec59f95
Merge remote-tracking branch 'origin/pr/330'
* origin/pr/330:
  gui: fixes from Marek's comments
  gui: improvements of feature keyboard layout checks
  tests: adapt tests for keyboard-layout
  gui: drop legacy qubes-keyboard support
2020-03-18 14:34:22 +01:00
Frédéric Pierret (fepitre)
5cb0e61a84
tests: adapt tests for keyboard-layout 2020-03-18 09:17:37 +01:00
Marek Marczykowski-Górecki
29020d0c53
Merge branch 'feature-pre-set'
* feature-pre-set:
  Add pre- events to vm features
2020-03-17 19:26:53 +01:00
Marek Marczykowski-Górecki
d05592ba52
Add pre- events to vm features
Allow extensions to validate feature values, before it is set.
2020-03-17 18:46:12 +01:00
Marek Marczykowski-Górecki
ddc13d6b62
Merge remote-tracking branch 'origin/pr/329'
* origin/pr/329:
  tests: make PEP8 happier
  tests: fix already registered event impl
2020-03-17 12:07:18 +01:00
Frédéric Pierret (fepitre)
d8f80c9687
tests: make PEP8 happier 2020-03-16 23:22:36 +01:00
Frédéric Pierret (fepitre)
40156c3e78
tests: fix already registered event impl 2020-03-16 23:21:15 +01:00
Marek Marczykowski-Górecki
f62a861140
tests: fix test_053_qrexec_vm_service_eof_reverse
and not mark it as expected failure anymore. Note the removal of the
expected failure isn't just about the changes here, but also about the
actual fix on the qrexec side (ffafd01 "Fix not closed file descriptors in
qubes-rpc-multiplexer" commit in core-qrexec repository).
2020-03-15 02:55:18 +01:00
Marek Marczykowski-Górecki
5423ead27a
tests/vm_qrexec_gui: do not swallow stderr on failure
QubesVM.run_for_stdio() by default captures stderr. In case of call fail
(non-zero return code), captured stderr is included in the exception
object, but isn't printed by default CalledProcessError message.
Make it visible by:
 - handling CalledProcessError and including in the test failure message
   (when exception is captured already)
 - not capturing stderr (if no exception handling is present in the
   test)
2020-03-14 03:43:22 +01:00
Marek Marczykowski-Górecki
d033457f3d
tests: adjust for qrexec-client-vm exit code
It now consistently returns remote exit code, not a local one.
Check 'cat' exit code instead.
2020-03-14 03:43:17 +01:00
Marek Marczykowski-Górecki
7a750ea9c8
tests: QubesVM.run_for_stdio 2020-03-10 11:56:48 +01:00
Marek Marczykowski-Górecki
9ec86f3c41
Workaround different behaviour of asyncio's Process.communicate()
In asyncio's Process.communicate() input=None does not close stdin.
Workaround it by using b'' instead of None.

https://bugs.python.org/issue39744
2020-03-10 11:56:48 +01:00
Marek Marczykowski-Górecki
3ce4e5eaa5
tests: adjust SystemTestCase.create_remote_file to create executables
The main use case for this function is to create qrexec services in VMs.
Since qrexec now require service scripts to be executable, make
create_remote_file() adjust permissions.
2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
2460adbdef
tests: socket-based qrexec services
QubesOS/qubes-issues#3912
2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
3f5fb0de9e
tests: hide git output when checking local checkout 2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
534de9bc1c
tests: fix cleanup before test
If any test-* VMs remains from previous test run, there are removed
before test. self.app doesn't exist at this point, so don't require it
in self.remove_vms().
2020-03-10 11:56:47 +01:00