Commit Graph

2028 Commits

Author SHA1 Message Date
Rusty Bird
8f1d59e9af
api/admin: report unexpected driver options in admin.pool.Add
Fixes QubesOS/qubes-issues#5895
2020-06-22 14:03:16 +00:00
Marek Marczykowski-Górecki
4f04a1db91
Revert "libvirt xen: Add gfx_passthru and device_video"
Libxl now automatically enables the gfx_passthru when IGD is attached.

This reverts commit 3e8f5a7a57.
2020-06-10 15:40:28 +02:00
Marek Marczykowski-Górecki
dc1b3b4d86
Do not announce RDRAND instruction on Ivy Bridge
XSA-320 / CVE-2020-0543 affects Ivy Bridge and later platforms, but a
fix (microcode update) won't be available for Ivy Bridge. Disable
affected instruction (do not announce it in CPUID - complying software
should not use it then).
2020-06-10 06:12:46 +02:00
Marek Marczykowski-Górecki
47d4030bdb
tests: move pure-qrexec tests into separate module
Move qrexec (as a mechianism) tests into qubes.tests.integ.qrexec.
Yet anoter split for better openQA jobs scheduling.
2020-06-06 04:07:55 +02:00
Marek Marczykowski-Górecki
60b8b40317
tests: move IPv6 tests into separate file
Move to qubes.tests.integ.network_ipv6.
Again, to allow scheduling as a separate openQA job.
2020-06-06 04:07:55 +02:00
Marek Marczykowski-Górecki
586526c072
tests: move vm update test to a seprate file
This allows scheduling them in a separate job on openQA
2020-06-06 04:07:54 +02:00
Marek Marczykowski-Górecki
719eb5c2a2
Fix revisions_to_keep volume property when creating VM in a specific pool
The revisions_to_keep should be inherited from the pool by default (or
whatever else logic is in the storage pool driver). When creating VM in
a specific pool, volumes config is re-initialized to include right
defaults. But the config cleaning logic in `_clean_volume_config()`
failed to remove revisions_to_keep property initialized by the default
pool driver. This prevented new pool driver to apply its own default
logic.

An extreme result was inability to create a VM in 'file' pool at all,
because it refuses any revisions_to_keep > 1, and the default LVM
pool has revisions_to_keep = 2.
2020-05-28 01:45:18 +02:00
Marek Marczykowski-Górecki
4e6472b018
tests: switch simple http server to python3
Python2 is phased out and not available anymore in some templates.
2020-05-27 13:41:02 +02:00
Marek Marczykowski-Górecki
096b7d611a
tests: adjust top.enable salt test
Don't assume any other enabled top. If there is none, 'base'
(environment name) also isn't listed.
2020-05-27 13:36:30 +02:00
Marek Marczykowski-Górecki
aebb1594d7
Merge branch 'qubesd-socket'
* qubesd-socket:
  rpm: update dependency on qrexec
  Remove qubesd-query-fast
  Make qubesd connected directly as an socket qrexec service
2020-05-24 03:26:04 +02:00
Marek Marczykowski-Górecki
64edbdf7d3
Merge remote-tracking branch 'origin/pr/344'
* origin/pr/344:
  travis: pip -> pip3
  Update .travis.yml
  Drop initial root thin pool definition
  Prevent double hyphens in thin_pool parsing
  Rename default root thin pool from 'lvm' to 'root'
2020-05-24 02:19:37 +02:00
Marek Marczykowski-Górecki
eba628e90f
Merge branch 'rename-property-del-reset'
* rename-property-del-reset:
  Fire property-reset event when default value might change
  Convert handler to use property-reset instead of property-del
  Remove leftovers of default_fw_netvm
  Deprecate property-del:name events and introduce property-reset:name instead
2020-05-24 02:17:28 +02:00
Marek Marczykowski-Górecki
d61d24b055
Fire property-reset event when default value might change
Those are only some cases, the most obvious ones:
 - defaults inherited from a template
 - xid and start_time on domain start/stop
 - IP related properties
 - icon

QubesOS/qubes-issues#5834
2020-05-23 04:50:02 +02:00
Marek Marczykowski-Górecki
dc2cf1db5d
Convert handler to use property-reset instead of property-del
There was also one case of triggering property-{del => reset}
synthetically on default value change. Adjust it too and drop -pre-
event call in that case.

QubesOS/qubes-issues#5834
2020-05-23 03:57:42 +02:00
Marek Marczykowski-Górecki
4e473dd190
Remove leftovers of default_fw_netvm
The property is long gone, remove handling its change.
2020-05-23 03:57:42 +02:00
Marek Marczykowski-Górecki
b06f831528
Deprecate property-del:name events and introduce property-reset:name instead
And the same for -pre- events.

The property-del name is really confusing (it makes sense only for those
with deep knowledge of the implementation), because the property isn't
really deleted - it is only reverted to the "default" state (which most
properties have). So, name the event property-reset, intentionally
similar to property-set, as it is also kind of a value change.

Additionally the property-reset event is meant to be called when the
(dynamic) default value changes. Due to the current implementation, it
is a manual process so it can't be guaranteed to be called in all those
cases, but lets try to cover as much as possible.

Fixes QubesOS/qubes-issues#5834
2020-05-23 03:57:42 +02:00
Frédéric Pierret (fepitre)
f06f41d594
Drop initial root thin pool definition
See https://github.com/QubesOS/qubes-core-admin/pull/344#issuecomment-629626978
2020-05-18 14:13:51 +02:00
Marek Marczykowski-Górecki
88c7c66bb9
tests/backup: ignore automatic 'servicevm' feature...
...when verifying old backup restored. It wasn't present in the backup,
but its presence is expected in some cases. Properly setting 'servicevm'
feature is tested elsewhere.
2020-05-16 13:00:40 +02:00
Frédéric Pierret (fepitre)
e4d7df4976
Prevent double hyphens in thin_pool parsing 2020-05-15 15:15:37 +02:00
Frédéric Pierret (fepitre)
71159bfca2
Rename default root thin pool from 'lvm' to 'root'
New partition output split dom0 and VM thin pools

https://github.com/QubesOS/qubes-anaconda-addon/pull/7
QubesOS/qubes-issues#5763
2020-05-12 17:32:07 +02:00
Marek Marczykowski-Górecki
c7d3635972
Merge remote-tracking branch 'origin/pr/342'
* origin/pr/342:
  Added a servicevm feature extension
  Removed unused Qubes Manager extension
  Add vm.icon property
2020-05-11 01:51:52 +02:00
Marek Marczykowski-Górecki
48ae89fe62
Make qubesd connected directly as an socket qrexec service
Remove intermediate qubesd-query-fast proxy process.
This requires changing socket protocol to match what qrexec is sending
in the header.

Fixes QubesOS/qubes-issues#3293
2020-05-11 01:13:15 +02:00
Marta Marczykowska-Górecka
419b4d5968
Added a servicevm feature extension
Used by vm.icon method, at the moment features['servicevm'] is set
when a VM provides_network.
2020-05-10 23:44:48 +02:00
Marek Marczykowski-Górecki
4a56064adb
Merge remote-tracking branch 'origin/pr/341'
* origin/pr/341:
  libvirt xen: Add gfx_passthru and device_video
2020-05-07 04:36:08 +02:00
Marta Marczykowska-Górecka
7241b54868
Removed unused Qubes Manager extension
A fossil from Qubes 3.2 times.
2020-05-05 16:33:45 +02:00
Marta Marczykowska-Górecka
c34b53d83a
Add vm.icon property
This is a property for handling vm icons that change depending on
vm type.
Depends on QubesOS/qubes-artwork#17

references QubesOS/qubes-issues#5767
2020-05-05 16:14:10 +02:00
Marek Marczykowski-Górecki
dbe072b762
ext/admin: fix async/non-async mismatch 2020-04-30 21:22:29 +02:00
Artur Puzio
3e8f5a7a57
libvirt xen: Add gfx_passthru and device_video
Add gfx_passthru and device_video options to libvirt Xen domain template

Signed-off-by: Artur Puzio <contact@puzio.waw.pl>
2020-04-30 16:33:10 +02:00
Marek Marczykowski-Górecki
e5edbc53fd
Merge remote-tracking branch 'origin/pr/332'
* origin/pr/332:
  tests: improve audio tests
  tests: fix qvm-copy-to-vm test
  tests: ensure proper cleanup of auxiliary process
2020-04-10 05:34:56 +02:00
Marek Marczykowski-Górecki
3b963030a6
Merge remote-tracking branch 'origin/pr/331'
* origin/pr/331:
  Fix multiple qmemman issues
2020-04-10 05:34:28 +02:00
Marek Marczykowski-Górecki
46f09f897c
Merge remote-tracking branch 'origin/pr/326'
* origin/pr/326:
  ext/admin: workaround for extension's __init__() called multiple times
  tests: teardown fixes
  travis: include core-qrexec in tests
  api/admin: (ext/admin) limit listing VMs based on qrexec policy
  api/internal: extract get_system_info() function
2020-04-10 05:32:58 +02:00
Marek Marczykowski-Górecki
8420adf973
tests: improve audio tests
- wait for the client be listed in dom0
- report parecord stderr
- allow up to 20ms to be missing, to account for potentially suspended
  device initially
2020-04-09 05:56:12 +02:00
Marek Marczykowski-Górecki
a6efd6a301
Merge remote-tracking branch 'origin/pr/334'
* origin/pr/334:
  collections.Callable -> collections.abc.Callable
2020-04-09 03:28:00 +02:00
Rusty Bird
6605bf406d
collections.Callable -> collections.abc.Callable
"Deprecated since version 3.3, will be removed in version 3.10"
- https://docs.python.org/3/library/collections.html
2020-04-07 21:30:21 +00:00
Rusty Bird
f9538a578d
tests/file: run_until_complete(); coro_maybe() for single arg 2020-04-07 21:04:04 +00:00
Rusty Bird
9122a14f94
tests/file: use self.loop 2020-04-07 21:04:02 +00:00
Marek Marczykowski-Górecki
3066190283
tests: fix qvm-copy-to-vm test
Make the check if remote file wasn't removed meaningful. Previously the
user didn't have permission to remote the source file, so even if the
tool would try, it would fail.
2020-04-01 17:57:12 +02:00
Marek Marczykowski-Górecki
1b7e2a5cbf
tests: ensure proper cleanup of auxiliary process
Various qrexec tests create auxiliary process (service_proc) as a local
variable. In case of test failure, process cleanup isn't called and may
lead to FD leaks and breaking subsequent tests.

Fix this by always saving such process instance in self.service_proc and
cleaning it up in self.tearDown() (this code is already there).

Add also waiting (and in case of timeout - killing) of a service call
process too.
2020-04-01 17:35:57 +02:00
Marek Marczykowski-Górecki
dd50e300c3
Fix multiple qmemman issues
First the main bug: when meminfo xenstore watch fires, in some cases
(just after starting some domain) XS_Watcher refreshes internal list of
domains before processing the event. This is done specifically to
include new domain in there. But the opposite could happen too - the
domain could be destroyed. In this case refres_meminfo() function raises
an exception, which isn't handled and interrupts the whole xenstore
watch loop. This issue is likely to be triggered by killing the domain,
as this way it could disappear shortly after writing updated meminfo
entry. In case of proper shutdown, meminfo-writer is stopped earlier and
do not write updates just before domain destroy.
Fix this by checking if the requested domain is still there just after
refreshing the list.

Then, catch exceptions in xenstore watch handling functions, to not
interrupt xenstore watch loop. If it gets interrupted, qmemman basically
stops memory balancing.

And finally, clear force_refresh_domain_list flag after refreshing the
domain list. That missing line caused domain refresh at every meminfo
change, making it use some more CPU time.

While at it, change "EOF" log message to something a bit more
meaningful.

Thanks @conorsch for capturing valuable logs.

Fixes QubesOS/qubes-issues#4890
2020-04-01 03:46:29 +02:00
Marek Marczykowski-Górecki
b11d6e058b
ext/admin: workaround for extension's __init__() called multiple times
... during tests.
qubes.ext.Extension class is a weird thing that tries to make each extension
a singleton. But this unfortunately have a side effect that __init__()
is called separately for each "instance" (created in Qubes()'s
__init__()), even though this is really the same object. During normal
execution this isn't an issue, because there is just one Qubes() object
instance. But during tests, multiple objects are created.

In this particular case, it caused PolicyCache() to be created twice and
the second one overriden the first one - without properly cleaning it
up. This leaks a file descriptor (inotify one). The fact that cleanup()
was called twice too didn't helped, because it was really called on
the same object, the one requiring cleanup was already gone.

Workaround this by checking if policy_cache field is initialize and
avoid re-initialize it. Also, on Qubes() object cleanup remove that
field, so it can be properly initialized on the next test iteration.
2020-03-31 01:57:22 +02:00
Marek Marczykowski-Górecki
0341cc5258
tests: teardown fixes
Add few missing app.close() calls on test teardown.
Fix socket cleanup in TC_00_QubesDaemonProtocol() - not only close the
FD, but also unregister it from asyncio event loop.
2020-03-28 03:23:29 +01:00
Marek Marczykowski-Górecki
3f96c72ee3
api/admin: (ext/admin) limit listing VMs based on qrexec policy
Various Admin API calls, when directed at dom0, retrieve global system
view instead of a specific VM. This applies to admin.vm.List (called at
dom0 retrieve full VM list) and admin.Events (called at dom0 listen for
events of all the VMs). This makes it tricky to configure a management
VM with access to limited set of VMs only, because many tools require
ability to list VMs, and that would return full list.

Fix this issue by adding a filter to admin.vm.List and admin.Events
calls (using event handlers in AdminExtension) that filters the output
using qrexec policy. This version evaluates policy for each VM or event
(but loads only once). If the performance will be an issue, it can be
optimized later.

Fixes QubesOS/qubes-issues#5509
2020-03-28 03:23:28 +01:00
Marek Marczykowski-Górecki
8f0ec59f95
Merge remote-tracking branch 'origin/pr/330'
* origin/pr/330:
  gui: fixes from Marek's comments
  gui: improvements of feature keyboard layout checks
  tests: adapt tests for keyboard-layout
  gui: drop legacy qubes-keyboard support
2020-03-18 14:34:22 +01:00
Frédéric Pierret (fepitre)
577e4b24a6
gui: fixes from Marek's comments 2020-03-18 14:17:04 +01:00
Frédéric Pierret (fepitre)
f130ec0bf3
gui: improvements of feature keyboard layout checks 2020-03-18 09:46:21 +01:00
Frédéric Pierret (fepitre)
5cb0e61a84
tests: adapt tests for keyboard-layout 2020-03-18 09:17:37 +01:00
Frédéric Pierret (fepitre)
732e291ab6
gui: drop legacy qubes-keyboard support
Add check for keyboard-layout feature set
2020-03-18 09:17:37 +01:00
Marek Marczykowski-Górecki
29020d0c53
Merge branch 'feature-pre-set'
* feature-pre-set:
  Add pre- events to vm features
2020-03-17 19:26:53 +01:00
Marek Marczykowski-Górecki
d05592ba52
Add pre- events to vm features
Allow extensions to validate feature values, before it is set.
2020-03-17 18:46:12 +01:00
Marek Marczykowski-Górecki
ddc13d6b62
Merge remote-tracking branch 'origin/pr/329'
* origin/pr/329:
  tests: make PEP8 happier
  tests: fix already registered event impl
2020-03-17 12:07:18 +01:00