Commit Graph

5978 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
b06f831528
Deprecate property-del:name events and introduce property-reset:name instead
And the same for -pre- events.

The property-del name is really confusing (it makes sense only for those
with deep knowledge of the implementation), because the property isn't
really deleted - it is only reverted to the "default" state (which most
properties have). So, name the event property-reset, intentionally
similar to property-set, as it is also kind of a value change.

Additionally the property-reset event is meant to be called when the
(dynamic) default value changes. Due to the current implementation, it
is a manual process so it can't be guaranteed to be called in all those
cases, but lets try to cover as much as possible.

Fixes QubesOS/qubes-issues#5834
2020-05-23 03:57:42 +02:00
Frédéric Pierret (fepitre)
bd837c49ec
travis: pip -> pip3 2020-05-18 16:27:26 +02:00
Frédéric Pierret (fepitre)
30900125d3
Update .travis.yml 2020-05-18 14:23:03 +02:00
Frédéric Pierret (fepitre)
f06f41d594
Drop initial root thin pool definition
See https://github.com/QubesOS/qubes-core-admin/pull/344#issuecomment-629626978
2020-05-18 14:13:51 +02:00
Marek Marczykowski-Górecki
88c7c66bb9
tests/backup: ignore automatic 'servicevm' feature...
...when verifying old backup restored. It wasn't present in the backup,
but its presence is expected in some cases. Properly setting 'servicevm'
feature is tested elsewhere.
2020-05-16 13:00:40 +02:00
Frédéric Pierret (fepitre)
e4d7df4976
Prevent double hyphens in thin_pool parsing 2020-05-15 15:15:37 +02:00
Frédéric Pierret (fepitre)
71159bfca2
Rename default root thin pool from 'lvm' to 'root'
New partition output split dom0 and VM thin pools

https://github.com/QubesOS/qubes-anaconda-addon/pull/7
QubesOS/qubes-issues#5763
2020-05-12 17:32:07 +02:00
Marek Marczykowski-Górecki
c7d3635972
Merge remote-tracking branch 'origin/pr/342'
* origin/pr/342:
  Added a servicevm feature extension
  Removed unused Qubes Manager extension
  Add vm.icon property
2020-05-11 01:51:52 +02:00
Marek Marczykowski-Górecki
61143a99a5
Remove qubesd-query-fast
Since qubesd is connected directly as a socket-based qrexec service, the
qubesd-query-fast tool isn't needed anymore.
2020-05-11 01:13:15 +02:00
Marek Marczykowski-Górecki
48ae89fe62
Make qubesd connected directly as an socket qrexec service
Remove intermediate qubesd-query-fast proxy process.
This requires changing socket protocol to match what qrexec is sending
in the header.

Fixes QubesOS/qubes-issues#3293
2020-05-11 01:13:15 +02:00
Marta Marczykowska-Górecka
419b4d5968
Added a servicevm feature extension
Used by vm.icon method, at the moment features['servicevm'] is set
when a VM provides_network.
2020-05-10 23:44:48 +02:00
Marek Marczykowski-Górecki
4a56064adb
Merge remote-tracking branch 'origin/pr/341'
* origin/pr/341:
  libvirt xen: Add gfx_passthru and device_video
2020-05-07 04:36:08 +02:00
Marta Marczykowska-Górecka
7241b54868
Removed unused Qubes Manager extension
A fossil from Qubes 3.2 times.
2020-05-05 16:33:45 +02:00
Marta Marczykowska-Górecka
c34b53d83a
Add vm.icon property
This is a property for handling vm icons that change depending on
vm type.
Depends on QubesOS/qubes-artwork#17

references QubesOS/qubes-issues#5767
2020-05-05 16:14:10 +02:00
Marek Marczykowski-Górecki
dbe072b762
ext/admin: fix async/non-async mismatch 2020-04-30 21:22:29 +02:00
Artur Puzio
3e8f5a7a57
libvirt xen: Add gfx_passthru and device_video
Add gfx_passthru and device_video options to libvirt Xen domain template

Signed-off-by: Artur Puzio <contact@puzio.waw.pl>
2020-04-30 16:33:10 +02:00
Frédéric Pierret (fepitre)
d1147366f2
spec: drop R for PyQt4 2020-04-11 22:31:31 +02:00
Marek Marczykowski-Górecki
6d19496ab5
version 4.1.11 2020-04-10 05:44:01 +02:00
Marek Marczykowski-Górecki
e5edbc53fd
Merge remote-tracking branch 'origin/pr/332'
* origin/pr/332:
  tests: improve audio tests
  tests: fix qvm-copy-to-vm test
  tests: ensure proper cleanup of auxiliary process
2020-04-10 05:34:56 +02:00
Marek Marczykowski-Górecki
3b963030a6
Merge remote-tracking branch 'origin/pr/331'
* origin/pr/331:
  Fix multiple qmemman issues
2020-04-10 05:34:28 +02:00
Marek Marczykowski-Górecki
46f09f897c
Merge remote-tracking branch 'origin/pr/326'
* origin/pr/326:
  ext/admin: workaround for extension's __init__() called multiple times
  tests: teardown fixes
  travis: include core-qrexec in tests
  api/admin: (ext/admin) limit listing VMs based on qrexec policy
  api/internal: extract get_system_info() function
2020-04-10 05:32:58 +02:00
Marek Marczykowski-Górecki
8420adf973
tests: improve audio tests
- wait for the client be listed in dom0
- report parecord stderr
- allow up to 20ms to be missing, to account for potentially suspended
  device initially
2020-04-09 05:56:12 +02:00
Marek Marczykowski-Górecki
a6efd6a301
Merge remote-tracking branch 'origin/pr/334'
* origin/pr/334:
  collections.Callable -> collections.abc.Callable
2020-04-09 03:28:00 +02:00
Rusty Bird
6605bf406d
collections.Callable -> collections.abc.Callable
"Deprecated since version 3.3, will be removed in version 3.10"
- https://docs.python.org/3/library/collections.html
2020-04-07 21:30:21 +00:00
Rusty Bird
f9538a578d
tests/file: run_until_complete(); coro_maybe() for single arg 2020-04-07 21:04:04 +00:00
Rusty Bird
9122a14f94
tests/file: use self.loop 2020-04-07 21:04:02 +00:00
Marek Marczykowski-Górecki
3066190283
tests: fix qvm-copy-to-vm test
Make the check if remote file wasn't removed meaningful. Previously the
user didn't have permission to remote the source file, so even if the
tool would try, it would fail.
2020-04-01 17:57:12 +02:00
Marek Marczykowski-Górecki
1b7e2a5cbf
tests: ensure proper cleanup of auxiliary process
Various qrexec tests create auxiliary process (service_proc) as a local
variable. In case of test failure, process cleanup isn't called and may
lead to FD leaks and breaking subsequent tests.

Fix this by always saving such process instance in self.service_proc and
cleaning it up in self.tearDown() (this code is already there).

Add also waiting (and in case of timeout - killing) of a service call
process too.
2020-04-01 17:35:57 +02:00
Marek Marczykowski-Górecki
dd50e300c3
Fix multiple qmemman issues
First the main bug: when meminfo xenstore watch fires, in some cases
(just after starting some domain) XS_Watcher refreshes internal list of
domains before processing the event. This is done specifically to
include new domain in there. But the opposite could happen too - the
domain could be destroyed. In this case refres_meminfo() function raises
an exception, which isn't handled and interrupts the whole xenstore
watch loop. This issue is likely to be triggered by killing the domain,
as this way it could disappear shortly after writing updated meminfo
entry. In case of proper shutdown, meminfo-writer is stopped earlier and
do not write updates just before domain destroy.
Fix this by checking if the requested domain is still there just after
refreshing the list.

Then, catch exceptions in xenstore watch handling functions, to not
interrupt xenstore watch loop. If it gets interrupted, qmemman basically
stops memory balancing.

And finally, clear force_refresh_domain_list flag after refreshing the
domain list. That missing line caused domain refresh at every meminfo
change, making it use some more CPU time.

While at it, change "EOF" log message to something a bit more
meaningful.

Thanks @conorsch for capturing valuable logs.

Fixes QubesOS/qubes-issues#4890
2020-04-01 03:46:29 +02:00
Marek Marczykowski-Górecki
b11d6e058b
ext/admin: workaround for extension's __init__() called multiple times
... during tests.
qubes.ext.Extension class is a weird thing that tries to make each extension
a singleton. But this unfortunately have a side effect that __init__()
is called separately for each "instance" (created in Qubes()'s
__init__()), even though this is really the same object. During normal
execution this isn't an issue, because there is just one Qubes() object
instance. But during tests, multiple objects are created.

In this particular case, it caused PolicyCache() to be created twice and
the second one overriden the first one - without properly cleaning it
up. This leaks a file descriptor (inotify one). The fact that cleanup()
was called twice too didn't helped, because it was really called on
the same object, the one requiring cleanup was already gone.

Workaround this by checking if policy_cache field is initialize and
avoid re-initialize it. Also, on Qubes() object cleanup remove that
field, so it can be properly initialized on the next test iteration.
2020-03-31 01:57:22 +02:00
Marek Marczykowski-Górecki
0341cc5258
tests: teardown fixes
Add few missing app.close() calls on test teardown.
Fix socket cleanup in TC_00_QubesDaemonProtocol() - not only close the
FD, but also unregister it from asyncio event loop.
2020-03-28 03:23:29 +01:00
Marek Marczykowski-Górecki
a90e7e365e
travis: include core-qrexec in tests 2020-03-28 03:23:29 +01:00
Marek Marczykowski-Górecki
3f96c72ee3
api/admin: (ext/admin) limit listing VMs based on qrexec policy
Various Admin API calls, when directed at dom0, retrieve global system
view instead of a specific VM. This applies to admin.vm.List (called at
dom0 retrieve full VM list) and admin.Events (called at dom0 listen for
events of all the VMs). This makes it tricky to configure a management
VM with access to limited set of VMs only, because many tools require
ability to list VMs, and that would return full list.

Fix this issue by adding a filter to admin.vm.List and admin.Events
calls (using event handlers in AdminExtension) that filters the output
using qrexec policy. This version evaluates policy for each VM or event
(but loads only once). If the performance will be an issue, it can be
optimized later.

Fixes QubesOS/qubes-issues#5509
2020-03-28 03:23:28 +01:00
Marek Marczykowski-Górecki
8f0ec59f95
Merge remote-tracking branch 'origin/pr/330'
* origin/pr/330:
  gui: fixes from Marek's comments
  gui: improvements of feature keyboard layout checks
  tests: adapt tests for keyboard-layout
  gui: drop legacy qubes-keyboard support
2020-03-18 14:34:22 +01:00
Frédéric Pierret (fepitre)
577e4b24a6
gui: fixes from Marek's comments 2020-03-18 14:17:04 +01:00
Frédéric Pierret (fepitre)
f130ec0bf3
gui: improvements of feature keyboard layout checks 2020-03-18 09:46:21 +01:00
Frédéric Pierret (fepitre)
5cb0e61a84
tests: adapt tests for keyboard-layout 2020-03-18 09:17:37 +01:00
Frédéric Pierret (fepitre)
732e291ab6
gui: drop legacy qubes-keyboard support
Add check for keyboard-layout feature set
2020-03-18 09:17:37 +01:00
Marek Marczykowski-Górecki
29020d0c53
Merge branch 'feature-pre-set'
* feature-pre-set:
  Add pre- events to vm features
2020-03-17 19:26:53 +01:00
Marek Marczykowski-Górecki
d05592ba52
Add pre- events to vm features
Allow extensions to validate feature values, before it is set.
2020-03-17 18:46:12 +01:00
Marek Marczykowski-Górecki
ddc13d6b62
Merge remote-tracking branch 'origin/pr/329'
* origin/pr/329:
  tests: make PEP8 happier
  tests: fix already registered event impl
2020-03-17 12:07:18 +01:00
Frédéric Pierret (fepitre)
d8f80c9687
tests: make PEP8 happier 2020-03-16 23:22:36 +01:00
Frédéric Pierret (fepitre)
40156c3e78
tests: fix already registered event impl 2020-03-16 23:21:15 +01:00
Marek Marczykowski-Górecki
86d07460c7
version 4.1.10 2020-03-15 03:50:01 +01:00
Marek Marczykowski-Górecki
f62a861140
tests: fix test_053_qrexec_vm_service_eof_reverse
and not mark it as expected failure anymore. Note the removal of the
expected failure isn't just about the changes here, but also about the
actual fix on the qrexec side (ffafd01 "Fix not closed file descriptors in
qubes-rpc-multiplexer" commit in core-qrexec repository).
2020-03-15 02:55:18 +01:00
Marek Marczykowski-Górecki
5423ead27a
tests/vm_qrexec_gui: do not swallow stderr on failure
QubesVM.run_for_stdio() by default captures stderr. In case of call fail
(non-zero return code), captured stderr is included in the exception
object, but isn't printed by default CalledProcessError message.
Make it visible by:
 - handling CalledProcessError and including in the test failure message
   (when exception is captured already)
 - not capturing stderr (if no exception handling is present in the
   test)
2020-03-14 03:43:22 +01:00
Marek Marczykowski-Górecki
d033457f3d
tests: adjust for qrexec-client-vm exit code
It now consistently returns remote exit code, not a local one.
Check 'cat' exit code instead.
2020-03-14 03:43:17 +01:00
Marek Marczykowski-Górecki
7a750ea9c8
tests: QubesVM.run_for_stdio 2020-03-10 11:56:48 +01:00
Marek Marczykowski-Górecki
9ec86f3c41
Workaround different behaviour of asyncio's Process.communicate()
In asyncio's Process.communicate() input=None does not close stdin.
Workaround it by using b'' instead of None.

https://bugs.python.org/issue39744
2020-03-10 11:56:48 +01:00
Marek Marczykowski-Górecki
3ce4e5eaa5
tests: adjust SystemTestCase.create_remote_file to create executables
The main use case for this function is to create qrexec services in VMs.
Since qrexec now require service scripts to be executable, make
create_remote_file() adjust permissions.
2020-03-10 11:56:47 +01:00