Commit Graph

6108 Commits

Author SHA1 Message Date
Rusty Bird
6605bf406d
collections.Callable -> collections.abc.Callable
"Deprecated since version 3.3, will be removed in version 3.10"
- https://docs.python.org/3/library/collections.html
2020-04-07 21:30:21 +00:00
Rusty Bird
f9538a578d
tests/file: run_until_complete(); coro_maybe() for single arg 2020-04-07 21:04:04 +00:00
Rusty Bird
9122a14f94
tests/file: use self.loop 2020-04-07 21:04:02 +00:00
Marek Marczykowski-Górecki
3066190283
tests: fix qvm-copy-to-vm test
Make the check if remote file wasn't removed meaningful. Previously the
user didn't have permission to remote the source file, so even if the
tool would try, it would fail.
2020-04-01 17:57:12 +02:00
Marek Marczykowski-Górecki
1b7e2a5cbf
tests: ensure proper cleanup of auxiliary process
Various qrexec tests create auxiliary process (service_proc) as a local
variable. In case of test failure, process cleanup isn't called and may
lead to FD leaks and breaking subsequent tests.

Fix this by always saving such process instance in self.service_proc and
cleaning it up in self.tearDown() (this code is already there).

Add also waiting (and in case of timeout - killing) of a service call
process too.
2020-04-01 17:35:57 +02:00
Marek Marczykowski-Górecki
dd50e300c3
Fix multiple qmemman issues
First the main bug: when meminfo xenstore watch fires, in some cases
(just after starting some domain) XS_Watcher refreshes internal list of
domains before processing the event. This is done specifically to
include new domain in there. But the opposite could happen too - the
domain could be destroyed. In this case refres_meminfo() function raises
an exception, which isn't handled and interrupts the whole xenstore
watch loop. This issue is likely to be triggered by killing the domain,
as this way it could disappear shortly after writing updated meminfo
entry. In case of proper shutdown, meminfo-writer is stopped earlier and
do not write updates just before domain destroy.
Fix this by checking if the requested domain is still there just after
refreshing the list.

Then, catch exceptions in xenstore watch handling functions, to not
interrupt xenstore watch loop. If it gets interrupted, qmemman basically
stops memory balancing.

And finally, clear force_refresh_domain_list flag after refreshing the
domain list. That missing line caused domain refresh at every meminfo
change, making it use some more CPU time.

While at it, change "EOF" log message to something a bit more
meaningful.

Thanks @conorsch for capturing valuable logs.

Fixes QubesOS/qubes-issues#4890
2020-04-01 03:46:29 +02:00
Marek Marczykowski-Górecki
b11d6e058b
ext/admin: workaround for extension's __init__() called multiple times
... during tests.
qubes.ext.Extension class is a weird thing that tries to make each extension
a singleton. But this unfortunately have a side effect that __init__()
is called separately for each "instance" (created in Qubes()'s
__init__()), even though this is really the same object. During normal
execution this isn't an issue, because there is just one Qubes() object
instance. But during tests, multiple objects are created.

In this particular case, it caused PolicyCache() to be created twice and
the second one overriden the first one - without properly cleaning it
up. This leaks a file descriptor (inotify one). The fact that cleanup()
was called twice too didn't helped, because it was really called on
the same object, the one requiring cleanup was already gone.

Workaround this by checking if policy_cache field is initialize and
avoid re-initialize it. Also, on Qubes() object cleanup remove that
field, so it can be properly initialized on the next test iteration.
2020-03-31 01:57:22 +02:00
Marek Marczykowski-Górecki
0341cc5258
tests: teardown fixes
Add few missing app.close() calls on test teardown.
Fix socket cleanup in TC_00_QubesDaemonProtocol() - not only close the
FD, but also unregister it from asyncio event loop.
2020-03-28 03:23:29 +01:00
Marek Marczykowski-Górecki
a90e7e365e
travis: include core-qrexec in tests 2020-03-28 03:23:29 +01:00
Marek Marczykowski-Górecki
3f96c72ee3
api/admin: (ext/admin) limit listing VMs based on qrexec policy
Various Admin API calls, when directed at dom0, retrieve global system
view instead of a specific VM. This applies to admin.vm.List (called at
dom0 retrieve full VM list) and admin.Events (called at dom0 listen for
events of all the VMs). This makes it tricky to configure a management
VM with access to limited set of VMs only, because many tools require
ability to list VMs, and that would return full list.

Fix this issue by adding a filter to admin.vm.List and admin.Events
calls (using event handlers in AdminExtension) that filters the output
using qrexec policy. This version evaluates policy for each VM or event
(but loads only once). If the performance will be an issue, it can be
optimized later.

Fixes QubesOS/qubes-issues#5509
2020-03-28 03:23:28 +01:00
Marek Marczykowski-Górecki
8f0ec59f95
Merge remote-tracking branch 'origin/pr/330'
* origin/pr/330:
  gui: fixes from Marek's comments
  gui: improvements of feature keyboard layout checks
  tests: adapt tests for keyboard-layout
  gui: drop legacy qubes-keyboard support
2020-03-18 14:34:22 +01:00
Frédéric Pierret (fepitre)
577e4b24a6
gui: fixes from Marek's comments 2020-03-18 14:17:04 +01:00
Frédéric Pierret (fepitre)
f130ec0bf3
gui: improvements of feature keyboard layout checks 2020-03-18 09:46:21 +01:00
Frédéric Pierret (fepitre)
5cb0e61a84
tests: adapt tests for keyboard-layout 2020-03-18 09:17:37 +01:00
Frédéric Pierret (fepitre)
732e291ab6
gui: drop legacy qubes-keyboard support
Add check for keyboard-layout feature set
2020-03-18 09:17:37 +01:00
Marek Marczykowski-Górecki
29020d0c53
Merge branch 'feature-pre-set'
* feature-pre-set:
  Add pre- events to vm features
2020-03-17 19:26:53 +01:00
Marek Marczykowski-Górecki
d05592ba52
Add pre- events to vm features
Allow extensions to validate feature values, before it is set.
2020-03-17 18:46:12 +01:00
Marek Marczykowski-Górecki
ddc13d6b62
Merge remote-tracking branch 'origin/pr/329'
* origin/pr/329:
  tests: make PEP8 happier
  tests: fix already registered event impl
2020-03-17 12:07:18 +01:00
Frédéric Pierret (fepitre)
d8f80c9687
tests: make PEP8 happier 2020-03-16 23:22:36 +01:00
Frédéric Pierret (fepitre)
40156c3e78
tests: fix already registered event impl 2020-03-16 23:21:15 +01:00
Marek Marczykowski-Górecki
86d07460c7
version 4.1.10 2020-03-15 03:50:01 +01:00
Marek Marczykowski-Górecki
f62a861140
tests: fix test_053_qrexec_vm_service_eof_reverse
and not mark it as expected failure anymore. Note the removal of the
expected failure isn't just about the changes here, but also about the
actual fix on the qrexec side (ffafd01 "Fix not closed file descriptors in
qubes-rpc-multiplexer" commit in core-qrexec repository).
2020-03-15 02:55:18 +01:00
Marek Marczykowski-Górecki
5423ead27a
tests/vm_qrexec_gui: do not swallow stderr on failure
QubesVM.run_for_stdio() by default captures stderr. In case of call fail
(non-zero return code), captured stderr is included in the exception
object, but isn't printed by default CalledProcessError message.
Make it visible by:
 - handling CalledProcessError and including in the test failure message
   (when exception is captured already)
 - not capturing stderr (if no exception handling is present in the
   test)
2020-03-14 03:43:22 +01:00
Marek Marczykowski-Górecki
d033457f3d
tests: adjust for qrexec-client-vm exit code
It now consistently returns remote exit code, not a local one.
Check 'cat' exit code instead.
2020-03-14 03:43:17 +01:00
Marek Marczykowski-Górecki
7a750ea9c8
tests: QubesVM.run_for_stdio 2020-03-10 11:56:48 +01:00
Marek Marczykowski-Górecki
9ec86f3c41
Workaround different behaviour of asyncio's Process.communicate()
In asyncio's Process.communicate() input=None does not close stdin.
Workaround it by using b'' instead of None.

https://bugs.python.org/issue39744
2020-03-10 11:56:48 +01:00
Marek Marczykowski-Górecki
3ce4e5eaa5
tests: adjust SystemTestCase.create_remote_file to create executables
The main use case for this function is to create qrexec services in VMs.
Since qrexec now require service scripts to be executable, make
create_remote_file() adjust permissions.
2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
27e8d9ef4d
Make /etc/qubes-rpc/qubes.ConnectTCP executable
QubesOS/qubes-issues#5686
2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
2460adbdef
tests: socket-based qrexec services
QubesOS/qubes-issues#3912
2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
3f5fb0de9e
tests: hide git output when checking local checkout 2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
534de9bc1c
tests: fix cleanup before test
If any test-* VMs remains from previous test run, there are removed
before test. self.app doesn't exist at this point, so don't require it
in self.remove_vms().
2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
e6e4908e71
version 4.1.9 2020-03-10 03:08:42 +01:00
Marek Marczykowski-Górecki
879ee9e7d6
api/internal: extract get_system_info() function
This will be useful in other places too.

QubesOS/qubes-issues#5099
2020-03-09 19:25:10 +01:00
Marek Marczykowski-Górecki
135eda0582
Merge remote-tracking branch 'origin/pr/320'
* origin/pr/320:
  storage/lvm: don't try to resize non-existing lvm volumes
  storage/lvm: update volume.size attribute on resize
2020-03-09 18:34:49 +01:00
M. Vefa Bicakci
5d755eb065
tests: Add a test case for QubesVM.is_fully_usable
This commit adds a test case for the QubesVM class's is_fully_usable
method. The verified scenarios are as follows:

* The VM has qrexec enabled, and the qrexec service has been
  successfully started.
  (The VM becomes "fully usable" in this case.)

* The VM has qrexec enabled, and the qrexec service has failed to start
  (Error handling case; the VM is *not* fully usable.)

* The VM does *not* have qrexec enabled.
  (The VM becomes "fully usable" in this case.)
2020-03-09 00:22:51 -04:00
M. Vefa Bicakci
2d55841a5e
qubesvm: Let HVMs enter 'Running' state
Prior to this commit, a properly configured Linux HVM would not
transition from the 'Transient' state to the 'Running' state according
to qvm-ls output, even if the HVM in question had the 'qrexec' feature
disabled.

This issue is caused by an unconditional qrexec check in the
'on_domain_is_fully_usable' method, and is resolved by adding
a check that short-circuits the qrexec check if the aforementioned
feature is not enabled for the VM in question.
2020-03-08 22:39:16 -04:00
M. Vefa Bicakci
d81d6a9267
qubes module: Typo fix
This commit fixes a typo, where the exception type DontSave was used,
even though the name of the function dontsave was intended.
2020-03-08 22:39:16 -04:00
M. Vefa Bicakci
7980d94807
qvm-sync-clock: Do not fail if clockvm is not set 2020-03-08 22:39:11 -04:00
Marek Marczykowski-Górecki
16bdeea2c0
Merge remote-tracking branch 'origin/pr/295'
* origin/pr/295:
  tests: fix tag name in audiovm test
  tests: ensure notin while setting Audio/Gui VM
  gui: add checks for changing/removing guivm
  audio: add checks for changing/removing audiovm
  audio/gui: use simply vm.tags instead of list()
  tests: fix tests for gui/audio vm
  Make pylint happy
  gui/audio: fixes from Marek's comments
  Allow AudioVM to be ran after any attached qubes
  Allow GuiVM to be ran after any attached qubes
  xid: ensure vm is not running
  tests: fix missing default audiovm and guivm tags
  gui, audio: better handling of start/stop guivm/audiovm
  gui, audio: ensure guivm and audiovm tag are set
  Support for AudioVM
2020-03-09 01:48:01 +01:00
Marek Marczykowski-Górecki
36e9afe1ff
Merge branch 'tests20200301'
* tests20200301:
  tests/salt: don't depend on initial tags set
  tests: make audio tests less racy
2020-03-09 01:45:58 +01:00
Marek Marczykowski-Górecki
4cfc423d5e
tests: fix tag name in audiovm test 2020-03-09 01:11:58 +01:00
Marek Marczykowski-Górecki
84ec49e98d
Merge remote-tracking branch 'origin/pr/322'
* origin/pr/322:
  Ensure empty service value delete /var/run/qubes-service/ file
  services: fixes from Marek's comments
  config: fix mistake in path for services
  tests: add/remove services in dom0
  config: specify dom0 services path
  services: handle dom0 write permission errors
  services: handle /var/run/qubes/'SERVICE NAME' for dom0
  services: make PEP8 happier
2020-03-09 00:37:44 +01:00
Frédéric Pierret (fepitre)
a61bb9a0cb
Ensure empty service value delete /var/run/qubes-service/ file
- Add tests and stop patch path from Marek's comment
2020-03-08 23:08:48 +01:00
Frédéric Pierret (fepitre)
532d9a3a98
tests: ensure notin while setting Audio/Gui VM 2020-03-08 22:59:46 +01:00
Frédéric Pierret (fepitre)
90584c487a
gui: add checks for changing/removing guivm 2020-03-08 17:07:29 +01:00
Frédéric Pierret (fepitre)
c36ad38eb3
audio: add checks for changing/removing audiovm 2020-03-08 17:07:29 +01:00
Frédéric Pierret (fepitre)
9051aff15a
audio/gui: use simply vm.tags instead of list() 2020-03-08 17:07:29 +01:00
Frédéric Pierret (fepitre)
4b5ae0833b
tests: fix tests for gui/audio vm 2020-03-08 17:07:24 +01:00
Frédéric Pierret (fepitre)
43786e1478
Make pylint happy
- Drop unused 'asyncio'
- Drop useless 'else' after return
2020-03-08 17:05:34 +01:00
Frédéric Pierret (fepitre)
7506482d08
gui/audio: fixes from Marek's comments 2020-03-08 17:05:34 +01:00