* origin/pr/326:
ext/admin: workaround for extension's __init__() called multiple times
tests: teardown fixes
travis: include core-qrexec in tests
api/admin: (ext/admin) limit listing VMs based on qrexec policy
api/internal: extract get_system_info() function
- wait for the client be listed in dom0
- report parecord stderr
- allow up to 20ms to be missing, to account for potentially suspended
device initially
Make the check if remote file wasn't removed meaningful. Previously the
user didn't have permission to remote the source file, so even if the
tool would try, it would fail.
Various qrexec tests create auxiliary process (service_proc) as a local
variable. In case of test failure, process cleanup isn't called and may
lead to FD leaks and breaking subsequent tests.
Fix this by always saving such process instance in self.service_proc and
cleaning it up in self.tearDown() (this code is already there).
Add also waiting (and in case of timeout - killing) of a service call
process too.
First the main bug: when meminfo xenstore watch fires, in some cases
(just after starting some domain) XS_Watcher refreshes internal list of
domains before processing the event. This is done specifically to
include new domain in there. But the opposite could happen too - the
domain could be destroyed. In this case refres_meminfo() function raises
an exception, which isn't handled and interrupts the whole xenstore
watch loop. This issue is likely to be triggered by killing the domain,
as this way it could disappear shortly after writing updated meminfo
entry. In case of proper shutdown, meminfo-writer is stopped earlier and
do not write updates just before domain destroy.
Fix this by checking if the requested domain is still there just after
refreshing the list.
Then, catch exceptions in xenstore watch handling functions, to not
interrupt xenstore watch loop. If it gets interrupted, qmemman basically
stops memory balancing.
And finally, clear force_refresh_domain_list flag after refreshing the
domain list. That missing line caused domain refresh at every meminfo
change, making it use some more CPU time.
While at it, change "EOF" log message to something a bit more
meaningful.
Thanks @conorsch for capturing valuable logs.
FixesQubesOS/qubes-issues#4890
... during tests.
qubes.ext.Extension class is a weird thing that tries to make each extension
a singleton. But this unfortunately have a side effect that __init__()
is called separately for each "instance" (created in Qubes()'s
__init__()), even though this is really the same object. During normal
execution this isn't an issue, because there is just one Qubes() object
instance. But during tests, multiple objects are created.
In this particular case, it caused PolicyCache() to be created twice and
the second one overriden the first one - without properly cleaning it
up. This leaks a file descriptor (inotify one). The fact that cleanup()
was called twice too didn't helped, because it was really called on
the same object, the one requiring cleanup was already gone.
Workaround this by checking if policy_cache field is initialize and
avoid re-initialize it. Also, on Qubes() object cleanup remove that
field, so it can be properly initialized on the next test iteration.
Add few missing app.close() calls on test teardown.
Fix socket cleanup in TC_00_QubesDaemonProtocol() - not only close the
FD, but also unregister it from asyncio event loop.
Various Admin API calls, when directed at dom0, retrieve global system
view instead of a specific VM. This applies to admin.vm.List (called at
dom0 retrieve full VM list) and admin.Events (called at dom0 listen for
events of all the VMs). This makes it tricky to configure a management
VM with access to limited set of VMs only, because many tools require
ability to list VMs, and that would return full list.
Fix this issue by adding a filter to admin.vm.List and admin.Events
calls (using event handlers in AdminExtension) that filters the output
using qrexec policy. This version evaluates policy for each VM or event
(but loads only once). If the performance will be an issue, it can be
optimized later.
FixesQubesOS/qubes-issues#5509
* origin/pr/330:
gui: fixes from Marek's comments
gui: improvements of feature keyboard layout checks
tests: adapt tests for keyboard-layout
gui: drop legacy qubes-keyboard support
and not mark it as expected failure anymore. Note the removal of the
expected failure isn't just about the changes here, but also about the
actual fix on the qrexec side (ffafd01 "Fix not closed file descriptors in
qubes-rpc-multiplexer" commit in core-qrexec repository).
QubesVM.run_for_stdio() by default captures stderr. In case of call fail
(non-zero return code), captured stderr is included in the exception
object, but isn't printed by default CalledProcessError message.
Make it visible by:
- handling CalledProcessError and including in the test failure message
(when exception is captured already)
- not capturing stderr (if no exception handling is present in the
test)
The main use case for this function is to create qrexec services in VMs.
Since qrexec now require service scripts to be executable, make
create_remote_file() adjust permissions.
If any test-* VMs remains from previous test run, there are removed
before test. self.app doesn't exist at this point, so don't require it
in self.remove_vms().
This commit adds a test case for the QubesVM class's is_fully_usable
method. The verified scenarios are as follows:
* The VM has qrexec enabled, and the qrexec service has been
successfully started.
(The VM becomes "fully usable" in this case.)
* The VM has qrexec enabled, and the qrexec service has failed to start
(Error handling case; the VM is *not* fully usable.)
* The VM does *not* have qrexec enabled.
(The VM becomes "fully usable" in this case.)
Prior to this commit, a properly configured Linux HVM would not
transition from the 'Transient' state to the 'Running' state according
to qvm-ls output, even if the HVM in question had the 'qrexec' feature
disabled.
This issue is caused by an unconditional qrexec check in the
'on_domain_is_fully_usable' method, and is resolved by adding
a check that short-circuits the qrexec check if the aforementioned
feature is not enabled for the VM in question.
* origin/pr/295:
tests: fix tag name in audiovm test
tests: ensure notin while setting Audio/Gui VM
gui: add checks for changing/removing guivm
audio: add checks for changing/removing audiovm
audio/gui: use simply vm.tags instead of list()
tests: fix tests for gui/audio vm
Make pylint happy
gui/audio: fixes from Marek's comments
Allow AudioVM to be ran after any attached qubes
Allow GuiVM to be ran after any attached qubes
xid: ensure vm is not running
tests: fix missing default audiovm and guivm tags
gui, audio: better handling of start/stop guivm/audiovm
gui, audio: ensure guivm and audiovm tag are set
Support for AudioVM
Wait specifically for pulseaudio to start in the VM, instead of just
waiting few seconds.
Also, improve failure message to distinguish total lack of audio from
just missing few samples.
Volume with save_on_stop=False and snap_on_start=False exists only when
VM is running and this is already handled under is_dirty() condition.
For non-running VMs, skip LVM resize step and only update internal size
attribute.
FixesQubesOS/qubes-issues#5645
When volume is resized, underlying LVM is adjusted if exists. For
persistent volumes it is enough, as generally the driver prefers
underlying LVM size than internal attribute. But volatile volume is
removed at vm shutdown, so that value is lost.
Fix it by updating Volume.size attribute too, to be updated in qubes.xml
later too.
FixesQubesOS/qubes-issues#5645
* origin/pr/316:
storage/reflink: trivial style tweaks
storage/reflink: comment on _get_size() use in start()
storage/reflink: don't bother using _get_size() in create()
storage/reflink: delete all images at beginning of create()
storage/reflink: get VM dir from less arbitrary-looking path
storage/reflink: factor out _remove_all_images()
storage/reflink: _remove_incomplete_{files -> images}()
storage/reflink: omit redundant comment
Ensure that there are no leftover image files for the volume, e.g. from
an unsuccessful removal of a previous incarnation of this vid, or from
an messily restored pool filesystem backup. We don't want to preserve
any stale data (revisions) or metadata (size) in the new incarnation.
