First, TemplateVM is not used anymore (see previous commit). Second,
don't harcode on client side that "only TemplateVM can be a template for
any VM" (which actually isn't true: AppVM can be a template for DispVM).
Very few calls at client side really needs VM class name. So, even in
non-blind mode use just QubesVM class, to avoid strange cases depending
on blind mode being enabled or not. Then, have VM class name in 'klass'
property. If known at object creation time, cache it, otherwise query
qubesd at first access.
It may happen that when client handle the event, domain no longer
exists. This is for example common for DispVMs, which get removed just
after shutdown.
This will cause some events to be dropped, but one can enable blind
mode, to get them anyway (because it will not cause KeyError, even if
domain is already removed).
QubesOS/qubes-issues#3100
This allows to perform actions on objects (VM, storage etc), without
listing them. This is useful when calling VM have minimal permissions
and only selected actions are allowed.
This means that app.domains['some-name'] will not raise KeyError, even
when domain do not exists. But performing actual action (like
vm.start()) will fail in that case.
Booting a VM from cdrom require attaching the device before VM startup,
which is possible only in persistent mode. But for qvm-start --cdrom
adding a cdrom only temporarily, use new update_persistence() function
to convert the assignment to temporary one.
FixesQubesOS/qubes-issues#3055
Abort tar process after extracting requested files - do not parse the
archive until the end (possibly tens of GB later).
FixesQubesOS/qubes-issues#2986
This may be confiusing, for example one may think that
`qvm-prefs --unset vmname netvm` will make vmname network-disconnected.
This type of mistakes may have severe security consequence, so better
drop those option names.
QubesOS/qubes-issues#3002
cc @rootkovska
Add option to uniformly start new DispVM from either VM or Dom0. This
use DispVMWrapper, which translate it to either qrexec call to $dispvm,
or (in dom0) to appropriate Admin API call to create fresh DispVM
first.
This require abandoning registering --all and --exclude by
QubesArgumentParser, because we need to add --dispvm mutually exclusive
with those two. But actually handling those two options is still done by
QubesArgumentParser.
This also updates man page and tests.
FixesQubesOS/qubes-issues#2974
This is a wrapper to use `$dispvm` target of qrexec call, just like any
other service call in qubesadmin module - using vm.run_service().
When running in dom0, qrexec-client-vm is not available, so DispVM needs
to be created "manually", using appropriate Admin API call
(admin.vm.CreateDisposable).
QubesOS/qubes-issues#2974
When data block is smaller than 4096 (and no EOF is reached), python's
io.read() will call read(2) again to get more data. This may deadlock if
the other end of connection will write anything only after receiveing
data (which is the case for qubes.Filecopy).
Disable this buffering by using syscall wrappers directly. To not affect
performance that much, increase buffer size to 64k.
FixesQubesOS/qubes-issues#2948
* devel-6:
qvm-ls: fix total VM size reporting
doc: update manpage of qvm-service
tools: qvm-service tool
tests: too much copy&paste
features: serialize True as '1'
tools/qvm-start-gui: add --force-stubdomain options
tools/qvm-shutdown: fix help message
tools/qvm-shutdown: drop --force option, it isn't supported anymore
New qvm-backup tool can either use pre-existing backup profile
(--profile), or - when running in dom0 - can create new one based on
used options (--save-profile).
This commit add a tool itself, update its man page, and add tests for
it.
FixesQubesOS/qubes-issues#2931
Do not check for qubesd socket (at module import time), because if not
running at this precise time, it will lead to wrong choice. And a weird
error message in consequence (looking for qrexec-client-vm in dom0).
FixesQubesOS/qubes-issues#2917
* devel-2-qvm-run-1:
Make pylint happy
tools/qvm-run: fix handling EOF
tests: mark qvm-run tests with "expected failure"
tools/qvm-run: fix handling copying stdin to the process
since qvm-run use multiprocessing.Process now, stdin sent to it is
processed in separate process and doesn't come back to
TestApp.actual_calls (self.app). Annotate tests for now, to be fixed
later.
The most important part is fixing resize handling - call size_func
before data_func, but after tar gets initial data (and output file
size).
But other than that, it makes the process a little faster.
QubesOS/qubes-issues#1214
Opt for a simple one-liner error messages, instead of meaningless stack
trace (it's most of the time about qubesd responding with error, so the
stack trace of actual problem is elsewhere).
This will be less realistic (private.img of 2MB?!), but makes tests much
quicker. And since tar is used to make files sparse, we don't really
test multi-part archives anyway.
Based on "backup compatibility" tests, which manually assemble the
backup. This is because we don't have access to actual backup creation
code here.
QubesOS/qubes-issues#1214
Launch stdin copy loop in a separate process (multiprocessing.Process)
and terminate it when target process is terminated.
Another idea here was threads, but there is no API to kill a thread
waiting on read().
VMs can have runtime dependencies - for example it isn't possible to
shutdown netvm used by some other running VM(s). Since client-side tools
may not have full knowledge about rules enforcing those dependencies
(for example may not have access to 'netvm' property), implement
best-effort approach:
1. Try to shutdown all requested VMs
2. For those where shutdown request succeed, wait for actual shutdown
3. For others - go back to step 1
And loop until all VMs are shutdown, or all shutdown requests fails.
Allow programming different responses for the same request when called
multiple times. This is useful for example for shutdown tests - first
domain is running, but after issuing shutdown request is is not.
Do not close event loop in utility function - handle it only in main().
For this reason, change appropriate functions to coroutines.
FixesQubesOS/qubes-issues#2865
Don't fail the whole process when "just" appmenus import fails.
But if data import fails, remove the VM
Also update for vm.run_service_for_stdio raising CalledProcessError.
Don't start GUI daemon for given VM twice when qvm-start-gui was started
during VM startup (while waiting for qrexec startup). This is especially
common while running tests.
Report failed qubes.SetMonitorLayout as warning (instead of unhandled
exception).
Clear VM cache on qubesd reconnect.
Fix logging.
qubesd may be restarted during different stages of connection - either
while attempting to connect, or while already listening on events.
Adjust exception list accordingly.
This is especially important for qvm-start-gui - otherwise it crashes on
qubesd restart.
This way we don't need separate admin.vm.Clone call, which is tricky to
handler properly with policy.
A VM may not have access to all the properties and other metadata, so
add ignore_errors argument, for best-effort approach (copy what is
possible). In any case, failure of cloning VM data fails the whole
operation.
When operation fails, VM is removed.
While at it, allow to specify alternative VM class - this allows
morphing one VM into another (for example AppVM -> StandaloneVM).
Adjust qvm-clone tool and tests accordingly.
QubesOS/qubes-issues#2622
Enable 'qrexec' VM feature to wait for qrexec initialization - it is
required to call qubes.PostInstall service. If VM start fails, assume
there is no qrexec and drop that feature.
Since Admin API, qvm-ls takes a long time to complete. Therefore,
Corporate Headquarters commanded that a Enterprise Spinner is to be
implemented and mandated it's use unto us.
We take amusement from its endless gyrations.
Empty parameter value is encoded as b'parameter\0\0', so we can't simply
read the data until b'\0\0', because it isn't necessary event end.
Instead, read event parts separately, according to specification.
Use newly introduced payload_stream= argument to qubesd_call to pass
data directly from some file-like object - without loading it all into
memory.
QubesOS/qubes-issues#853
Add qubesd_call(..., payload_stream=...) argument to allow streaming
payload directly from some file/process stdout. This is mainly (only?)
useful for admin.vm.volume.Import, where disk volume raw data is passed
to the service.
Since migration to Admin API, qvm-ls no longer have list of all VM
properties in advance, so can't really validate fields list. Simply
assume that unknown columns are properties.
Don't terminate qvm-run on any SIGCHLD, check if the process we're
waiting for have finished.
Currently the only situation when it's broken is a test (which starts
additional process, whose SIGCHLD may be caught here), but lets do not
assume that much (starting only one process) about environment.
Make it easy to retrieve DeviceInfo object out of DeviceAssignment
object. The only missing piece of information for that is device class,
so add it. Make it optional, as it can be filled on demand when passing
the object through DeviceCollection (either by listing devices, or
attaching/detaching).
This also makes DeviceCollection._device method not needed anymore.
It is supported only from dom0, but it's still useful to have, to save
on simultaneous vchan connections (only waiting for MSG_DATA_EXIT_CODE).
This is especially important for Windows VMs, as qrexec-agent there have
pretty low limit on simultaneous connections (about 20).
Make qvm-run use it.
Since we use qubes.VMShell service now and send requested command on its
stdin, we need to terminate that shell after requested command -
otherwise the service will not terminate automatically waiting for
further input (next commands).
qubesd_call expect "bytes" type. Additionally serialize false value as
empty string (which is treated by python as false value), because
otherwise would be serialized to (non-empty) string, which is true value
in python.
Do not color qvm-run diagnostic messages, but also avoid ANSI control
sequences in logs. While at it, do not print 'Running ...' message when
--pass-io is used.
