* paranoid-restore:
backup/restore: make error reporting work also for StandaloneVM based DispVM
backup/restore: better error detection for --paranoid-mode
doc: document 'tag-created-vm-with' feature
tests: add paranoid backup restore unit tests
tests: remove extra empty lines
tests: extend run_service mockup for pre-recorded output
rpm: add BR: python3-lxml and python3-xcffib
backup/restore: add option for unattended restore and extracting log
tools: remove obsolete _want_app argument
Add "paranoid restore" mode
rpm/deb: add dependency on scrypt
utils: add simple locking primitive
backup/restore: use qfile-unpacker in a VM, request disk space monitoring
backup/restore: option for alternative qrexec service
backup/restore: improve error message about restoring tags
backup/restore: distinguish dom0 by name
Xterm doesn't preserve exit code of the process running inside. This
means, the whole xterm always exits with 0, even if qvm-backup-restore
failed.
Fix this by printing the exit code at the end to the log and then extract
that last line from the log on the calling side. This way we can also
distinguish whether qvm-backup-restore or xterm failed.
Allow running unattended, with qvm-backup-restore --passphrase-file.
This require few modifications:
- copy the passphrase file into the DisposableVM (that VM knows the
passphrase anyway, so there is no extra data leak)
- close the terminal when operation finishes
Closing the terminal would eliminate almost all the feedback (operation
log, errors, warnings etc), so write it into a file in DisposableVM and
later extract it and show on the stdout. Similar to qvm-run, color it
red as a content coming from a VM.
QubesOS/qubes-issues#5310
Having Admin API, it is possible to do this properly now:
- create DisposableVM
- assign it proper permissions to create VMs and control those created
VMs
- run restore process inside
- cleanup DisposableVM afterwards
Since the RestoreInDisposableVM class contains de facto reverse parser
for qvm-backup-restore command line, add a test that will spot when it
gets out of sync.
This feature depends on modifications in various other components,
including:
- linux-utils and core-agent-linux for update qfile-unpacker
- core-admin for qrexec policy modification
QubesOS/qubes-issues#5310
Standard python locking modules do not provide detection if lock-holding
process is still alive. Add a simple wrapper around fcntl.lockf that do
just that.
Make it possible to use qvm-backup-restore in a VM. This commit is about
accessing backup itself, when stored in another VM. This involve using
qfile-unpacker instead of qfile-dom0-unpacker and also requesting disk
space monitoring, as a VM probably won't have enough space to hold the
whole backup at once.
QubesOS/qubes-issues#4791QubesOS/qubes-issues#5310
Allow setting alternative qrexec service to retrieve backup content. The
service API is slightly different than the default one: it will get only
list of files/directories to extract on its stdin, but not backup
location. The latter could be provided as a service argument, or using
other out-of-band mechanism.
This will be useful for paranoid backup restore mode, to take away
control over location/command from sandboxed qvm-backup-restore process.
QubesOS/qubes-issues#5310
Before reporting a tag as not restored, verify if it really wasn't
restored. Generally created-by-* tags cannot be created manually. But
when restoring a backup in dom0, created-by-dom0 tag is added, which in
many cases will match what want to be restored.
Adjust tests to check this too.
- `qvm-template list`: show template state
- `qvm-template list`: only call qubes.TemplateSearch once
- `qvm-template list`: use `qubesadmin.tools.print_table()` instead of own implementation
- `qvm-template download`: custom progress bar
- Use `run_service` instead of own implementation
- Remove some erroneous/redundant lines
* origin/pr/143:
qvm-start-daemon: common_guid_args is now a staticmethod
tests: kde_args are passed with property of launcher
Fix missing semi-colon and new line
Handle KDE with specific arg/desktop file
core-admin-client side had less strict rules for hostname than core-admin,
leading to unexpected empty qubesd response. Fixed by copying the logic from core-admin.
Qubes().domains.refresh_cache() tries to preserve cached VM objects if
the class matches - this way if an application keeps reference to any,
it will still be the same as freshly obtained from the collection, and
also it will receive cache updates/invalidates based on events.
The check for class change was invalid - on core-admin-client side we
have just one QubesVM class with 'klass' attribute. This leads to VM
objects being disconnected from VMCollection and stale properties cache
there (because they no longer receive events).
Fix the check.
And also add a test if indeed the same object is returned.
Allow configuring options per VM or globally per GuiVM. The
qvm-start-daemon program reads the options from VM features, and
generates a configuration file for qubes-guid.
Requires QubesOS/qubes-gui-daemon#47 (customizing the configuration
file).
Do not require permission to list sys-gui itself just to get keyboard
layout. Listing itself is not sensitive (sys-gui knows it exists), but
it will make other tools request its properties, which may not be
desirable.
Application that runs EventsDispatcher can safely use also cache , which
greatly improve performance. This is because cache then is properly
updated/invalidated when needed.
Instead of modifying each application to explicitly enable cache based
on this simple rule, make it implicit when EventsDispatcher is created.
Do not enable caching when EventsDispatcher is created only temporarily
in wait_for_domain_shutdown.
QubesOS/qubes-issues#3293
Both tools issue a large number of Admin API calls and greatly benefit
from a cache filled with a single per-vm Admin API call
(admin.vm.property.GetAll). In case of qvm-ls, this also saves multiple
admin.vm.CurrentState calls (power state is given in the admin.vm.List
response too).
QubesOS/qubes-issues#3293
Power state changes are signaled with events too, so it is possible to
cache it and update/invalidate cache with events.
Additionally, admin.vm.List returns a power state, so the cache can be
populated early. This in particular greatly improves qvm-ls performance -
eliminate admin.vm.CurrentState call at all.
QubesOS/qubes-issues#3293
Qrexec itself does not support loopback calls. Since a call into dom0
is a common case for GUIVM (which may be dom0) to do, add a convenient
handling inside vm.run_service() function.
FixesQubesOS/qubes-issues#5140
When caching is enabled, reduce number of calls by getting all the
properties at once. If the call is not available (for example because of
the policy), fallback to getting individual values.
QubesOS/qubes-issues#5415
Reduce Admin API calls by caching returned values. The cache is not
enabled by default, because it could result in stale values being
returned. It can be enabled by setting 'cache_enabled' to True on
Qubes() object. This is safe in two cases:
- the application don't care about changed values - like a short-lived
process that retrieve values once (for example qvm-ls)
- the application listen for events and invalidate cache when property
is changed
For the second case, invalidating the cache on appropriate event
(property-set:*, property-reset:*) is done before calling other event
handlers. This is because the event may try to access the property value
(not necessary from the event arguments), so we need to be sure it will
see the new value.
FixesQubesOS/qubes-issues#5415
When qubesd returns an name of VM or other object, as part of another
call (reading a property, listing devices etc), it's safe to assume that
object exists. Do not try to list it, which could be prevented by qrexec
policy. This means a VM object would be returned (for example in
vm.netvm property), which potentially could not be listed through
app.domains collection. This may lead to some corner cases, but
generally should ease handling of restricted policy.
This does not affect practical information the management VM have access
too, as those names are already returned. It's just client side python
wrapper that didn't allowed to access them.
QubesOS/qubes-issues#5099
* origin/pr/118:
qvm-start-daemon: adjust pacat pid file path
qvm-start-daemon: check if layout is parsed
qvm-start-daemon: allow multiple options in keyboard layout
qvm-start-daemon: improve parsing args for setting keyboard layout
qvm-start-daemon: set keyboard-layout only for the first set layout
gui: set keyboard layout when starting daemon
daemon: start it for dom0 unconditionnaly
qvm-start-daemon: ensure separate task between GUI/AUDIO
qvm-start-daemon: allow starting only if service enabled
Fix and improvements from Marek's comments
Change qvm-start-gui to qvm-start-daemon for handling audio too
Support for AudioVM
* origin/pr/136:
Added tests for cloning VMs with persistent devices
Remove Python 2 workarounds
Cloning a vm now clones persistent PCI device assignments
Rules will be reset to a single 'accept' line, which is something
that the GUI tools like. It's an easy way to get out of CLI firewall
modifications if someone wants to go back to using GUI for them.
fixesQubesOS/qubes-issues#4710
The code assumes that QubesVMInUseError is thrown for a failed
dependency check, but there are also other reasons. We need to
handle the case when dependency list is empty, and also display
the original message.
See also QubesOS/qubes-core-admin#308 for which this was needed.
* origin/pr/124:
Add missing docstrings
Add a test for vm.get_mem()
Update tests to mock admin.vm.CurrentState
qvm-ls: display memory as integer number
Use admin.vm.CurrentState method for power state and memory
FixesQubesOS/qubes-issues#5166
qvm-shutdown with the --wait option checks if the machine
state is 'Halted', but a disposable VM is usually deleted by
the time of the final check, resulting in a non-zero exit code.
This change handles properly disposable VMs, and makes sure
we always output an error message when finishing with a non-zero
exit code.
FixesQubesOS/qubes-issues#5245.
Depends on the method introduced in QubesOS/qubes-core-admin#307.
Adding the get_mem() method using the same CurrentState admin
method also fixes the bug with qvm-ls not showing memory, see
QubesOS/qubes-issues#5166.
Due to strangeness of KeyError (it overrrides str method) in some
cases exceptions received superflous quotation marks when inheriting
from it.
fixesQubesOS/qubes-issues#5106
