Commit Graph

880 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
7fb90e0233
tests: extend run_service mockup for pre-recorded output
And also handle input written if just stdin.flush() is called but not
stdin.close().
2020-08-05 04:37:45 +02:00
Marek Marczykowski-Górecki
889e606d7c
rpm: add BR: python3-lxml and python3-xcffib
Needed for sphinx documentation build.
2020-08-05 04:37:44 +02:00
Marek Marczykowski-Górecki
7d6cb655f8
backup/restore: add option for unattended restore and extracting log
Allow running unattended, with qvm-backup-restore --passphrase-file.
This require few modifications:
 - copy the passphrase file into the DisposableVM (that VM knows the
         passphrase anyway, so there is no extra data leak)
 - close the terminal when operation finishes

Closing the terminal would eliminate almost all the feedback (operation
log, errors, warnings etc), so write it into a file in DisposableVM and
later extract it and show on the stdout. Similar to qvm-run, color it
red as a content coming from a VM.

QubesOS/qubes-issues#5310
2020-08-05 04:37:44 +02:00
Marek Marczykowski-Górecki
f16f14b020
utils: fix encoding '+' for qubes.VMExec
'+' in an argument needs to be encoded too, otherwise it is interpreted
as arguments separator.
2020-08-04 22:17:04 +02:00
Marek Marczykowski-Górecki
e9120e3196
tools: remove obsolete _want_app argument
It was copied from core-admin but it doesn't make sense here (there is
no loading of qubes.xml).
2020-08-04 04:06:53 +02:00
Marek Marczykowski-Górecki
cc71dd5876
Add "paranoid restore" mode
Having Admin API, it is possible to do this properly now:
 - create DisposableVM
 - assign it proper permissions to create VMs and control those created
   VMs
 - run restore process inside
 - cleanup DisposableVM afterwards

Since the RestoreInDisposableVM class contains de facto reverse parser
for qvm-backup-restore command line, add a test that will spot when it
gets out of sync.

This feature depends on modifications in various other components,
including:
 - linux-utils and core-agent-linux for update qfile-unpacker
 - core-admin for qrexec policy modification

QubesOS/qubes-issues#5310
2020-08-04 04:06:53 +02:00
Marek Marczykowski-Górecki
2089e9e730
rpm/deb: add dependency on scrypt
Needed to extract backup archive

QubesOS/qubes-issues#5310
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
81559d139c
utils: add simple locking primitive
Standard python locking modules do not provide detection if lock-holding
process is still alive. Add a simple wrapper around fcntl.lockf that do
just that.
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
6338b936e2
backup/restore: use qfile-unpacker in a VM, request disk space monitoring
Make it possible to use qvm-backup-restore in a VM. This commit is about
accessing backup itself, when stored in another VM. This involve using
qfile-unpacker instead of qfile-dom0-unpacker and also requesting disk
space monitoring, as a VM probably won't have enough space to hold the
whole backup at once.

QubesOS/qubes-issues#4791
QubesOS/qubes-issues#5310
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
db1d4b5d48
backup/restore: option for alternative qrexec service
Allow setting alternative qrexec service to retrieve backup content. The
service API is slightly different than the default one: it will get only
list of files/directories to extract on its stdin, but not backup
location. The latter could be provided as a service argument, or using
other out-of-band mechanism.
This will be useful for paranoid backup restore mode, to take away
control over location/command from sandboxed qvm-backup-restore process.

QubesOS/qubes-issues#5310
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
114f6fb250
backup/restore: improve error message about restoring tags
Before reporting a tag as not restored, verify if it really wasn't
restored. Generally created-by-* tags cannot be created manually. But
when restoring a backup in dom0, created-by-dom0 tag is added, which in
many cases will match what want to be restored.

Adjust tests to check this too.
2020-08-04 04:06:29 +02:00
Marek Marczykowski-Górecki
2b6b4e7954
backup/restore: distinguish dom0 by name
Getting name of QubesVM object doesn't require Admin API call that could
be not allowed by this VM

QubesOS/qubes-issues#5310
2020-08-04 04:06:29 +02:00
WillyPillow
41cf9f948e
qvm-template: Partially include docstrings and type hints. 2020-08-04 02:51:36 +08:00
WillyPillow
69cd285810
qvm-template: Defer qrexec calls so that they can be omitted if exceptions are raised. 2020-08-04 01:40:59 +08:00
WillyPillow
e482b9eb0f
qvm-template: Use "vm.features.get" instead of explicit membership check. 2020-08-04 01:38:52 +08:00
WillyPillow
582c87644d
qvm-template: Use repo file from qubes-repo-templates. 2020-08-04 01:35:14 +08:00
WillyPillow
bf0635218a
qvm-template: Better args parsing: Use subparsers and complain about unknown args if the operation is not "remove". 2020-08-04 01:34:14 +08:00
Marek Marczykowski-Górecki
b7eca976b5
tests: use asynctest some more
Instead of hoping normal Mock will work (which does not on Python 3.8).
2020-08-03 04:55:46 +02:00
Marek Marczykowski-Górecki
3e1d36fe4a
travis: use sourced config, switch to R4.1 2020-08-03 04:55:46 +02:00
WillyPillow
377e2a77ff
qvm-template: Check that template is managed by qvm-template before accessing relevant features. 2020-08-01 03:21:31 +08:00
WillyPillow
5319e7a41a
qvm-template: Fix typo. 2020-08-01 03:06:04 +08:00
WillyPillow
a9a19428f3
qvm-template: Check that template spec is not "---". 2020-08-01 03:05:21 +08:00
WillyPillow
40e7304f17
qvm-template: Make pylint happy. 2020-08-01 02:56:59 +08:00
WillyPillow
3d0a39523b
qvm-template: Reorder functions. 2020-08-01 02:40:27 +08:00
WillyPillow
233e411c2f
qvm-template: Switch to namedtuples and other slight cleanup. 2020-08-01 02:24:29 +08:00
Marek Marczykowski-Górecki
77e1e080d7
Merge remote-tracking branch 'origin/pr/154'
* origin/pr/154:
  Add qubes-guivm-session utility
  qvm-start-daemon: allow --watch without --all
  qvm-start-daemon: convert to async/await syntax
2020-07-31 00:41:27 +02:00
WillyPillow
3ada7af0eb
qvm-template: {reinstall,{up,down}grade}: Better handling and checks for existing version. 2020-07-31 01:27:40 +08:00
WillyPillow
90e4f65bea
qvm-template*: Add option to specify pool to store created VM. 2020-07-29 20:55:56 +08:00
WillyPillow
ef59a658f4 qvm-template: Make pylint happy by changing "license" to "licence". 2020-07-29 20:55:02 +08:00
WillyPillow
f960ed4726 qvm-template: Add --refresh option and allow DNF cache to be used. 2020-07-29 20:55:02 +08:00
WillyPillow
8aa9ab9e89 qvm-template: Remove downloaded file if the download is interrupted. 2020-07-29 20:55:02 +08:00
WillyPillow
88ee572cac qvm-template: Incorporate additional metadata in qubes.TemplateSearch. 2020-07-29 20:55:02 +08:00
WillyPillow
421dd74dd2 Check number of fields for qubes.TemplateSearch output. 2020-07-29 20:55:02 +08:00
WillyPillow
5e76bdb5f1 Revamp "qvm-template search" and finish TODOs. 2020-07-29 20:55:02 +08:00
WillyPillow
37a72ecebf Print error messages if qubes.TemplateSearch fails. 2020-07-29 20:55:02 +08:00
WillyPillow
e6392ba4ec Add lock-file functionality for qvm-template install. 2020-07-29 20:55:02 +08:00
WillyPillow
c573faa9c0 Initial implementation for "qvm-template search". 2020-07-29 20:55:02 +08:00
WillyPillow
d656554822 Initial implementation for "qvm-template info". 2020-07-29 20:55:02 +08:00
WillyPillow
51324da24d Allow <package-spec>-like arguments for the list operation. 2020-07-29 20:55:02 +08:00
WillyPillow
41323d004f Support for {reinstall,downgrade,upgrade} operations.
Requires QubesOS/qubes-issues#5946 to be resolved.
2020-07-29 20:55:02 +08:00
WillyPillow
faef52e61a Fix pylint warnings. 2020-07-29 20:55:02 +08:00
WillyPillow
8a4b5e683a Add suffix for unverified RPMs. 2020-07-29 20:55:02 +08:00
WillyPillow
addb677506 Check for newlines in qrexec arguments & improve error handling. 2020-07-29 20:55:02 +08:00
WillyPillow
73eb4cd08c Use tqdm for progress bar. 2020-07-29 20:55:02 +08:00
WillyPillow
3d42c988f0 Various cleanup and improvements.
- `qvm-template list`: show template state
- `qvm-template list`: only call qubes.TemplateSearch once
- `qvm-template list`: use `qubesadmin.tools.print_table()` instead of own implementation
- `qvm-template download`: custom progress bar
- Use `run_service` instead of own implementation
- Remove some erroneous/redundant lines
2020-07-29 20:55:02 +08:00
WillyPillow
0e8e8d98de Better way of detecting VM. 2020-07-29 20:55:02 +08:00
WillyPillow
b634c7c785 Initial commit of qvm-template.
Refer to <https://gist.github.com/WillyPillow/61ee5f48b7c5b7cc90c9fd2ec5c1b20d>
for previous revisions.
2020-07-29 20:55:02 +08:00
WillyPillow
bab8e699d7 Change "whitelist" to "menu-items" in qvm-features for clarity. 2020-07-29 20:55:02 +08:00
WillyPillow
e8ba117c26 Allow virt_mode other than pv. 2020-07-29 20:55:02 +08:00
WillyPillow
eda68cce6d Verify values of boolean flags in template config. 2020-07-29 20:55:02 +08:00