Commit Graph

896 Commits

Author SHA1 Message Date
WillyPillow
87c08c9941
qvm-template: Fix missing args for install operations. 2020-08-07 23:40:38 +08:00
WillyPillow
42a741cac5
qvm-template: Remove default 'repo_files' entry if other entries have been specified by the user. 2020-08-07 15:02:53 +08:00
WillyPillow
c523d78d59
qvm-template: Initial implementation of repolist. 2020-08-07 14:48:08 +08:00
Marek Marczykowski-Górecki
b04a14685c
Merge branch 'paranoid-restore'
* paranoid-restore:
  backup/restore: make error reporting work also for StandaloneVM based DispVM
  backup/restore: better error detection for --paranoid-mode
  doc: document 'tag-created-vm-with' feature
  tests: add paranoid backup restore unit tests
  tests: remove extra empty lines
  tests: extend run_service mockup for pre-recorded output
  rpm: add BR: python3-lxml and python3-xcffib
  backup/restore: add option for unattended restore and extracting log
  tools: remove obsolete _want_app argument
  Add "paranoid restore" mode
  rpm/deb: add dependency on scrypt
  utils: add simple locking primitive
  backup/restore: use qfile-unpacker in a VM, request disk space monitoring
  backup/restore: option for alternative qrexec service
  backup/restore: improve error message about restoring tags
  backup/restore: distinguish dom0 by name
2020-08-07 02:55:27 +02:00
Marek Marczykowski-Górecki
c75aeb400a
backup/restore: make error reporting work also for StandaloneVM based DispVM
If management dispvm is based on a StandaloneVM not a AppVM, then there
is just one level of templates, not two.

QubesOS/qubes-issues#4670
2020-08-07 02:12:39 +02:00
WillyPillow
ba7b113206
qvm-template: Replace newlines in machine-readable output. 2020-08-07 02:14:37 +08:00
WillyPillow
ed35802ca2
qvm-template: Tidy up code responsible for output in {info,list} operations. 2020-08-07 02:11:35 +08:00
Marta Marczykowska-Górecka
4a6b5dbae2
qvm-run will unpause paused VMs by defaults
If qvm-run is run with the autostart option (true by default), it will
also unpause paused VMs.

fixes QubesOS/qubes-issues#5967
2020-08-05 20:49:38 +02:00
WillyPillow
d11e74ae99
qvm-template: Include module in documentation. 2020-08-06 02:44:12 +08:00
WillyPillow
336b5c68c1
qvm-template: Initial support for machine-readable listings. 2020-08-06 02:42:05 +08:00
WillyPillow
7b6fa39d1c
qvm-template: More docstrings. 2020-08-06 02:05:57 +08:00
Marta Marczykowska-Górecka
37f0641c26
Added more resilience to missing permissions to utils
vm_dependencies function will now no longer throw an exception
when encountering rejection for property_is_default method.
2020-08-05 17:05:27 +02:00
Marek Marczykowski-Górecki
1660a1cbf6
backup/restore: better error detection for --paranoid-mode
Xterm doesn't preserve exit code of the process running inside. This
means, the whole xterm always exits with 0, even if qvm-backup-restore
failed.
Fix this by printing the exit code at the end to the log and then extract
that last line from the log on the calling side. This way we can also
distinguish whether qvm-backup-restore or xterm failed.
2020-08-05 05:06:54 +02:00
Marek Marczykowski-Górecki
0c73e83627
doc: document 'tag-created-vm-with' feature
The actual implementation lives in core-admin repository, but the
canonical list of know features is here.
2020-08-05 05:06:53 +02:00
Marek Marczykowski-Górecki
852b689984
tests: add paranoid backup restore unit tests 2020-08-05 05:06:53 +02:00
Marek Marczykowski-Górecki
bf7b6225e3
tests: remove extra empty lines 2020-08-05 04:37:45 +02:00
Marek Marczykowski-Górecki
7fb90e0233
tests: extend run_service mockup for pre-recorded output
And also handle input written if just stdin.flush() is called but not
stdin.close().
2020-08-05 04:37:45 +02:00
Marek Marczykowski-Górecki
889e606d7c
rpm: add BR: python3-lxml and python3-xcffib
Needed for sphinx documentation build.
2020-08-05 04:37:44 +02:00
Marek Marczykowski-Górecki
7d6cb655f8
backup/restore: add option for unattended restore and extracting log
Allow running unattended, with qvm-backup-restore --passphrase-file.
This require few modifications:
 - copy the passphrase file into the DisposableVM (that VM knows the
         passphrase anyway, so there is no extra data leak)
 - close the terminal when operation finishes

Closing the terminal would eliminate almost all the feedback (operation
log, errors, warnings etc), so write it into a file in DisposableVM and
later extract it and show on the stdout. Similar to qvm-run, color it
red as a content coming from a VM.

QubesOS/qubes-issues#5310
2020-08-05 04:37:44 +02:00
Marek Marczykowski-Górecki
f16f14b020
utils: fix encoding '+' for qubes.VMExec
'+' in an argument needs to be encoded too, otherwise it is interpreted
as arguments separator.
2020-08-04 22:17:04 +02:00
Marek Marczykowski-Górecki
e9120e3196
tools: remove obsolete _want_app argument
It was copied from core-admin but it doesn't make sense here (there is
no loading of qubes.xml).
2020-08-04 04:06:53 +02:00
Marek Marczykowski-Górecki
cc71dd5876
Add "paranoid restore" mode
Having Admin API, it is possible to do this properly now:
 - create DisposableVM
 - assign it proper permissions to create VMs and control those created
   VMs
 - run restore process inside
 - cleanup DisposableVM afterwards

Since the RestoreInDisposableVM class contains de facto reverse parser
for qvm-backup-restore command line, add a test that will spot when it
gets out of sync.

This feature depends on modifications in various other components,
including:
 - linux-utils and core-agent-linux for update qfile-unpacker
 - core-admin for qrexec policy modification

QubesOS/qubes-issues#5310
2020-08-04 04:06:53 +02:00
Marek Marczykowski-Górecki
2089e9e730
rpm/deb: add dependency on scrypt
Needed to extract backup archive

QubesOS/qubes-issues#5310
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
81559d139c
utils: add simple locking primitive
Standard python locking modules do not provide detection if lock-holding
process is still alive. Add a simple wrapper around fcntl.lockf that do
just that.
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
6338b936e2
backup/restore: use qfile-unpacker in a VM, request disk space monitoring
Make it possible to use qvm-backup-restore in a VM. This commit is about
accessing backup itself, when stored in another VM. This involve using
qfile-unpacker instead of qfile-dom0-unpacker and also requesting disk
space monitoring, as a VM probably won't have enough space to hold the
whole backup at once.

QubesOS/qubes-issues#4791
QubesOS/qubes-issues#5310
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
db1d4b5d48
backup/restore: option for alternative qrexec service
Allow setting alternative qrexec service to retrieve backup content. The
service API is slightly different than the default one: it will get only
list of files/directories to extract on its stdin, but not backup
location. The latter could be provided as a service argument, or using
other out-of-band mechanism.
This will be useful for paranoid backup restore mode, to take away
control over location/command from sandboxed qvm-backup-restore process.

QubesOS/qubes-issues#5310
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
114f6fb250
backup/restore: improve error message about restoring tags
Before reporting a tag as not restored, verify if it really wasn't
restored. Generally created-by-* tags cannot be created manually. But
when restoring a backup in dom0, created-by-dom0 tag is added, which in
many cases will match what want to be restored.

Adjust tests to check this too.
2020-08-04 04:06:29 +02:00
Marek Marczykowski-Górecki
2b6b4e7954
backup/restore: distinguish dom0 by name
Getting name of QubesVM object doesn't require Admin API call that could
be not allowed by this VM

QubesOS/qubes-issues#5310
2020-08-04 04:06:29 +02:00
WillyPillow
41cf9f948e
qvm-template: Partially include docstrings and type hints. 2020-08-04 02:51:36 +08:00
WillyPillow
69cd285810
qvm-template: Defer qrexec calls so that they can be omitted if exceptions are raised. 2020-08-04 01:40:59 +08:00
WillyPillow
e482b9eb0f
qvm-template: Use "vm.features.get" instead of explicit membership check. 2020-08-04 01:38:52 +08:00
WillyPillow
582c87644d
qvm-template: Use repo file from qubes-repo-templates. 2020-08-04 01:35:14 +08:00
WillyPillow
bf0635218a
qvm-template: Better args parsing: Use subparsers and complain about unknown args if the operation is not "remove". 2020-08-04 01:34:14 +08:00
Marek Marczykowski-Górecki
b7eca976b5
tests: use asynctest some more
Instead of hoping normal Mock will work (which does not on Python 3.8).
2020-08-03 04:55:46 +02:00
Marek Marczykowski-Górecki
3e1d36fe4a
travis: use sourced config, switch to R4.1 2020-08-03 04:55:46 +02:00
WillyPillow
377e2a77ff
qvm-template: Check that template is managed by qvm-template before accessing relevant features. 2020-08-01 03:21:31 +08:00
WillyPillow
5319e7a41a
qvm-template: Fix typo. 2020-08-01 03:06:04 +08:00
WillyPillow
a9a19428f3
qvm-template: Check that template spec is not "---". 2020-08-01 03:05:21 +08:00
WillyPillow
40e7304f17
qvm-template: Make pylint happy. 2020-08-01 02:56:59 +08:00
WillyPillow
3d0a39523b
qvm-template: Reorder functions. 2020-08-01 02:40:27 +08:00
WillyPillow
233e411c2f
qvm-template: Switch to namedtuples and other slight cleanup. 2020-08-01 02:24:29 +08:00
Marek Marczykowski-Górecki
77e1e080d7
Merge remote-tracking branch 'origin/pr/154'
* origin/pr/154:
  Add qubes-guivm-session utility
  qvm-start-daemon: allow --watch without --all
  qvm-start-daemon: convert to async/await syntax
2020-07-31 00:41:27 +02:00
WillyPillow
3ada7af0eb
qvm-template: {reinstall,{up,down}grade}: Better handling and checks for existing version. 2020-07-31 01:27:40 +08:00
WillyPillow
90e4f65bea
qvm-template*: Add option to specify pool to store created VM. 2020-07-29 20:55:56 +08:00
WillyPillow
ef59a658f4 qvm-template: Make pylint happy by changing "license" to "licence". 2020-07-29 20:55:02 +08:00
WillyPillow
f960ed4726 qvm-template: Add --refresh option and allow DNF cache to be used. 2020-07-29 20:55:02 +08:00
WillyPillow
8aa9ab9e89 qvm-template: Remove downloaded file if the download is interrupted. 2020-07-29 20:55:02 +08:00
WillyPillow
88ee572cac qvm-template: Incorporate additional metadata in qubes.TemplateSearch. 2020-07-29 20:55:02 +08:00
WillyPillow
421dd74dd2 Check number of fields for qubes.TemplateSearch output. 2020-07-29 20:55:02 +08:00
WillyPillow
5e76bdb5f1 Revamp "qvm-template search" and finish TODOs. 2020-07-29 20:55:02 +08:00