Commit Graph

231 Commits

Author SHA1 Message Date
Marta Marczykowska-Górecka
fe583cd88e
Make incorrect label name error more descriptive
Instead of unintuitive Value Error now we have dedicated QubesLabelNotFoundError.
2020-08-13 20:21:46 +02:00
Marek Marczykowski-Górecki
6e91fba942
Merge remote-tracking branch 'origin/pr/157'
* origin/pr/157:
  qvm-run will unpause paused VMs by defaults
2020-08-11 18:42:35 +02:00
Marek Marczykowski-Górecki
b04a14685c
Merge branch 'paranoid-restore'
* paranoid-restore:
  backup/restore: make error reporting work also for StandaloneVM based DispVM
  backup/restore: better error detection for --paranoid-mode
  doc: document 'tag-created-vm-with' feature
  tests: add paranoid backup restore unit tests
  tests: remove extra empty lines
  tests: extend run_service mockup for pre-recorded output
  rpm: add BR: python3-lxml and python3-xcffib
  backup/restore: add option for unattended restore and extracting log
  tools: remove obsolete _want_app argument
  Add "paranoid restore" mode
  rpm/deb: add dependency on scrypt
  utils: add simple locking primitive
  backup/restore: use qfile-unpacker in a VM, request disk space monitoring
  backup/restore: option for alternative qrexec service
  backup/restore: improve error message about restoring tags
  backup/restore: distinguish dom0 by name
2020-08-07 02:55:27 +02:00
Marta Marczykowska-Górecka
4a6b5dbae2
qvm-run will unpause paused VMs by defaults
If qvm-run is run with the autostart option (true by default), it will
also unpause paused VMs.

fixes QubesOS/qubes-issues#5967
2020-08-05 20:49:38 +02:00
Marek Marczykowski-Górecki
1660a1cbf6
backup/restore: better error detection for --paranoid-mode
Xterm doesn't preserve exit code of the process running inside. This
means, the whole xterm always exits with 0, even if qvm-backup-restore
failed.
Fix this by printing the exit code at the end to the log and then extract
that last line from the log on the calling side. This way we can also
distinguish whether qvm-backup-restore or xterm failed.
2020-08-05 05:06:54 +02:00
Marek Marczykowski-Górecki
852b689984
tests: add paranoid backup restore unit tests 2020-08-05 05:06:53 +02:00
Marek Marczykowski-Górecki
bf7b6225e3
tests: remove extra empty lines 2020-08-05 04:37:45 +02:00
Marek Marczykowski-Górecki
7fb90e0233
tests: extend run_service mockup for pre-recorded output
And also handle input written if just stdin.flush() is called but not
stdin.close().
2020-08-05 04:37:45 +02:00
Marek Marczykowski-Górecki
f16f14b020
utils: fix encoding '+' for qubes.VMExec
'+' in an argument needs to be encoded too, otherwise it is interpreted
as arguments separator.
2020-08-04 22:17:04 +02:00
Marek Marczykowski-Górecki
cc71dd5876
Add "paranoid restore" mode
Having Admin API, it is possible to do this properly now:
 - create DisposableVM
 - assign it proper permissions to create VMs and control those created
   VMs
 - run restore process inside
 - cleanup DisposableVM afterwards

Since the RestoreInDisposableVM class contains de facto reverse parser
for qvm-backup-restore command line, add a test that will spot when it
gets out of sync.

This feature depends on modifications in various other components,
including:
 - linux-utils and core-agent-linux for update qfile-unpacker
 - core-admin for qrexec policy modification

QubesOS/qubes-issues#5310
2020-08-04 04:06:53 +02:00
Marek Marczykowski-Górecki
db1d4b5d48
backup/restore: option for alternative qrexec service
Allow setting alternative qrexec service to retrieve backup content. The
service API is slightly different than the default one: it will get only
list of files/directories to extract on its stdin, but not backup
location. The latter could be provided as a service argument, or using
other out-of-band mechanism.
This will be useful for paranoid backup restore mode, to take away
control over location/command from sandboxed qvm-backup-restore process.

QubesOS/qubes-issues#5310
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
114f6fb250
backup/restore: improve error message about restoring tags
Before reporting a tag as not restored, verify if it really wasn't
restored. Generally created-by-* tags cannot be created manually. But
when restoring a backup in dom0, created-by-dom0 tag is added, which in
many cases will match what want to be restored.

Adjust tests to check this too.
2020-08-04 04:06:29 +02:00
Marek Marczykowski-Górecki
b7eca976b5
tests: use asynctest some more
Instead of hoping normal Mock will work (which does not on Python 3.8).
2020-08-03 04:55:46 +02:00
Paweł Marczewski
cb7f191bd2
qvm-start-daemon: convert to async/await syntax 2020-07-29 11:27:19 +02:00
Marek Marczykowski-Górecki
af1c1e5234
tests: update for admin.vm.volume.Clear usage in qvm-template-postprocess 2020-07-16 04:07:07 +02:00
Marek Marczykowski-Górecki
470514d0dc
Merge remote-tracking branch 'origin/pr/144'
* origin/pr/144:
  Clean up the guid-conf file on domain stop
  Generate qubes-guid options based on features
2020-07-15 14:00:31 +02:00
Marek Marczykowski-Górecki
2bdeb7684a
Merge remote-tracking branch 'origin/pr/151'
* origin/pr/151:
  Fixed inconsistent firewall address checking
2020-07-15 13:50:12 +02:00
Marek Marczykowski-Górecki
f9c5b61961
Merge remote-tracking branch 'origin/pr/150'
* origin/pr/150:
  Fix VM validity check for cached VM objects
2020-07-15 13:49:39 +02:00
Marta Marczykowska-Górecka
9377addd1e
Fixed inconsistent firewall address checking
core-admin-client side had less strict rules for hostname than core-admin,
leading to unexpected empty qubesd response. Fixed by copying the logic from core-admin.
2020-07-14 20:10:34 +02:00
Marek Marczykowski-Górecki
45a28c29ae
Fix VM validity check for cached VM objects
Qubes().domains.refresh_cache() tries to preserve cached VM objects if
the class matches - this way if an application keeps reference to any,
it will still be the same as freshly obtained from the collection, and
also it will receive cache updates/invalidates based on events.

The check for class change was invalid - on core-admin-client side we
have just one QubesVM class with 'klass' attribute. This leads to VM
objects being disconnected from VMCollection and stale properties cache
there (because they no longer receive events).

Fix the check.

And also add a test if indeed the same object is returned.
2020-07-14 16:10:49 +02:00
Paweł Marczewski
3540f04a42
Generate qubes-guid options based on features
Allow configuring options per VM or globally per GuiVM. The
qvm-start-daemon program reads the options from VM features, and
generates a configuration file for qubes-guid.

Requires QubesOS/qubes-gui-daemon#47 (customizing the configuration
file).
2020-06-29 12:01:09 +02:00
Frédéric Pierret (fepitre)
6295b0ba2f
tests: kde_args are passed with property of launcher 2020-06-26 12:05:18 +02:00
Marek Marczykowski-Górecki
b1453953f9
Merge remote-tracking branch 'origin/pr/139'
* origin/pr/139:
  Added a safeguard for invalid firewall rules
2020-05-27 04:18:20 +02:00
Marek Marczykowski-Górecki
c081ed8c82
Enable caching in qvm-ls and qvm-prefs
Both tools issue a large number of Admin API calls and greatly benefit
from a cache filled with a single per-vm Admin API call
(admin.vm.property.GetAll). In case of qvm-ls, this also saves multiple
admin.vm.CurrentState calls (power state is given in the admin.vm.List
response too).

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
79c7392424
Cache power state when caching is enabled
Power state changes are signaled with events too, so it is possible to
cache it and update/invalidate cache with events.
Additionally, admin.vm.List returns a power state, so the cache can be
populated early. This in particular greatly improves qvm-ls performance -
eliminate admin.vm.CurrentState call at all.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
bfe1a3d541
Adjust for changed qubesd socket protocol
The socket protocol is adjusted to match qrexec socket service protocol.

QubesOS/qubes-issues#3293
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
02cfab8257
Try to use new property.GetAll method to pre-fill the cache
When caching is enabled, reduce number of calls by getting all the
properties at once. If the call is not available (for example because of
the policy), fallback to getting individual values.

QubesOS/qubes-issues#5415
2020-05-22 19:01:24 +02:00
Marta Marczykowska-Górecka
d2f4a4533a
Added a safeguard for invalid firewall rules
Firewall rule cannot be missing value in declaration
(e.g. 'dsthost=' is not a valid rule).

fixes QubesOS/qubes-issues#5772
2020-05-16 12:18:13 +02:00
Marek Marczykowski-Górecki
83b1fc6c58
tools/qvm-volume import: do not retrieve old size anymore
Since admin.vm.volume.ImportWithSize method is used now, there is no
need to retrieve old size anymore (to decide whether to resize or not).
2020-05-10 03:57:28 +02:00
Marek Marczykowski-Górecki
4971faa462
Merge remote-tracking branch 'origin/pr/118'
* origin/pr/118:
  qvm-start-daemon: adjust pacat pid file path
  qvm-start-daemon: check if layout is parsed
  qvm-start-daemon: allow multiple options in keyboard layout
  qvm-start-daemon: improve parsing args for setting keyboard layout
  qvm-start-daemon: set keyboard-layout only for the first set layout
  gui: set keyboard layout when starting daemon
  daemon: start it for dom0 unconditionnaly
  qvm-start-daemon: ensure separate task between GUI/AUDIO
  qvm-start-daemon: allow starting only if service enabled
  Fix and improvements from Marek's comments
  Change qvm-start-gui to qvm-start-daemon for handling audio too
  Support for AudioVM
2020-04-09 05:24:26 +02:00
Marta Marczykowska-Górecka
f08d41eec5
Added tests for cloning VMs with persistent devices 2020-03-30 16:15:49 +02:00
Marta Marczykowska-Górecka
8e7db06918
Remove Python 2 workarounds
Python 2 support was discontinued and workaround
for Python2 mock is no longer needed.
2020-03-30 15:48:53 +02:00
Frédéric Pierret (fepitre)
0e049e682a
Change qvm-start-gui to qvm-start-daemon for handling audio too 2020-03-28 17:33:02 +01:00
Frédéric Pierret (fepitre)
6d9e5bbf07
Support for AudioVM 2020-03-28 17:33:02 +01:00
Marek Marczykowski-Górecki
c43407d28d
Merge remote-tracking branch 'origin/pr/135'
* origin/pr/135:
  Added an option to qvm-firewall to reset all rules
2020-03-10 03:11:26 +01:00
Marek Marczykowski-Górecki
916db9db96
tests/firewall: update future time to be really in the future
Mar 4 2020 is no longer future, move it much more into the future.
2020-03-08 17:16:34 +01:00
Marta Marczykowska-Górecka
97ab1d7adf
Added an option to qvm-firewall to reset all rules
Rules will be reset to a single 'accept' line, which is something
that the GUI tools like. It's an easy way to get out of CLI firewall
modifications if someone wants to go back to using GUI for them.

fixes QubesOS/qubes-issues#4710
2020-02-24 14:58:53 +01:00
Marek Marczykowski-Górecki
0120e32340
Merge remote-tracking branch 'origin/pr/130'
* origin/pr/130:
  Added explicit 'force' option to qvm-shutdown
  Added force shutdown option to vm.shutdown
2020-02-07 04:35:22 +01:00
Marek Marczykowski-Górecki
9b70e54719
Merge remote-tracking branch 'origin/pr/134'
* origin/pr/134:
  qvm-kill: make error message more clear
  qvm-kill: ignore already off domains
2020-01-31 15:01:26 +01:00
Pawel Marczewski
bcb91594b2
qvm-kill: make error message more clear
See QubesOS/qubes-issues#5607.
2020-01-31 11:26:57 +01:00
Pawel Marczewski
b8d1884cb1
qvm-kill: ignore already off domains
See QubesOS/qubes-issues#5607.
2020-01-31 11:26:54 +01:00
Marta Marczykowska-Górecka
2be77f58b3
Added force shutdown option to vm.shutdown
Furthermore makes qvm-shutdown --all use that option to force
shutdown and avoid unnecessary errors.

requires https://github.com/QubesOS/qubes-core-admin/pull/312
fixes QubesOS/qubes-issues#5591
fixes QubesOS/qubes-issues#4572
2020-01-29 19:35:29 +01:00
Pawel Marczewski
2597ed0795
Use qubes.VMExec in run_with_args() 2020-01-27 10:24:22 +01:00
Pawel Marczewski
ff9b81cc3e
qvm-run: use qubes.VMExec, if available
See QubesOS/qubes-issues#4850.
2020-01-24 18:40:11 +01:00
Marek Marczykowski-Górecki
47188a38e5
Merge remote-tracking branch 'origin/pr/128'
* origin/pr/128:
  Import data using ImportWithSize call, not manual resizing

Fixes QubesOS/qubes-issues#5239
2020-01-24 01:49:04 +01:00
Pawel Marczewski
9cf05e5180
Import data using ImportWithSize call, not manual resizing
See the API call in QubesOS/qubes-core-admin#309.
2020-01-23 10:29:50 +01:00
Pawel Marczewski
bf78aebf7c
qvm-remove: improve error message
The code assumes that QubesVMInUseError is thrown for a failed
dependency check, but there are also other reasons. We need to
handle the case when dependency list is empty, and also display
the original message.

See also QubesOS/qubes-core-admin#308 for which this was needed.
2020-01-21 15:54:18 +01:00
Marek Marczykowski-Górecki
204c33afd1
Merge remote-tracking branch 'origin/pr/127'
* origin/pr/127:
  qvm-shutdown: report errors, don't crash on DispVMs
2020-01-17 05:06:18 +01:00
Marek Marczykowski-Górecki
31db7ecbc0
Merge remote-tracking branch 'origin/pr/124'
* origin/pr/124:
  Add missing docstrings
  Add a test for vm.get_mem()
  Update tests to mock admin.vm.CurrentState
  qvm-ls: display memory as integer number
  Use admin.vm.CurrentState method for power state and memory

Fixes QubesOS/qubes-issues#5166
2020-01-17 04:54:33 +01:00
Pawel Marczewski
405464a67e
qvm-shutdown: report errors, don't crash on DispVMs
qvm-shutdown with the --wait option checks if the machine
state is 'Halted', but a disposable VM is usually deleted by
the time of the final check, resulting in a non-zero exit code.

This change handles properly disposable VMs, and makes sure
we always output an error message when finishing with a non-zero
exit code.

Fixes QubesOS/qubes-issues#5245.
2020-01-16 12:38:10 +01:00