Commit Graph

608 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
2b6b4e7954
backup/restore: distinguish dom0 by name
Getting name of QubesVM object doesn't require Admin API call that could
be not allowed by this VM

QubesOS/qubes-issues#5310
2020-08-04 04:06:29 +02:00
WillyPillow
41cf9f948e
qvm-template: Partially include docstrings and type hints. 2020-08-04 02:51:36 +08:00
WillyPillow
69cd285810
qvm-template: Defer qrexec calls so that they can be omitted if exceptions are raised. 2020-08-04 01:40:59 +08:00
WillyPillow
e482b9eb0f
qvm-template: Use "vm.features.get" instead of explicit membership check. 2020-08-04 01:38:52 +08:00
WillyPillow
582c87644d
qvm-template: Use repo file from qubes-repo-templates. 2020-08-04 01:35:14 +08:00
WillyPillow
bf0635218a
qvm-template: Better args parsing: Use subparsers and complain about unknown args if the operation is not "remove". 2020-08-04 01:34:14 +08:00
Marek Marczykowski-Górecki
b7eca976b5
tests: use asynctest some more
Instead of hoping normal Mock will work (which does not on Python 3.8).
2020-08-03 04:55:46 +02:00
WillyPillow
377e2a77ff
qvm-template: Check that template is managed by qvm-template before accessing relevant features. 2020-08-01 03:21:31 +08:00
WillyPillow
5319e7a41a
qvm-template: Fix typo. 2020-08-01 03:06:04 +08:00
WillyPillow
a9a19428f3
qvm-template: Check that template spec is not "---". 2020-08-01 03:05:21 +08:00
WillyPillow
40e7304f17
qvm-template: Make pylint happy. 2020-08-01 02:56:59 +08:00
WillyPillow
3d0a39523b
qvm-template: Reorder functions. 2020-08-01 02:40:27 +08:00
WillyPillow
233e411c2f
qvm-template: Switch to namedtuples and other slight cleanup. 2020-08-01 02:24:29 +08:00
Marek Marczykowski-Górecki
77e1e080d7
Merge remote-tracking branch 'origin/pr/154'
* origin/pr/154:
  Add qubes-guivm-session utility
  qvm-start-daemon: allow --watch without --all
  qvm-start-daemon: convert to async/await syntax
2020-07-31 00:41:27 +02:00
WillyPillow
3ada7af0eb
qvm-template: {reinstall,{up,down}grade}: Better handling and checks for existing version. 2020-07-31 01:27:40 +08:00
WillyPillow
90e4f65bea
qvm-template*: Add option to specify pool to store created VM. 2020-07-29 20:55:56 +08:00
WillyPillow
ef59a658f4 qvm-template: Make pylint happy by changing "license" to "licence". 2020-07-29 20:55:02 +08:00
WillyPillow
f960ed4726 qvm-template: Add --refresh option and allow DNF cache to be used. 2020-07-29 20:55:02 +08:00
WillyPillow
8aa9ab9e89 qvm-template: Remove downloaded file if the download is interrupted. 2020-07-29 20:55:02 +08:00
WillyPillow
88ee572cac qvm-template: Incorporate additional metadata in qubes.TemplateSearch. 2020-07-29 20:55:02 +08:00
WillyPillow
421dd74dd2 Check number of fields for qubes.TemplateSearch output. 2020-07-29 20:55:02 +08:00
WillyPillow
5e76bdb5f1 Revamp "qvm-template search" and finish TODOs. 2020-07-29 20:55:02 +08:00
WillyPillow
37a72ecebf Print error messages if qubes.TemplateSearch fails. 2020-07-29 20:55:02 +08:00
WillyPillow
e6392ba4ec Add lock-file functionality for qvm-template install. 2020-07-29 20:55:02 +08:00
WillyPillow
c573faa9c0 Initial implementation for "qvm-template search". 2020-07-29 20:55:02 +08:00
WillyPillow
d656554822 Initial implementation for "qvm-template info". 2020-07-29 20:55:02 +08:00
WillyPillow
51324da24d Allow <package-spec>-like arguments for the list operation. 2020-07-29 20:55:02 +08:00
WillyPillow
41323d004f Support for {reinstall,downgrade,upgrade} operations.
Requires QubesOS/qubes-issues#5946 to be resolved.
2020-07-29 20:55:02 +08:00
WillyPillow
faef52e61a Fix pylint warnings. 2020-07-29 20:55:02 +08:00
WillyPillow
8a4b5e683a Add suffix for unverified RPMs. 2020-07-29 20:55:02 +08:00
WillyPillow
addb677506 Check for newlines in qrexec arguments & improve error handling. 2020-07-29 20:55:02 +08:00
WillyPillow
73eb4cd08c Use tqdm for progress bar. 2020-07-29 20:55:02 +08:00
WillyPillow
3d42c988f0 Various cleanup and improvements.
- `qvm-template list`: show template state
- `qvm-template list`: only call qubes.TemplateSearch once
- `qvm-template list`: use `qubesadmin.tools.print_table()` instead of own implementation
- `qvm-template download`: custom progress bar
- Use `run_service` instead of own implementation
- Remove some erroneous/redundant lines
2020-07-29 20:55:02 +08:00
WillyPillow
0e8e8d98de Better way of detecting VM. 2020-07-29 20:55:02 +08:00
WillyPillow
b634c7c785 Initial commit of qvm-template.
Refer to <https://gist.github.com/WillyPillow/61ee5f48b7c5b7cc90c9fd2ec5c1b20d>
for previous revisions.
2020-07-29 20:55:02 +08:00
WillyPillow
bab8e699d7 Change "whitelist" to "menu-items" in qvm-features for clarity. 2020-07-29 20:55:02 +08:00
WillyPillow
e8ba117c26 Allow virt_mode other than pv. 2020-07-29 20:55:02 +08:00
WillyPillow
eda68cce6d Verify values of boolean flags in template config. 2020-07-29 20:55:02 +08:00
WillyPillow
6c7360f25c Separate whitelist entries with spaces instead of newlines. 2020-07-29 20:55:02 +08:00
WillyPillow
9d9ee6a4b7 Initial support for qvm-template. 2020-07-29 20:55:02 +08:00
Paweł Marczewski
624e4e32fb
Add qubes-guivm-session utility
To be used in an xsession file (/usr/share/xsessions).
2020-07-29 12:11:03 +02:00
Paweł Marczewski
c6be7ca5cc
qvm-start-daemon: allow --watch without --all
Allow specifying VM names to look for.
2020-07-29 11:35:32 +02:00
Paweł Marczewski
cb7f191bd2
qvm-start-daemon: convert to async/await syntax 2020-07-29 11:27:19 +02:00
Marek Marczykowski-Górecki
a078e1f617
Make Label() object hashable
Since it got custom __eq__ function, __hash__ needs to be implemented
too. Otherwise it can't be used as a key in dict.
2020-07-29 03:50:11 +02:00
Marek Marczykowski-Górecki
af1c1e5234
tests: update for admin.vm.volume.Clear usage in qvm-template-postprocess 2020-07-16 04:07:07 +02:00
Marek Marczykowski-Górecki
4da218c332
Merge remote-tracking branch 'origin/pr/149'
* origin/pr/149:
  Add admin.vm.volume.Clear call (QubesOS/qubes-issues#5946)
2020-07-16 04:05:38 +02:00
Marek Marczykowski-Górecki
b99e45f081
Merge remote-tracking branch 'origin/pr/146'
* origin/pr/146:
  Added dynamic X keyboard event monitoring to qvm_start_daemon.py

Fixes QubesOS/qubes-issues#1396
Fixes QubesOS/qubes-issues#4294
2020-07-15 15:34:07 +02:00
Marta Marczykowska-Górecka
1446a6d7ee
Added dynamic X keyboard event monitoring to qvm_start_daemon.py
Update keyboard_layout property whenever guivm's layout changes, instead of
only at the start.

requires QubesOS/qubes-core-admin#350

references QubesOS/qubes-issues#1396
references QubesOS/qubes-issues#4294
2020-07-15 14:04:25 +02:00
Marek Marczykowski-Górecki
6f335800b0
Wrap too long line 2020-07-15 14:01:29 +02:00
Marek Marczykowski-Górecki
470514d0dc
Merge remote-tracking branch 'origin/pr/144'
* origin/pr/144:
  Clean up the guid-conf file on domain stop
  Generate qubes-guid options based on features
2020-07-15 14:00:31 +02:00
Marek Marczykowski-Górecki
2bdeb7684a
Merge remote-tracking branch 'origin/pr/151'
* origin/pr/151:
  Fixed inconsistent firewall address checking
2020-07-15 13:50:12 +02:00
Marek Marczykowski-Górecki
f9c5b61961
Merge remote-tracking branch 'origin/pr/150'
* origin/pr/150:
  Fix VM validity check for cached VM objects
2020-07-15 13:49:39 +02:00
Marek Marczykowski-Górecki
e0f17f1b4a
Merge remote-tracking branch 'origin/pr/143'
* origin/pr/143:
  qvm-start-daemon: common_guid_args is now a staticmethod
  tests: kde_args are passed with property of launcher
  Fix missing semi-colon and new line
  Handle KDE with specific arg/desktop file
2020-07-15 13:43:15 +02:00
Marta Marczykowska-Górecka
9377addd1e
Fixed inconsistent firewall address checking
core-admin-client side had less strict rules for hostname than core-admin,
leading to unexpected empty qubesd response. Fixed by copying the logic from core-admin.
2020-07-14 20:10:34 +02:00
Marek Marczykowski-Górecki
45a28c29ae
Fix VM validity check for cached VM objects
Qubes().domains.refresh_cache() tries to preserve cached VM objects if
the class matches - this way if an application keeps reference to any,
it will still be the same as freshly obtained from the collection, and
also it will receive cache updates/invalidates based on events.

The check for class change was invalid - on core-admin-client side we
have just one QubesVM class with 'klass' attribute. This leads to VM
objects being disconnected from VMCollection and stale properties cache
there (because they no longer receive events).

Fix the check.

And also add a test if indeed the same object is returned.
2020-07-14 16:10:49 +02:00
WillyPillow
455542ac7f
Add admin.vm.volume.Clear call (QubesOS/qubes-issues#5946) 2020-07-14 01:48:19 +08:00
Marta Marczykowska-Górecka
1a4cdba7f7
Added better __eq__ method to Label class
Labels with the same name should not be distinguishable.

fixes QubesOS/qubes-issues#5944
2020-07-09 19:28:25 +02:00
Paweł Marczewski
7616a8913a
Clean up the guid-conf file on domain stop 2020-06-29 12:20:23 +02:00
Paweł Marczewski
3540f04a42
Generate qubes-guid options based on features
Allow configuring options per VM or globally per GuiVM. The
qvm-start-daemon program reads the options from VM features, and
generates a configuration file for qubes-guid.

Requires QubesOS/qubes-gui-daemon#47 (customizing the configuration
file).
2020-06-29 12:01:09 +02:00
Frédéric Pierret (fepitre)
f89c4cad56
qvm-start-daemon: common_guid_args is now a staticmethod 2020-06-26 12:18:59 +02:00
Frédéric Pierret (fepitre)
6295b0ba2f
tests: kde_args are passed with property of launcher 2020-06-26 12:05:18 +02:00
Frédéric Pierret (fepitre)
92e87b122e
Handle KDE with specific arg/desktop file 2020-06-24 10:51:39 +02:00
Marek Marczykowski-Górecki
ae39c75867
Merge remote-tracking branch 'origin/pr/140'
* origin/pr/140:
  use function to determine pacat domid
  connect to PA in stubdom if audio-model enabled run pacat in low latency mode by default
2020-06-20 05:01:59 +02:00
Marek Marczykowski-Górecki
9d6b7257c4
tools/qvm-start-daemon: reduce required permissions to sys-gui itself
Do not require permission to list sys-gui itself just to get keyboard
layout. Listing itself is not sensitive (sys-gui knows it exists), but
it will make other tools request its properties, which may not be
desirable.
2020-06-16 15:58:57 +02:00
Dmitry Fedorov
d99045f05e
use function to determine pacat domid 2020-05-31 21:11:33 +03:00
Dmitry Fedorov
ec90829695
connect to PA in stubdom if audio-model enabled
run pacat in low latency mode by default
2020-05-29 12:07:38 +03:00
Marek Marczykowski-Górecki
b1453953f9
Merge remote-tracking branch 'origin/pr/139'
* origin/pr/139:
  Added a safeguard for invalid firewall rules
2020-05-27 04:18:20 +02:00
Marek Marczykowski-Górecki
aea41511de
events: implicitly enable caching when creating EventsDispatcher
Application that runs EventsDispatcher can safely use also cache , which
greatly improve performance. This is because cache then is properly
updated/invalidated when needed.
Instead of modifying each application to explicitly enable cache based
on this simple rule, make it implicit when EventsDispatcher is created.

Do not enable caching when EventsDispatcher is created only temporarily
in wait_for_domain_shutdown.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
c081ed8c82
Enable caching in qvm-ls and qvm-prefs
Both tools issue a large number of Admin API calls and greatly benefit
from a cache filled with a single per-vm Admin API call
(admin.vm.property.GetAll). In case of qvm-ls, this also saves multiple
admin.vm.CurrentState calls (power state is given in the admin.vm.List
response too).

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
79c7392424
Cache power state when caching is enabled
Power state changes are signaled with events too, so it is possible to
cache it and update/invalidate cache with events.
Additionally, admin.vm.List returns a power state, so the cache can be
populated early. This in particular greatly improves qvm-ls performance -
eliminate admin.vm.CurrentState call at all.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
bfe1a3d541
Adjust for changed qubesd socket protocol
The socket protocol is adjusted to match qrexec socket service protocol.

QubesOS/qubes-issues#3293
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
1d1289619c
Implement a wrapper for dom0 -> dom0 qrexec calls
Qrexec itself does not support  loopback calls. Since a call into dom0
is a common case for GUIVM (which may be dom0) to do, add a convenient
handling inside vm.run_service() function.

Fixes QubesOS/qubes-issues#5140
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
02cfab8257
Try to use new property.GetAll method to pre-fill the cache
When caching is enabled, reduce number of calls by getting all the
properties at once. If the call is not available (for example because of
the policy), fallback to getting individual values.

QubesOS/qubes-issues#5415
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
218d43a2e0
Add simple properties caching
Reduce Admin API calls by caching returned values. The cache is not
enabled by default, because it could result in stale values being
returned. It can be enabled by setting 'cache_enabled' to True on
Qubes() object. This is safe in two cases:
 - the application don't care about changed values - like a short-lived
   process that retrieve values once (for example qvm-ls)
 - the application listen for events and invalidate cache when property
   is changed

For the second case, invalidating the cache on appropriate event
(property-set:*, property-reset:*) is done before calling other event
handlers. This is because the event may try to access the property value
(not necessary from the event arguments), so we need to be sure it will
see the new value.

Fixes QubesOS/qubes-issues#5415
2020-05-22 18:59:44 +02:00
Marek Marczykowski-Górecki
5315bbf8f9
Do not check for object existence when got it with another Admin API call
When qubesd returns an name of VM or other object, as part of another
call (reading a property, listing devices etc), it's safe to assume that
object exists. Do not try to list it, which could be prevented by qrexec
policy. This means a VM object would be returned (for example in
vm.netvm property), which potentially could not be listed through
app.domains collection. This may lead to some corner cases, but
generally should ease handling of restricted policy.

This does not affect practical information the management VM have access
too, as those names are already returned. It's just client side python
wrapper that didn't allowed to access them.

QubesOS/qubes-issues#5099
2020-05-18 04:37:48 +02:00
Marta Marczykowska-Górecka
d2f4a4533a
Added a safeguard for invalid firewall rules
Firewall rule cannot be missing value in declaration
(e.g. 'dsthost=' is not a valid rule).

fixes QubesOS/qubes-issues#5772
2020-05-16 12:18:13 +02:00
Marek Marczykowski-Górecki
83b1fc6c58
tools/qvm-volume import: do not retrieve old size anymore
Since admin.vm.volume.ImportWithSize method is used now, there is no
need to retrieve old size anymore (to decide whether to resize or not).
2020-05-10 03:57:28 +02:00
Marek Marczykowski-Górecki
65dcee424a
Make pylint happy
New pylint reports (duplicated?) warning signature-differs in addition
to arguments-differ. Ignore this one too.
2020-05-10 03:21:40 +02:00
Marek Marczykowski-Górecki
4971faa462
Merge remote-tracking branch 'origin/pr/118'
* origin/pr/118:
  qvm-start-daemon: adjust pacat pid file path
  qvm-start-daemon: check if layout is parsed
  qvm-start-daemon: allow multiple options in keyboard layout
  qvm-start-daemon: improve parsing args for setting keyboard layout
  qvm-start-daemon: set keyboard-layout only for the first set layout
  gui: set keyboard layout when starting daemon
  daemon: start it for dom0 unconditionnaly
  qvm-start-daemon: ensure separate task between GUI/AUDIO
  qvm-start-daemon: allow starting only if service enabled
  Fix and improvements from Marek's comments
  Change qvm-start-gui to qvm-start-daemon for handling audio too
  Support for AudioVM
2020-04-09 05:24:26 +02:00
Marek Marczykowski-Górecki
c80314523f
Merge remote-tracking branch 'origin/pr/136'
* origin/pr/136:
  Added tests for cloning VMs with persistent devices
  Remove Python 2 workarounds
  Cloning a vm now clones persistent PCI device assignments
2020-04-09 05:24:13 +02:00
Rusty Bird
f9ee7f5816
collections.Callable -> collections.abc.Callable
"Deprecated since version 3.3, will be removed in version 3.10"
- https://docs.python.org/3/library/collections.html
2020-04-07 21:30:54 +00:00
Frédéric Pierret (fepitre)
e87559b318
qvm-start-daemon: adjust pacat pid file path 2020-04-07 23:12:09 +02:00
Marta Marczykowska-Górecka
f08d41eec5
Added tests for cloning VMs with persistent devices 2020-03-30 16:15:49 +02:00
Marta Marczykowska-Górecka
8e7db06918
Remove Python 2 workarounds
Python 2 support was discontinued and workaround
for Python2 mock is no longer needed.
2020-03-30 15:48:53 +02:00
Marta Marczykowska-Górecka
c0a8c6528a
Cloning a vm now clones persistent PCI device assignments
fixes QubesOS/qubes-issues#4992
2020-03-30 15:48:53 +02:00
Frédéric Pierret (fepitre)
e39f280405
qvm-start-daemon: check if layout is parsed 2020-03-28 17:33:04 +01:00
Frédéric Pierret (fepitre)
2e72f75611
qvm-start-daemon: allow multiple options in keyboard layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
023d94a0b3
qvm-start-daemon: improve parsing args for setting keyboard layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
c13097d458
qvm-start-daemon: set keyboard-layout only for the first set layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
c58e5b8eb8
gui: set keyboard layout when starting daemon 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
526c862b5d
daemon: start it for dom0 unconditionnaly 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
49957971ff
qvm-start-daemon: ensure separate task between GUI/AUDIO 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
05e479cf92
qvm-start-daemon: allow starting only if service enabled 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
f7fa577c47
Fix and improvements from Marek's comments 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
0e049e682a
Change qvm-start-gui to qvm-start-daemon for handling audio too 2020-03-28 17:33:02 +01:00
Frédéric Pierret (fepitre)
6d9e5bbf07
Support for AudioVM 2020-03-28 17:33:02 +01:00
Marek Marczykowski-Górecki
c43407d28d
Merge remote-tracking branch 'origin/pr/135'
* origin/pr/135:
  Added an option to qvm-firewall to reset all rules
2020-03-10 03:11:26 +01:00
Marek Marczykowski-Górecki
916db9db96
tests/firewall: update future time to be really in the future
Mar 4 2020 is no longer future, move it much more into the future.
2020-03-08 17:16:34 +01:00
Marta Marczykowska-Górecka
97ab1d7adf
Added an option to qvm-firewall to reset all rules
Rules will be reset to a single 'accept' line, which is something
that the GUI tools like. It's an easy way to get out of CLI firewall
modifications if someone wants to go back to using GUI for them.

fixes QubesOS/qubes-issues#4710
2020-02-24 14:58:53 +01:00
Marek Marczykowski-Górecki
162f41caf4
Remove no longer relevant TODO comment
vm.shutdown(force=True) is now implemented.
2020-02-07 04:35:38 +01:00