Commit Graph

492 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
b1453953f9
Merge remote-tracking branch 'origin/pr/139'
* origin/pr/139:
  Added a safeguard for invalid firewall rules
2020-05-27 04:18:20 +02:00
Marek Marczykowski-Górecki
aea41511de
events: implicitly enable caching when creating EventsDispatcher
Application that runs EventsDispatcher can safely use also cache , which
greatly improve performance. This is because cache then is properly
updated/invalidated when needed.
Instead of modifying each application to explicitly enable cache based
on this simple rule, make it implicit when EventsDispatcher is created.

Do not enable caching when EventsDispatcher is created only temporarily
in wait_for_domain_shutdown.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
c081ed8c82
Enable caching in qvm-ls and qvm-prefs
Both tools issue a large number of Admin API calls and greatly benefit
from a cache filled with a single per-vm Admin API call
(admin.vm.property.GetAll). In case of qvm-ls, this also saves multiple
admin.vm.CurrentState calls (power state is given in the admin.vm.List
response too).

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
79c7392424
Cache power state when caching is enabled
Power state changes are signaled with events too, so it is possible to
cache it and update/invalidate cache with events.
Additionally, admin.vm.List returns a power state, so the cache can be
populated early. This in particular greatly improves qvm-ls performance -
eliminate admin.vm.CurrentState call at all.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
bfe1a3d541
Adjust for changed qubesd socket protocol
The socket protocol is adjusted to match qrexec socket service protocol.

QubesOS/qubes-issues#3293
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
1d1289619c
Implement a wrapper for dom0 -> dom0 qrexec calls
Qrexec itself does not support  loopback calls. Since a call into dom0
is a common case for GUIVM (which may be dom0) to do, add a convenient
handling inside vm.run_service() function.

Fixes QubesOS/qubes-issues#5140
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
02cfab8257
Try to use new property.GetAll method to pre-fill the cache
When caching is enabled, reduce number of calls by getting all the
properties at once. If the call is not available (for example because of
the policy), fallback to getting individual values.

QubesOS/qubes-issues#5415
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
218d43a2e0
Add simple properties caching
Reduce Admin API calls by caching returned values. The cache is not
enabled by default, because it could result in stale values being
returned. It can be enabled by setting 'cache_enabled' to True on
Qubes() object. This is safe in two cases:
 - the application don't care about changed values - like a short-lived
   process that retrieve values once (for example qvm-ls)
 - the application listen for events and invalidate cache when property
   is changed

For the second case, invalidating the cache on appropriate event
(property-set:*, property-reset:*) is done before calling other event
handlers. This is because the event may try to access the property value
(not necessary from the event arguments), so we need to be sure it will
see the new value.

Fixes QubesOS/qubes-issues#5415
2020-05-22 18:59:44 +02:00
Marek Marczykowski-Górecki
5315bbf8f9
Do not check for object existence when got it with another Admin API call
When qubesd returns an name of VM or other object, as part of another
call (reading a property, listing devices etc), it's safe to assume that
object exists. Do not try to list it, which could be prevented by qrexec
policy. This means a VM object would be returned (for example in
vm.netvm property), which potentially could not be listed through
app.domains collection. This may lead to some corner cases, but
generally should ease handling of restricted policy.

This does not affect practical information the management VM have access
too, as those names are already returned. It's just client side python
wrapper that didn't allowed to access them.

QubesOS/qubes-issues#5099
2020-05-18 04:37:48 +02:00
Marta Marczykowska-Górecka
d2f4a4533a
Added a safeguard for invalid firewall rules
Firewall rule cannot be missing value in declaration
(e.g. 'dsthost=' is not a valid rule).

fixes QubesOS/qubes-issues#5772
2020-05-16 12:18:13 +02:00
Marek Marczykowski-Górecki
83b1fc6c58
tools/qvm-volume import: do not retrieve old size anymore
Since admin.vm.volume.ImportWithSize method is used now, there is no
need to retrieve old size anymore (to decide whether to resize or not).
2020-05-10 03:57:28 +02:00
Marek Marczykowski-Górecki
65dcee424a
Make pylint happy
New pylint reports (duplicated?) warning signature-differs in addition
to arguments-differ. Ignore this one too.
2020-05-10 03:21:40 +02:00
Marek Marczykowski-Górecki
4971faa462
Merge remote-tracking branch 'origin/pr/118'
* origin/pr/118:
  qvm-start-daemon: adjust pacat pid file path
  qvm-start-daemon: check if layout is parsed
  qvm-start-daemon: allow multiple options in keyboard layout
  qvm-start-daemon: improve parsing args for setting keyboard layout
  qvm-start-daemon: set keyboard-layout only for the first set layout
  gui: set keyboard layout when starting daemon
  daemon: start it for dom0 unconditionnaly
  qvm-start-daemon: ensure separate task between GUI/AUDIO
  qvm-start-daemon: allow starting only if service enabled
  Fix and improvements from Marek's comments
  Change qvm-start-gui to qvm-start-daemon for handling audio too
  Support for AudioVM
2020-04-09 05:24:26 +02:00
Marek Marczykowski-Górecki
c80314523f
Merge remote-tracking branch 'origin/pr/136'
* origin/pr/136:
  Added tests for cloning VMs with persistent devices
  Remove Python 2 workarounds
  Cloning a vm now clones persistent PCI device assignments
2020-04-09 05:24:13 +02:00
Rusty Bird
f9ee7f5816
collections.Callable -> collections.abc.Callable
"Deprecated since version 3.3, will be removed in version 3.10"
- https://docs.python.org/3/library/collections.html
2020-04-07 21:30:54 +00:00
Frédéric Pierret (fepitre)
e87559b318
qvm-start-daemon: adjust pacat pid file path 2020-04-07 23:12:09 +02:00
Marta Marczykowska-Górecka
f08d41eec5
Added tests for cloning VMs with persistent devices 2020-03-30 16:15:49 +02:00
Marta Marczykowska-Górecka
8e7db06918
Remove Python 2 workarounds
Python 2 support was discontinued and workaround
for Python2 mock is no longer needed.
2020-03-30 15:48:53 +02:00
Marta Marczykowska-Górecka
c0a8c6528a
Cloning a vm now clones persistent PCI device assignments
fixes QubesOS/qubes-issues#4992
2020-03-30 15:48:53 +02:00
Frédéric Pierret (fepitre)
e39f280405
qvm-start-daemon: check if layout is parsed 2020-03-28 17:33:04 +01:00
Frédéric Pierret (fepitre)
2e72f75611
qvm-start-daemon: allow multiple options in keyboard layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
023d94a0b3
qvm-start-daemon: improve parsing args for setting keyboard layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
c13097d458
qvm-start-daemon: set keyboard-layout only for the first set layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
c58e5b8eb8
gui: set keyboard layout when starting daemon 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
526c862b5d
daemon: start it for dom0 unconditionnaly 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
49957971ff
qvm-start-daemon: ensure separate task between GUI/AUDIO 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
05e479cf92
qvm-start-daemon: allow starting only if service enabled 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
f7fa577c47
Fix and improvements from Marek's comments 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
0e049e682a
Change qvm-start-gui to qvm-start-daemon for handling audio too 2020-03-28 17:33:02 +01:00
Frédéric Pierret (fepitre)
6d9e5bbf07
Support for AudioVM 2020-03-28 17:33:02 +01:00
Marek Marczykowski-Górecki
c43407d28d
Merge remote-tracking branch 'origin/pr/135'
* origin/pr/135:
  Added an option to qvm-firewall to reset all rules
2020-03-10 03:11:26 +01:00
Marek Marczykowski-Górecki
916db9db96
tests/firewall: update future time to be really in the future
Mar 4 2020 is no longer future, move it much more into the future.
2020-03-08 17:16:34 +01:00
Marta Marczykowska-Górecka
97ab1d7adf
Added an option to qvm-firewall to reset all rules
Rules will be reset to a single 'accept' line, which is something
that the GUI tools like. It's an easy way to get out of CLI firewall
modifications if someone wants to go back to using GUI for them.

fixes QubesOS/qubes-issues#4710
2020-02-24 14:58:53 +01:00
Marek Marczykowski-Górecki
162f41caf4
Remove no longer relevant TODO comment
vm.shutdown(force=True) is now implemented.
2020-02-07 04:35:38 +01:00
Marek Marczykowski-Górecki
0120e32340
Merge remote-tracking branch 'origin/pr/130'
* origin/pr/130:
  Added explicit 'force' option to qvm-shutdown
  Added force shutdown option to vm.shutdown
2020-02-07 04:35:22 +01:00
Marek Marczykowski-Górecki
9b70e54719
Merge remote-tracking branch 'origin/pr/134'
* origin/pr/134:
  qvm-kill: make error message more clear
  qvm-kill: ignore already off domains
2020-01-31 15:01:26 +01:00
Pawel Marczewski
bcb91594b2
qvm-kill: make error message more clear
See QubesOS/qubes-issues#5607.
2020-01-31 11:26:57 +01:00
Pawel Marczewski
b8d1884cb1
qvm-kill: ignore already off domains
See QubesOS/qubes-issues#5607.
2020-01-31 11:26:54 +01:00
Marta Marczykowska-Górecka
5c94c72ecd
Added explicit 'force' option to qvm-shutdown 2020-01-29 19:51:04 +01:00
Marta Marczykowska-Górecka
2be77f58b3
Added force shutdown option to vm.shutdown
Furthermore makes qvm-shutdown --all use that option to force
shutdown and avoid unnecessary errors.

requires https://github.com/QubesOS/qubes-core-admin/pull/312
fixes QubesOS/qubes-issues#5591
fixes QubesOS/qubes-issues#4572
2020-01-29 19:35:29 +01:00
Pawel Marczewski
2597ed0795
Use qubes.VMExec in run_with_args() 2020-01-27 10:24:22 +01:00
Pawel Marczewski
ff9b81cc3e
qvm-run: use qubes.VMExec, if available
See QubesOS/qubes-issues#4850.
2020-01-24 18:40:11 +01:00
Marek Marczykowski-Górecki
47188a38e5
Merge remote-tracking branch 'origin/pr/128'
* origin/pr/128:
  Import data using ImportWithSize call, not manual resizing

Fixes QubesOS/qubes-issues#5239
2020-01-24 01:49:04 +01:00
Pawel Marczewski
9cf05e5180
Import data using ImportWithSize call, not manual resizing
See the API call in QubesOS/qubes-core-admin#309.
2020-01-23 10:29:50 +01:00
Pawel Marczewski
bf78aebf7c
qvm-remove: improve error message
The code assumes that QubesVMInUseError is thrown for a failed
dependency check, but there are also other reasons. We need to
handle the case when dependency list is empty, and also display
the original message.

See also QubesOS/qubes-core-admin#308 for which this was needed.
2020-01-21 15:54:18 +01:00
Marek Marczykowski-Górecki
204c33afd1
Merge remote-tracking branch 'origin/pr/127'
* origin/pr/127:
  qvm-shutdown: report errors, don't crash on DispVMs
2020-01-17 05:06:18 +01:00
Marek Marczykowski-Górecki
31db7ecbc0
Merge remote-tracking branch 'origin/pr/124'
* origin/pr/124:
  Add missing docstrings
  Add a test for vm.get_mem()
  Update tests to mock admin.vm.CurrentState
  qvm-ls: display memory as integer number
  Use admin.vm.CurrentState method for power state and memory

Fixes QubesOS/qubes-issues#5166
2020-01-17 04:54:33 +01:00
Pawel Marczewski
405464a67e
qvm-shutdown: report errors, don't crash on DispVMs
qvm-shutdown with the --wait option checks if the machine
state is 'Halted', but a disposable VM is usually deleted by
the time of the final check, resulting in a non-zero exit code.

This change handles properly disposable VMs, and makes sure
we always output an error message when finishing with a non-zero
exit code.

Fixes QubesOS/qubes-issues#5245.
2020-01-16 12:38:10 +01:00
Pawel Marczewski
88f177f564
Add missing docstrings 2020-01-16 09:52:06 +01:00
Pawel Marczewski
023f2f593c
Add a test for vm.get_mem() 2020-01-15 17:32:25 +01:00