Commit Graph

556 Commits

Author SHA1 Message Date
WillyPillow
73eb4cd08c Use tqdm for progress bar. 2020-07-29 20:55:02 +08:00
WillyPillow
3d42c988f0 Various cleanup and improvements.
- `qvm-template list`: show template state
- `qvm-template list`: only call qubes.TemplateSearch once
- `qvm-template list`: use `qubesadmin.tools.print_table()` instead of own implementation
- `qvm-template download`: custom progress bar
- Use `run_service` instead of own implementation
- Remove some erroneous/redundant lines
2020-07-29 20:55:02 +08:00
WillyPillow
0e8e8d98de Better way of detecting VM. 2020-07-29 20:55:02 +08:00
WillyPillow
b634c7c785 Initial commit of qvm-template.
Refer to <https://gist.github.com/WillyPillow/61ee5f48b7c5b7cc90c9fd2ec5c1b20d>
for previous revisions.
2020-07-29 20:55:02 +08:00
WillyPillow
bab8e699d7 Change "whitelist" to "menu-items" in qvm-features for clarity. 2020-07-29 20:55:02 +08:00
WillyPillow
e8ba117c26 Allow virt_mode other than pv. 2020-07-29 20:55:02 +08:00
WillyPillow
eda68cce6d Verify values of boolean flags in template config. 2020-07-29 20:55:02 +08:00
WillyPillow
6c7360f25c Separate whitelist entries with spaces instead of newlines. 2020-07-29 20:55:02 +08:00
WillyPillow
9d9ee6a4b7 Initial support for qvm-template. 2020-07-29 20:55:02 +08:00
WillyPillow
455542ac7f
Add admin.vm.volume.Clear call (QubesOS/qubes-issues#5946) 2020-07-14 01:48:19 +08:00
Marek Marczykowski-Górecki
ae39c75867
Merge remote-tracking branch 'origin/pr/140'
* origin/pr/140:
  use function to determine pacat domid
  connect to PA in stubdom if audio-model enabled run pacat in low latency mode by default
2020-06-20 05:01:59 +02:00
Marek Marczykowski-Górecki
9d6b7257c4
tools/qvm-start-daemon: reduce required permissions to sys-gui itself
Do not require permission to list sys-gui itself just to get keyboard
layout. Listing itself is not sensitive (sys-gui knows it exists), but
it will make other tools request its properties, which may not be
desirable.
2020-06-16 15:58:57 +02:00
Dmitry Fedorov
d99045f05e
use function to determine pacat domid 2020-05-31 21:11:33 +03:00
Dmitry Fedorov
ec90829695
connect to PA in stubdom if audio-model enabled
run pacat in low latency mode by default
2020-05-29 12:07:38 +03:00
Marek Marczykowski-Górecki
b1453953f9
Merge remote-tracking branch 'origin/pr/139'
* origin/pr/139:
  Added a safeguard for invalid firewall rules
2020-05-27 04:18:20 +02:00
Marek Marczykowski-Górecki
aea41511de
events: implicitly enable caching when creating EventsDispatcher
Application that runs EventsDispatcher can safely use also cache , which
greatly improve performance. This is because cache then is properly
updated/invalidated when needed.
Instead of modifying each application to explicitly enable cache based
on this simple rule, make it implicit when EventsDispatcher is created.

Do not enable caching when EventsDispatcher is created only temporarily
in wait_for_domain_shutdown.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
c081ed8c82
Enable caching in qvm-ls and qvm-prefs
Both tools issue a large number of Admin API calls and greatly benefit
from a cache filled with a single per-vm Admin API call
(admin.vm.property.GetAll). In case of qvm-ls, this also saves multiple
admin.vm.CurrentState calls (power state is given in the admin.vm.List
response too).

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
79c7392424
Cache power state when caching is enabled
Power state changes are signaled with events too, so it is possible to
cache it and update/invalidate cache with events.
Additionally, admin.vm.List returns a power state, so the cache can be
populated early. This in particular greatly improves qvm-ls performance -
eliminate admin.vm.CurrentState call at all.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
bfe1a3d541
Adjust for changed qubesd socket protocol
The socket protocol is adjusted to match qrexec socket service protocol.

QubesOS/qubes-issues#3293
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
1d1289619c
Implement a wrapper for dom0 -> dom0 qrexec calls
Qrexec itself does not support  loopback calls. Since a call into dom0
is a common case for GUIVM (which may be dom0) to do, add a convenient
handling inside vm.run_service() function.

Fixes QubesOS/qubes-issues#5140
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
02cfab8257
Try to use new property.GetAll method to pre-fill the cache
When caching is enabled, reduce number of calls by getting all the
properties at once. If the call is not available (for example because of
the policy), fallback to getting individual values.

QubesOS/qubes-issues#5415
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
218d43a2e0
Add simple properties caching
Reduce Admin API calls by caching returned values. The cache is not
enabled by default, because it could result in stale values being
returned. It can be enabled by setting 'cache_enabled' to True on
Qubes() object. This is safe in two cases:
 - the application don't care about changed values - like a short-lived
   process that retrieve values once (for example qvm-ls)
 - the application listen for events and invalidate cache when property
   is changed

For the second case, invalidating the cache on appropriate event
(property-set:*, property-reset:*) is done before calling other event
handlers. This is because the event may try to access the property value
(not necessary from the event arguments), so we need to be sure it will
see the new value.

Fixes QubesOS/qubes-issues#5415
2020-05-22 18:59:44 +02:00
Marek Marczykowski-Górecki
5315bbf8f9
Do not check for object existence when got it with another Admin API call
When qubesd returns an name of VM or other object, as part of another
call (reading a property, listing devices etc), it's safe to assume that
object exists. Do not try to list it, which could be prevented by qrexec
policy. This means a VM object would be returned (for example in
vm.netvm property), which potentially could not be listed through
app.domains collection. This may lead to some corner cases, but
generally should ease handling of restricted policy.

This does not affect practical information the management VM have access
too, as those names are already returned. It's just client side python
wrapper that didn't allowed to access them.

QubesOS/qubes-issues#5099
2020-05-18 04:37:48 +02:00
Marta Marczykowska-Górecka
d2f4a4533a
Added a safeguard for invalid firewall rules
Firewall rule cannot be missing value in declaration
(e.g. 'dsthost=' is not a valid rule).

fixes QubesOS/qubes-issues#5772
2020-05-16 12:18:13 +02:00
Marek Marczykowski-Górecki
83b1fc6c58
tools/qvm-volume import: do not retrieve old size anymore
Since admin.vm.volume.ImportWithSize method is used now, there is no
need to retrieve old size anymore (to decide whether to resize or not).
2020-05-10 03:57:28 +02:00
Marek Marczykowski-Górecki
65dcee424a
Make pylint happy
New pylint reports (duplicated?) warning signature-differs in addition
to arguments-differ. Ignore this one too.
2020-05-10 03:21:40 +02:00
Marek Marczykowski-Górecki
4971faa462
Merge remote-tracking branch 'origin/pr/118'
* origin/pr/118:
  qvm-start-daemon: adjust pacat pid file path
  qvm-start-daemon: check if layout is parsed
  qvm-start-daemon: allow multiple options in keyboard layout
  qvm-start-daemon: improve parsing args for setting keyboard layout
  qvm-start-daemon: set keyboard-layout only for the first set layout
  gui: set keyboard layout when starting daemon
  daemon: start it for dom0 unconditionnaly
  qvm-start-daemon: ensure separate task between GUI/AUDIO
  qvm-start-daemon: allow starting only if service enabled
  Fix and improvements from Marek's comments
  Change qvm-start-gui to qvm-start-daemon for handling audio too
  Support for AudioVM
2020-04-09 05:24:26 +02:00
Marek Marczykowski-Górecki
c80314523f
Merge remote-tracking branch 'origin/pr/136'
* origin/pr/136:
  Added tests for cloning VMs with persistent devices
  Remove Python 2 workarounds
  Cloning a vm now clones persistent PCI device assignments
2020-04-09 05:24:13 +02:00
Rusty Bird
f9ee7f5816
collections.Callable -> collections.abc.Callable
"Deprecated since version 3.3, will be removed in version 3.10"
- https://docs.python.org/3/library/collections.html
2020-04-07 21:30:54 +00:00
Frédéric Pierret (fepitre)
e87559b318
qvm-start-daemon: adjust pacat pid file path 2020-04-07 23:12:09 +02:00
Marta Marczykowska-Górecka
f08d41eec5
Added tests for cloning VMs with persistent devices 2020-03-30 16:15:49 +02:00
Marta Marczykowska-Górecka
8e7db06918
Remove Python 2 workarounds
Python 2 support was discontinued and workaround
for Python2 mock is no longer needed.
2020-03-30 15:48:53 +02:00
Marta Marczykowska-Górecka
c0a8c6528a
Cloning a vm now clones persistent PCI device assignments
fixes QubesOS/qubes-issues#4992
2020-03-30 15:48:53 +02:00
Frédéric Pierret (fepitre)
e39f280405
qvm-start-daemon: check if layout is parsed 2020-03-28 17:33:04 +01:00
Frédéric Pierret (fepitre)
2e72f75611
qvm-start-daemon: allow multiple options in keyboard layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
023d94a0b3
qvm-start-daemon: improve parsing args for setting keyboard layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
c13097d458
qvm-start-daemon: set keyboard-layout only for the first set layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
c58e5b8eb8
gui: set keyboard layout when starting daemon 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
526c862b5d
daemon: start it for dom0 unconditionnaly 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
49957971ff
qvm-start-daemon: ensure separate task between GUI/AUDIO 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
05e479cf92
qvm-start-daemon: allow starting only if service enabled 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
f7fa577c47
Fix and improvements from Marek's comments 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
0e049e682a
Change qvm-start-gui to qvm-start-daemon for handling audio too 2020-03-28 17:33:02 +01:00
Frédéric Pierret (fepitre)
6d9e5bbf07
Support for AudioVM 2020-03-28 17:33:02 +01:00
Marek Marczykowski-Górecki
c43407d28d
Merge remote-tracking branch 'origin/pr/135'
* origin/pr/135:
  Added an option to qvm-firewall to reset all rules
2020-03-10 03:11:26 +01:00
Marek Marczykowski-Górecki
916db9db96
tests/firewall: update future time to be really in the future
Mar 4 2020 is no longer future, move it much more into the future.
2020-03-08 17:16:34 +01:00
Marta Marczykowska-Górecka
97ab1d7adf
Added an option to qvm-firewall to reset all rules
Rules will be reset to a single 'accept' line, which is something
that the GUI tools like. It's an easy way to get out of CLI firewall
modifications if someone wants to go back to using GUI for them.

fixes QubesOS/qubes-issues#4710
2020-02-24 14:58:53 +01:00
Marek Marczykowski-Górecki
162f41caf4
Remove no longer relevant TODO comment
vm.shutdown(force=True) is now implemented.
2020-02-07 04:35:38 +01:00
Marek Marczykowski-Górecki
0120e32340
Merge remote-tracking branch 'origin/pr/130'
* origin/pr/130:
  Added explicit 'force' option to qvm-shutdown
  Added force shutdown option to vm.shutdown
2020-02-07 04:35:22 +01:00
Marek Marczykowski-Górecki
9b70e54719
Merge remote-tracking branch 'origin/pr/134'
* origin/pr/134:
  qvm-kill: make error message more clear
  qvm-kill: ignore already off domains
2020-01-31 15:01:26 +01:00
Pawel Marczewski
bcb91594b2
qvm-kill: make error message more clear
See QubesOS/qubes-issues#5607.
2020-01-31 11:26:57 +01:00
Pawel Marczewski
b8d1884cb1
qvm-kill: ignore already off domains
See QubesOS/qubes-issues#5607.
2020-01-31 11:26:54 +01:00
Marta Marczykowska-Górecka
5c94c72ecd
Added explicit 'force' option to qvm-shutdown 2020-01-29 19:51:04 +01:00
Marta Marczykowska-Górecka
2be77f58b3
Added force shutdown option to vm.shutdown
Furthermore makes qvm-shutdown --all use that option to force
shutdown and avoid unnecessary errors.

requires https://github.com/QubesOS/qubes-core-admin/pull/312
fixes QubesOS/qubes-issues#5591
fixes QubesOS/qubes-issues#4572
2020-01-29 19:35:29 +01:00
Pawel Marczewski
2597ed0795
Use qubes.VMExec in run_with_args() 2020-01-27 10:24:22 +01:00
Pawel Marczewski
ff9b81cc3e
qvm-run: use qubes.VMExec, if available
See QubesOS/qubes-issues#4850.
2020-01-24 18:40:11 +01:00
Marek Marczykowski-Górecki
47188a38e5
Merge remote-tracking branch 'origin/pr/128'
* origin/pr/128:
  Import data using ImportWithSize call, not manual resizing

Fixes QubesOS/qubes-issues#5239
2020-01-24 01:49:04 +01:00
Pawel Marczewski
9cf05e5180
Import data using ImportWithSize call, not manual resizing
See the API call in QubesOS/qubes-core-admin#309.
2020-01-23 10:29:50 +01:00
Pawel Marczewski
bf78aebf7c
qvm-remove: improve error message
The code assumes that QubesVMInUseError is thrown for a failed
dependency check, but there are also other reasons. We need to
handle the case when dependency list is empty, and also display
the original message.

See also QubesOS/qubes-core-admin#308 for which this was needed.
2020-01-21 15:54:18 +01:00
Marek Marczykowski-Górecki
204c33afd1
Merge remote-tracking branch 'origin/pr/127'
* origin/pr/127:
  qvm-shutdown: report errors, don't crash on DispVMs
2020-01-17 05:06:18 +01:00
Marek Marczykowski-Górecki
31db7ecbc0
Merge remote-tracking branch 'origin/pr/124'
* origin/pr/124:
  Add missing docstrings
  Add a test for vm.get_mem()
  Update tests to mock admin.vm.CurrentState
  qvm-ls: display memory as integer number
  Use admin.vm.CurrentState method for power state and memory

Fixes QubesOS/qubes-issues#5166
2020-01-17 04:54:33 +01:00
Pawel Marczewski
405464a67e
qvm-shutdown: report errors, don't crash on DispVMs
qvm-shutdown with the --wait option checks if the machine
state is 'Halted', but a disposable VM is usually deleted by
the time of the final check, resulting in a non-zero exit code.

This change handles properly disposable VMs, and makes sure
we always output an error message when finishing with a non-zero
exit code.

Fixes QubesOS/qubes-issues#5245.
2020-01-16 12:38:10 +01:00
Pawel Marczewski
88f177f564
Add missing docstrings 2020-01-16 09:52:06 +01:00
Pawel Marczewski
023f2f593c
Add a test for vm.get_mem() 2020-01-15 17:32:25 +01:00
Pawel Marczewski
e5612592d4
Update tests to mock admin.vm.CurrentState 2020-01-15 17:32:24 +01:00
Pawel Marczewski
0659e372ee
qvm-ls: display memory as integer number 2020-01-15 17:32:24 +01:00
Pawel Marczewski
d5c9328a8c
Use admin.vm.CurrentState method for power state and memory
Depends on the method introduced in QubesOS/qubes-core-admin#307.

Adding the get_mem() method using the same CurrentState admin
method also fixes the bug with qvm-ls not showing memory, see
QubesOS/qubes-issues#5166.
2020-01-15 17:32:23 +01:00
Pawel Marczewski
c35c2f9695
Add test for qvm-ls --all 2020-01-15 15:10:15 +01:00
Pawel Marczewski
b112ef250d
qvm-ls: don't exclude dom0 when using --all
Fixes QubesOS/qubes-issues#5248.
2020-01-15 14:16:45 +01:00
Marta Marczykowska-Górecka
1b53de8365
Fixed Exceptions inferiting from KeyError
Due to strangeness of KeyError (it overrrides str method) in some
cases exceptions received superflous quotation marks when inheriting
from it.

fixes QubesOS/qubes-issues#5106
2019-12-09 21:05:18 +01:00
Marek Marczykowski-Górecki
a722e4234e
Merge remote-tracking branch 'origin/pr/119'
* origin/pr/119:
  Fixed metadata usage being served as strings
2019-11-26 23:40:37 +01:00
Marta Marczykowska-Górecka
05edea703d
Fixed metadata usage being served as strings
Metadata usage should be served as ints, as it is numeric.
2019-11-19 00:39:17 +01:00
Marta Marczykowska-Górecka
ba1601b5bb
Fixed overeager assert
Pool usage details can actually be empty.
2019-11-17 19:17:42 +01:00
Marek Marczykowski-Górecki
6444d31af7
Merge remote-tracking branch 'origin/pr/115'
* origin/pr/115:
  Added an exception for backup already in progress
2019-11-12 22:08:18 +01:00
Marta Marczykowska-Górecka
d877671cc3
Added an exception for backup already in progress
Thrown when user tries to run the same backup twice.

references QubesOS/qubes-issues#5432
2019-11-11 17:17:07 +01:00
Frédéric Pierret (fepitre)
7ae8b86a6e
tests: make PEP8 and pylint happy 2019-11-11 12:14:18 +01:00
Frédéric Pierret (fepitre)
056d7474c3
tests: add default_guivm and guivm in utils properties 2019-11-11 12:13:59 +01:00
Frédéric Pierret (fepitre)
6b50953a33
utils: make PEP8 happy 2019-11-11 11:19:32 +01:00
Frédéric Pierret (fepitre)
f71b63de05
utils: handle default_guivm and guivm properties 2019-11-11 11:17:11 +01:00
Frédéric Pierret (fepitre)
6b5e695f15
prefs: handle 'none' value for guivm 2019-11-10 10:12:58 +01:00
Marek Marczykowski-Górecki
6e76d1c928
Merge remote-tracking branch 'origin/pr/111'
* origin/pr/111:
  Add tests for --tree option of qvm-ls
  Sort domains in network tree list
  network tree manpage entry added
  added feature to list domains as network tree
2019-11-09 15:26:59 +01:00
Rot127
008daa9e56
Add tests for --tree option of qvm-ls 2019-11-08 00:16:04 +01:00
Rot127
e5d4cd1e2c
Sort domains in network tree list 2019-11-08 00:15:27 +01:00
Saswat Padhi
28321ddac1
Preserve backward compatibility 2019-11-04 00:57:21 -08:00
Saswat Padhi
f9cf088e90
Added a new test and minor updates to others 2019-11-03 05:02:28 -08:00
Saswat Padhi
5da0216b94
Fixed a bug and removed unused import 2019-11-03 05:02:21 -08:00
Saswat Padhi
704bf50e0a
Updated tests for qvm-pool 2019-11-03 04:18:57 -08:00
Saswat Padhi
434d8c60bd
Improvements to qvm-pool argument parser
This change introduces a complete rewrite of the argument parser for the `qvm-pool` tool.
As suggested by @marmarek in qubes-issues#5407, the goal is to be consistent with other tools such as `qvm-device`, `qvm-volume` etc.

(resolves QubesOS/qubes-issues#5407)
2019-11-03 04:06:43 -08:00
Rot127
566aefd352 added feature to list domains as network tree 2019-11-01 17:16:43 +01:00
Marek Marczykowski-Górecki
81b3152fa8
Merge remote-tracking branch 'origin/pr/110'
* origin/pr/110:
  Added pool.usage_details API method
  Fixed error with import
2019-10-23 13:54:01 +02:00
Marta Marczykowska-Górecka
d92fdd1264
Added pool.usage_details API method
It provides information about data and metadata usage.
2019-10-23 02:00:10 +02:00
Marta Marczykowska-Górecka
cb82606328
Fixed error with import
The import workaround didn't actually work.
Python does not handle "import x as y; from y import z"
and it worked only by accident.
2019-10-22 22:27:02 +02:00
Frédéric Pierret (fepitre)
03bc15dd38
app: switch get_local_name method to property 2019-10-20 21:19:16 +02:00
Frédéric Pierret (fepitre)
f7740546c9
tests: qvm_start_gui: make PEP8 happier 2019-10-20 19:25:21 +02:00
Frédéric Pierret (fepitre)
63e63f0bcf
tests: fix with respect to gui properties 2019-10-20 19:20:58 +02:00
Frédéric Pierret (fepitre)
8a27c1919f
app: get_local_name set/from self attribute 2019-10-20 19:20:40 +02:00
Frédéric Pierret (fepitre)
40d9696b90
app: fix missing docstring for get_local_name 2019-10-20 17:41:41 +02:00
Frédéric Pierret (fepitre)
411e2884cc
qvm-start-gui: simplify log info for start_gui function 2019-10-20 16:07:27 +02:00
Frédéric Pierret (fepitre)
e63510b4cf
guivm: use getattr instead of try/except and direct property access 2019-10-20 15:43:34 +02:00
Frédéric Pierret (fepitre)
6b4a99f83f
qvm-start-gui: handle GuiVM 2019-10-20 15:37:57 +02:00