Commit Graph

676 Commits

Author SHA1 Message Date
WillyPillow
3d42c988f0 Various cleanup and improvements.
- `qvm-template list`: show template state
- `qvm-template list`: only call qubes.TemplateSearch once
- `qvm-template list`: use `qubesadmin.tools.print_table()` instead of own implementation
- `qvm-template download`: custom progress bar
- Use `run_service` instead of own implementation
- Remove some erroneous/redundant lines
2020-07-29 20:55:02 +08:00
WillyPillow
0e8e8d98de Better way of detecting VM. 2020-07-29 20:55:02 +08:00
WillyPillow
b634c7c785 Initial commit of qvm-template.
Refer to <https://gist.github.com/WillyPillow/61ee5f48b7c5b7cc90c9fd2ec5c1b20d>
for previous revisions.
2020-07-29 20:55:02 +08:00
WillyPillow
bab8e699d7 Change "whitelist" to "menu-items" in qvm-features for clarity. 2020-07-29 20:55:02 +08:00
WillyPillow
e8ba117c26 Allow virt_mode other than pv. 2020-07-29 20:55:02 +08:00
WillyPillow
eda68cce6d Verify values of boolean flags in template config. 2020-07-29 20:55:02 +08:00
WillyPillow
6c7360f25c Separate whitelist entries with spaces instead of newlines. 2020-07-29 20:55:02 +08:00
WillyPillow
9d9ee6a4b7 Initial support for qvm-template. 2020-07-29 20:55:02 +08:00
Paweł Marczewski
624e4e32fb
Add qubes-guivm-session utility
To be used in an xsession file (/usr/share/xsessions).
2020-07-29 12:11:03 +02:00
Paweł Marczewski
c6be7ca5cc
qvm-start-daemon: allow --watch without --all
Allow specifying VM names to look for.
2020-07-29 11:35:32 +02:00
Paweł Marczewski
cb7f191bd2
qvm-start-daemon: convert to async/await syntax 2020-07-29 11:27:19 +02:00
Marek Marczykowski-Górecki
a078e1f617
Make Label() object hashable
Since it got custom __eq__ function, __hash__ needs to be implemented
too. Otherwise it can't be used as a key in dict.
2020-07-29 03:50:11 +02:00
Marek Marczykowski-Górecki
af1c1e5234
tests: update for admin.vm.volume.Clear usage in qvm-template-postprocess 2020-07-16 04:07:07 +02:00
Marek Marczykowski-Górecki
4da218c332
Merge remote-tracking branch 'origin/pr/149'
* origin/pr/149:
  Add admin.vm.volume.Clear call (QubesOS/qubes-issues#5946)
2020-07-16 04:05:38 +02:00
Marek Marczykowski-Górecki
b99e45f081
Merge remote-tracking branch 'origin/pr/146'
* origin/pr/146:
  Added dynamic X keyboard event monitoring to qvm_start_daemon.py

Fixes QubesOS/qubes-issues#1396
Fixes QubesOS/qubes-issues#4294
2020-07-15 15:34:07 +02:00
Marta Marczykowska-Górecka
1446a6d7ee
Added dynamic X keyboard event monitoring to qvm_start_daemon.py
Update keyboard_layout property whenever guivm's layout changes, instead of
only at the start.

requires QubesOS/qubes-core-admin#350

references QubesOS/qubes-issues#1396
references QubesOS/qubes-issues#4294
2020-07-15 14:04:25 +02:00
Marek Marczykowski-Górecki
6f335800b0
Wrap too long line 2020-07-15 14:01:29 +02:00
Marek Marczykowski-Górecki
470514d0dc
Merge remote-tracking branch 'origin/pr/144'
* origin/pr/144:
  Clean up the guid-conf file on domain stop
  Generate qubes-guid options based on features
2020-07-15 14:00:31 +02:00
Marek Marczykowski-Górecki
2bdeb7684a
Merge remote-tracking branch 'origin/pr/151'
* origin/pr/151:
  Fixed inconsistent firewall address checking
2020-07-15 13:50:12 +02:00
Marek Marczykowski-Górecki
f9c5b61961
Merge remote-tracking branch 'origin/pr/150'
* origin/pr/150:
  Fix VM validity check for cached VM objects
2020-07-15 13:49:39 +02:00
Marek Marczykowski-Górecki
e0f17f1b4a
Merge remote-tracking branch 'origin/pr/143'
* origin/pr/143:
  qvm-start-daemon: common_guid_args is now a staticmethod
  tests: kde_args are passed with property of launcher
  Fix missing semi-colon and new line
  Handle KDE with specific arg/desktop file
2020-07-15 13:43:15 +02:00
Marta Marczykowska-Górecka
9377addd1e
Fixed inconsistent firewall address checking
core-admin-client side had less strict rules for hostname than core-admin,
leading to unexpected empty qubesd response. Fixed by copying the logic from core-admin.
2020-07-14 20:10:34 +02:00
Marek Marczykowski-Górecki
45a28c29ae
Fix VM validity check for cached VM objects
Qubes().domains.refresh_cache() tries to preserve cached VM objects if
the class matches - this way if an application keeps reference to any,
it will still be the same as freshly obtained from the collection, and
also it will receive cache updates/invalidates based on events.

The check for class change was invalid - on core-admin-client side we
have just one QubesVM class with 'klass' attribute. This leads to VM
objects being disconnected from VMCollection and stale properties cache
there (because they no longer receive events).

Fix the check.

And also add a test if indeed the same object is returned.
2020-07-14 16:10:49 +02:00
WillyPillow
455542ac7f
Add admin.vm.volume.Clear call (QubesOS/qubes-issues#5946) 2020-07-14 01:48:19 +08:00
Marta Marczykowska-Górecka
1a4cdba7f7
Added better __eq__ method to Label class
Labels with the same name should not be distinguishable.

fixes QubesOS/qubes-issues#5944
2020-07-09 19:28:25 +02:00
Paweł Marczewski
7616a8913a
Clean up the guid-conf file on domain stop 2020-06-29 12:20:23 +02:00
Paweł Marczewski
3540f04a42
Generate qubes-guid options based on features
Allow configuring options per VM or globally per GuiVM. The
qvm-start-daemon program reads the options from VM features, and
generates a configuration file for qubes-guid.

Requires QubesOS/qubes-gui-daemon#47 (customizing the configuration
file).
2020-06-29 12:01:09 +02:00
Frédéric Pierret (fepitre)
f89c4cad56
qvm-start-daemon: common_guid_args is now a staticmethod 2020-06-26 12:18:59 +02:00
Frédéric Pierret (fepitre)
6295b0ba2f
tests: kde_args are passed with property of launcher 2020-06-26 12:05:18 +02:00
Frédéric Pierret (fepitre)
92e87b122e
Handle KDE with specific arg/desktop file 2020-06-24 10:51:39 +02:00
Marek Marczykowski-Górecki
ae39c75867
Merge remote-tracking branch 'origin/pr/140'
* origin/pr/140:
  use function to determine pacat domid
  connect to PA in stubdom if audio-model enabled run pacat in low latency mode by default
2020-06-20 05:01:59 +02:00
Marek Marczykowski-Górecki
9d6b7257c4
tools/qvm-start-daemon: reduce required permissions to sys-gui itself
Do not require permission to list sys-gui itself just to get keyboard
layout. Listing itself is not sensitive (sys-gui knows it exists), but
it will make other tools request its properties, which may not be
desirable.
2020-06-16 15:58:57 +02:00
Dmitry Fedorov
d99045f05e
use function to determine pacat domid 2020-05-31 21:11:33 +03:00
Dmitry Fedorov
ec90829695
connect to PA in stubdom if audio-model enabled
run pacat in low latency mode by default
2020-05-29 12:07:38 +03:00
Marek Marczykowski-Górecki
b1453953f9
Merge remote-tracking branch 'origin/pr/139'
* origin/pr/139:
  Added a safeguard for invalid firewall rules
2020-05-27 04:18:20 +02:00
Marek Marczykowski-Górecki
aea41511de
events: implicitly enable caching when creating EventsDispatcher
Application that runs EventsDispatcher can safely use also cache , which
greatly improve performance. This is because cache then is properly
updated/invalidated when needed.
Instead of modifying each application to explicitly enable cache based
on this simple rule, make it implicit when EventsDispatcher is created.

Do not enable caching when EventsDispatcher is created only temporarily
in wait_for_domain_shutdown.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
c081ed8c82
Enable caching in qvm-ls and qvm-prefs
Both tools issue a large number of Admin API calls and greatly benefit
from a cache filled with a single per-vm Admin API call
(admin.vm.property.GetAll). In case of qvm-ls, this also saves multiple
admin.vm.CurrentState calls (power state is given in the admin.vm.List
response too).

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
79c7392424
Cache power state when caching is enabled
Power state changes are signaled with events too, so it is possible to
cache it and update/invalidate cache with events.
Additionally, admin.vm.List returns a power state, so the cache can be
populated early. This in particular greatly improves qvm-ls performance -
eliminate admin.vm.CurrentState call at all.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
bfe1a3d541
Adjust for changed qubesd socket protocol
The socket protocol is adjusted to match qrexec socket service protocol.

QubesOS/qubes-issues#3293
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
1d1289619c
Implement a wrapper for dom0 -> dom0 qrexec calls
Qrexec itself does not support  loopback calls. Since a call into dom0
is a common case for GUIVM (which may be dom0) to do, add a convenient
handling inside vm.run_service() function.

Fixes QubesOS/qubes-issues#5140
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
02cfab8257
Try to use new property.GetAll method to pre-fill the cache
When caching is enabled, reduce number of calls by getting all the
properties at once. If the call is not available (for example because of
the policy), fallback to getting individual values.

QubesOS/qubes-issues#5415
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
218d43a2e0
Add simple properties caching
Reduce Admin API calls by caching returned values. The cache is not
enabled by default, because it could result in stale values being
returned. It can be enabled by setting 'cache_enabled' to True on
Qubes() object. This is safe in two cases:
 - the application don't care about changed values - like a short-lived
   process that retrieve values once (for example qvm-ls)
 - the application listen for events and invalidate cache when property
   is changed

For the second case, invalidating the cache on appropriate event
(property-set:*, property-reset:*) is done before calling other event
handlers. This is because the event may try to access the property value
(not necessary from the event arguments), so we need to be sure it will
see the new value.

Fixes QubesOS/qubes-issues#5415
2020-05-22 18:59:44 +02:00
Marek Marczykowski-Górecki
5315bbf8f9
Do not check for object existence when got it with another Admin API call
When qubesd returns an name of VM or other object, as part of another
call (reading a property, listing devices etc), it's safe to assume that
object exists. Do not try to list it, which could be prevented by qrexec
policy. This means a VM object would be returned (for example in
vm.netvm property), which potentially could not be listed through
app.domains collection. This may lead to some corner cases, but
generally should ease handling of restricted policy.

This does not affect practical information the management VM have access
too, as those names are already returned. It's just client side python
wrapper that didn't allowed to access them.

QubesOS/qubes-issues#5099
2020-05-18 04:37:48 +02:00
Marta Marczykowska-Górecka
d2f4a4533a
Added a safeguard for invalid firewall rules
Firewall rule cannot be missing value in declaration
(e.g. 'dsthost=' is not a valid rule).

fixes QubesOS/qubes-issues#5772
2020-05-16 12:18:13 +02:00
Marek Marczykowski-Górecki
83b1fc6c58
tools/qvm-volume import: do not retrieve old size anymore
Since admin.vm.volume.ImportWithSize method is used now, there is no
need to retrieve old size anymore (to decide whether to resize or not).
2020-05-10 03:57:28 +02:00
Marek Marczykowski-Górecki
65dcee424a
Make pylint happy
New pylint reports (duplicated?) warning signature-differs in addition
to arguments-differ. Ignore this one too.
2020-05-10 03:21:40 +02:00
Marek Marczykowski-Górecki
4971faa462
Merge remote-tracking branch 'origin/pr/118'
* origin/pr/118:
  qvm-start-daemon: adjust pacat pid file path
  qvm-start-daemon: check if layout is parsed
  qvm-start-daemon: allow multiple options in keyboard layout
  qvm-start-daemon: improve parsing args for setting keyboard layout
  qvm-start-daemon: set keyboard-layout only for the first set layout
  gui: set keyboard layout when starting daemon
  daemon: start it for dom0 unconditionnaly
  qvm-start-daemon: ensure separate task between GUI/AUDIO
  qvm-start-daemon: allow starting only if service enabled
  Fix and improvements from Marek's comments
  Change qvm-start-gui to qvm-start-daemon for handling audio too
  Support for AudioVM
2020-04-09 05:24:26 +02:00
Marek Marczykowski-Górecki
c80314523f
Merge remote-tracking branch 'origin/pr/136'
* origin/pr/136:
  Added tests for cloning VMs with persistent devices
  Remove Python 2 workarounds
  Cloning a vm now clones persistent PCI device assignments
2020-04-09 05:24:13 +02:00
Rusty Bird
f9ee7f5816
collections.Callable -> collections.abc.Callable
"Deprecated since version 3.3, will be removed in version 3.10"
- https://docs.python.org/3/library/collections.html
2020-04-07 21:30:54 +00:00
Frédéric Pierret (fepitre)
e87559b318
qvm-start-daemon: adjust pacat pid file path 2020-04-07 23:12:09 +02:00
Marta Marczykowska-Górecka
f08d41eec5
Added tests for cloning VMs with persistent devices 2020-03-30 16:15:49 +02:00
Marta Marczykowska-Górecka
8e7db06918
Remove Python 2 workarounds
Python 2 support was discontinued and workaround
for Python2 mock is no longer needed.
2020-03-30 15:48:53 +02:00
Marta Marczykowska-Górecka
c0a8c6528a
Cloning a vm now clones persistent PCI device assignments
fixes QubesOS/qubes-issues#4992
2020-03-30 15:48:53 +02:00
Frédéric Pierret (fepitre)
e39f280405
qvm-start-daemon: check if layout is parsed 2020-03-28 17:33:04 +01:00
Frédéric Pierret (fepitre)
2e72f75611
qvm-start-daemon: allow multiple options in keyboard layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
023d94a0b3
qvm-start-daemon: improve parsing args for setting keyboard layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
c13097d458
qvm-start-daemon: set keyboard-layout only for the first set layout 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
c58e5b8eb8
gui: set keyboard layout when starting daemon 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
526c862b5d
daemon: start it for dom0 unconditionnaly 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
49957971ff
qvm-start-daemon: ensure separate task between GUI/AUDIO 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
05e479cf92
qvm-start-daemon: allow starting only if service enabled 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
f7fa577c47
Fix and improvements from Marek's comments 2020-03-28 17:33:03 +01:00
Frédéric Pierret (fepitre)
0e049e682a
Change qvm-start-gui to qvm-start-daemon for handling audio too 2020-03-28 17:33:02 +01:00
Frédéric Pierret (fepitre)
6d9e5bbf07
Support for AudioVM 2020-03-28 17:33:02 +01:00
Marek Marczykowski-Górecki
c43407d28d
Merge remote-tracking branch 'origin/pr/135'
* origin/pr/135:
  Added an option to qvm-firewall to reset all rules
2020-03-10 03:11:26 +01:00
Marek Marczykowski-Górecki
916db9db96
tests/firewall: update future time to be really in the future
Mar 4 2020 is no longer future, move it much more into the future.
2020-03-08 17:16:34 +01:00
Marta Marczykowska-Górecka
97ab1d7adf
Added an option to qvm-firewall to reset all rules
Rules will be reset to a single 'accept' line, which is something
that the GUI tools like. It's an easy way to get out of CLI firewall
modifications if someone wants to go back to using GUI for them.

fixes QubesOS/qubes-issues#4710
2020-02-24 14:58:53 +01:00
Marek Marczykowski-Górecki
162f41caf4
Remove no longer relevant TODO comment
vm.shutdown(force=True) is now implemented.
2020-02-07 04:35:38 +01:00
Marek Marczykowski-Górecki
0120e32340
Merge remote-tracking branch 'origin/pr/130'
* origin/pr/130:
  Added explicit 'force' option to qvm-shutdown
  Added force shutdown option to vm.shutdown
2020-02-07 04:35:22 +01:00
Marek Marczykowski-Górecki
9b70e54719
Merge remote-tracking branch 'origin/pr/134'
* origin/pr/134:
  qvm-kill: make error message more clear
  qvm-kill: ignore already off domains
2020-01-31 15:01:26 +01:00
Pawel Marczewski
bcb91594b2
qvm-kill: make error message more clear
See QubesOS/qubes-issues#5607.
2020-01-31 11:26:57 +01:00
Pawel Marczewski
b8d1884cb1
qvm-kill: ignore already off domains
See QubesOS/qubes-issues#5607.
2020-01-31 11:26:54 +01:00
Marta Marczykowska-Górecka
5c94c72ecd
Added explicit 'force' option to qvm-shutdown 2020-01-29 19:51:04 +01:00
Marta Marczykowska-Górecka
2be77f58b3
Added force shutdown option to vm.shutdown
Furthermore makes qvm-shutdown --all use that option to force
shutdown and avoid unnecessary errors.

requires https://github.com/QubesOS/qubes-core-admin/pull/312
fixes QubesOS/qubes-issues#5591
fixes QubesOS/qubes-issues#4572
2020-01-29 19:35:29 +01:00
Pawel Marczewski
2597ed0795
Use qubes.VMExec in run_with_args() 2020-01-27 10:24:22 +01:00
Pawel Marczewski
ff9b81cc3e
qvm-run: use qubes.VMExec, if available
See QubesOS/qubes-issues#4850.
2020-01-24 18:40:11 +01:00
Marek Marczykowski-Górecki
47188a38e5
Merge remote-tracking branch 'origin/pr/128'
* origin/pr/128:
  Import data using ImportWithSize call, not manual resizing

Fixes QubesOS/qubes-issues#5239
2020-01-24 01:49:04 +01:00
Pawel Marczewski
9cf05e5180
Import data using ImportWithSize call, not manual resizing
See the API call in QubesOS/qubes-core-admin#309.
2020-01-23 10:29:50 +01:00
Pawel Marczewski
bf78aebf7c
qvm-remove: improve error message
The code assumes that QubesVMInUseError is thrown for a failed
dependency check, but there are also other reasons. We need to
handle the case when dependency list is empty, and also display
the original message.

See also QubesOS/qubes-core-admin#308 for which this was needed.
2020-01-21 15:54:18 +01:00
Marek Marczykowski-Górecki
204c33afd1
Merge remote-tracking branch 'origin/pr/127'
* origin/pr/127:
  qvm-shutdown: report errors, don't crash on DispVMs
2020-01-17 05:06:18 +01:00
Marek Marczykowski-Górecki
31db7ecbc0
Merge remote-tracking branch 'origin/pr/124'
* origin/pr/124:
  Add missing docstrings
  Add a test for vm.get_mem()
  Update tests to mock admin.vm.CurrentState
  qvm-ls: display memory as integer number
  Use admin.vm.CurrentState method for power state and memory

Fixes QubesOS/qubes-issues#5166
2020-01-17 04:54:33 +01:00
Pawel Marczewski
405464a67e
qvm-shutdown: report errors, don't crash on DispVMs
qvm-shutdown with the --wait option checks if the machine
state is 'Halted', but a disposable VM is usually deleted by
the time of the final check, resulting in a non-zero exit code.

This change handles properly disposable VMs, and makes sure
we always output an error message when finishing with a non-zero
exit code.

Fixes QubesOS/qubes-issues#5245.
2020-01-16 12:38:10 +01:00
Pawel Marczewski
88f177f564
Add missing docstrings 2020-01-16 09:52:06 +01:00
Pawel Marczewski
023f2f593c
Add a test for vm.get_mem() 2020-01-15 17:32:25 +01:00
Pawel Marczewski
e5612592d4
Update tests to mock admin.vm.CurrentState 2020-01-15 17:32:24 +01:00
Pawel Marczewski
0659e372ee
qvm-ls: display memory as integer number 2020-01-15 17:32:24 +01:00
Pawel Marczewski
d5c9328a8c
Use admin.vm.CurrentState method for power state and memory
Depends on the method introduced in QubesOS/qubes-core-admin#307.

Adding the get_mem() method using the same CurrentState admin
method also fixes the bug with qvm-ls not showing memory, see
QubesOS/qubes-issues#5166.
2020-01-15 17:32:23 +01:00
Pawel Marczewski
c35c2f9695
Add test for qvm-ls --all 2020-01-15 15:10:15 +01:00
Pawel Marczewski
b112ef250d
qvm-ls: don't exclude dom0 when using --all
Fixes QubesOS/qubes-issues#5248.
2020-01-15 14:16:45 +01:00
Marta Marczykowska-Górecka
1b53de8365
Fixed Exceptions inferiting from KeyError
Due to strangeness of KeyError (it overrrides str method) in some
cases exceptions received superflous quotation marks when inheriting
from it.

fixes QubesOS/qubes-issues#5106
2019-12-09 21:05:18 +01:00
Marek Marczykowski-Górecki
a722e4234e
Merge remote-tracking branch 'origin/pr/119'
* origin/pr/119:
  Fixed metadata usage being served as strings
2019-11-26 23:40:37 +01:00
Marta Marczykowska-Górecka
05edea703d
Fixed metadata usage being served as strings
Metadata usage should be served as ints, as it is numeric.
2019-11-19 00:39:17 +01:00
Marta Marczykowska-Górecka
ba1601b5bb
Fixed overeager assert
Pool usage details can actually be empty.
2019-11-17 19:17:42 +01:00
Marek Marczykowski-Górecki
6444d31af7
Merge remote-tracking branch 'origin/pr/115'
* origin/pr/115:
  Added an exception for backup already in progress
2019-11-12 22:08:18 +01:00
Marta Marczykowska-Górecka
d877671cc3
Added an exception for backup already in progress
Thrown when user tries to run the same backup twice.

references QubesOS/qubes-issues#5432
2019-11-11 17:17:07 +01:00
Frédéric Pierret (fepitre)
7ae8b86a6e
tests: make PEP8 and pylint happy 2019-11-11 12:14:18 +01:00
Frédéric Pierret (fepitre)
056d7474c3
tests: add default_guivm and guivm in utils properties 2019-11-11 12:13:59 +01:00
Frédéric Pierret (fepitre)
6b50953a33
utils: make PEP8 happy 2019-11-11 11:19:32 +01:00
Frédéric Pierret (fepitre)
f71b63de05
utils: handle default_guivm and guivm properties 2019-11-11 11:17:11 +01:00
Frédéric Pierret (fepitre)
6b5e695f15
prefs: handle 'none' value for guivm 2019-11-10 10:12:58 +01:00
Marek Marczykowski-Górecki
6e76d1c928
Merge remote-tracking branch 'origin/pr/111'
* origin/pr/111:
  Add tests for --tree option of qvm-ls
  Sort domains in network tree list
  network tree manpage entry added
  added feature to list domains as network tree
2019-11-09 15:26:59 +01:00
Rot127
008daa9e56
Add tests for --tree option of qvm-ls 2019-11-08 00:16:04 +01:00
Rot127
e5d4cd1e2c
Sort domains in network tree list 2019-11-08 00:15:27 +01:00
Saswat Padhi
28321ddac1
Preserve backward compatibility 2019-11-04 00:57:21 -08:00
Saswat Padhi
f9cf088e90
Added a new test and minor updates to others 2019-11-03 05:02:28 -08:00
Saswat Padhi
5da0216b94
Fixed a bug and removed unused import 2019-11-03 05:02:21 -08:00
Saswat Padhi
704bf50e0a
Updated tests for qvm-pool 2019-11-03 04:18:57 -08:00
Saswat Padhi
434d8c60bd
Improvements to qvm-pool argument parser
This change introduces a complete rewrite of the argument parser for the `qvm-pool` tool.
As suggested by @marmarek in qubes-issues#5407, the goal is to be consistent with other tools such as `qvm-device`, `qvm-volume` etc.

(resolves QubesOS/qubes-issues#5407)
2019-11-03 04:06:43 -08:00
Rot127
566aefd352 added feature to list domains as network tree 2019-11-01 17:16:43 +01:00
Marek Marczykowski-Górecki
81b3152fa8
Merge remote-tracking branch 'origin/pr/110'
* origin/pr/110:
  Added pool.usage_details API method
  Fixed error with import
2019-10-23 13:54:01 +02:00
Marta Marczykowska-Górecka
d92fdd1264
Added pool.usage_details API method
It provides information about data and metadata usage.
2019-10-23 02:00:10 +02:00
Marta Marczykowska-Górecka
cb82606328
Fixed error with import
The import workaround didn't actually work.
Python does not handle "import x as y; from y import z"
and it worked only by accident.
2019-10-22 22:27:02 +02:00
Frédéric Pierret (fepitre)
03bc15dd38
app: switch get_local_name method to property 2019-10-20 21:19:16 +02:00
Frédéric Pierret (fepitre)
f7740546c9
tests: qvm_start_gui: make PEP8 happier 2019-10-20 19:25:21 +02:00
Frédéric Pierret (fepitre)
63e63f0bcf
tests: fix with respect to gui properties 2019-10-20 19:20:58 +02:00
Frédéric Pierret (fepitre)
8a27c1919f
app: get_local_name set/from self attribute 2019-10-20 19:20:40 +02:00
Frédéric Pierret (fepitre)
40d9696b90
app: fix missing docstring for get_local_name 2019-10-20 17:41:41 +02:00
Frédéric Pierret (fepitre)
411e2884cc
qvm-start-gui: simplify log info for start_gui function 2019-10-20 16:07:27 +02:00
Frédéric Pierret (fepitre)
e63510b4cf
guivm: use getattr instead of try/except and direct property access 2019-10-20 15:43:34 +02:00
Frédéric Pierret (fepitre)
6b4a99f83f
qvm-start-gui: handle GuiVM 2019-10-20 15:37:57 +02:00
Frédéric Pierret (fepitre)
a5102d752f
Make PEP8 happier 2019-10-20 13:31:40 +02:00
Marek Marczykowski-Górecki
2d736f5aa8
Make pylint happy 2019-10-05 21:57:48 +02:00
Marek Marczykowski-Górecki
e8c48ff7c8
tools/qvm-start-gui: avoid starting a VM just to send it monitor layout
If the VM is powered off shortly after startup, it may happen that
qubes.SetMonitorLayout would start it again. Avoid this by using
autostart=False argument to run_service() function.

Note that there is a vm.is_running() check at the beginning of the
function already, but if it happens while the VM is cleaned up, it may
still report that it's running, but at the time of run_service() call it
is not.
2019-09-26 19:21:13 +02:00
Marek Marczykowski-Górecki
98260ff148
Add run_service(..., autostart=False) argument
This allows to run a service but do not cause a qube to be started it
isn't already running. This is especially useful for background /
internal calls designed to service a running target VM - if VM is not
running, those do not make sense to be called in the first place.

Specifically, this will allow qvm-start-gui to avoid re-starting a
domain while calling qubes.NotifyMonitorLayout, when a VM is shutdown
shortly after its startup.
2019-09-26 19:21:07 +02:00
Marek Marczykowski-Górecki
1fcb031192
Add support for run_service(..., filter_esc=True) in a VM
Since qrexec-client-vm got support for filtering escape characters, use
it here too.

QubesOS/qubes-issues#5322
2019-09-21 04:55:18 +02:00
Marek Marczykowski-Górecki
22bc5d880a
Merge remote-tracking branch 'origin/pr/105'
* origin/pr/105:
  Fixed error with VM being incorrectly listed as its own dependency
2019-09-21 03:34:31 +02:00
Marek Marczykowski-Górecki
556cec4e3f
Merge remote-tracking branch 'origin/pr/104'
* origin/pr/104:
  dochelpers: make PEP8 happier
  dochelpers: handle legacy sphinx library
  Fix Sphinx 2 new API for Fedora 31+
2019-09-21 03:34:27 +02:00
Marek Marczykowski-Górecki
1f1564dcf0
backup/restore: make pylint happy 2019-09-21 03:20:43 +02:00
Marta Marczykowska-Górecka
3e2b432644
Fixed error with VM being incorrectly listed as its own dependency
fixes QubesOS/qubes-issues#4224
2019-09-17 00:59:59 +02:00
Marek Marczykowski-Górecki
48ad8325d0
tests: check rejecting/accepting compression filters
3 new tests:
- custom (common) compression filter
- custom (uncommon) compression filter - should be rejected
- custom (uncommon) compression filter forced - accepted
2019-09-10 15:24:15 +02:00
Marek Marczykowski-Górecki
14f77860bf
backup/restore: add option to use uncommon compression filter anyway
Previous commit introduced protection against uncommon (potentially
malicious) compression filters. This breaks restoring backups made with
a custom compression filter. Add an option to override this check, by
naming compression filter to use explicitly.
2019-09-10 15:24:15 +02:00
Marek Marczykowski-Górecki
10f15e6669
backup/restore: reject uncommon compression filters and improve header validation
Compression filter named in a backup header is executed in restore
environment (commonly dom0). While this field is properly authenticated,
there may be cases where backup archive comes from less
trusted source, like migrating from potentially compromised
system.

Modify backup header parsing code to add field specific validators.
Whitelist only know crypto, hmac and compression algorithms.

Based on a patch by Jean-Philippe Ouellet <jpo@vt.edu>
Reported-by: Jean-Philippe Ouellet <jpo@vt.edu>
2019-09-10 15:24:15 +02:00
Frédéric Pierret (fepitre)
3ddeb2046a
dochelpers: make PEP8 happier 2019-09-06 16:29:56 +02:00
Frédéric Pierret (fepitre)
6df676c259
dochelpers: handle legacy sphinx library 2019-09-06 16:29:52 +02:00
Frédéric Pierret (fepitre)
ab8f487b50
Fix Sphinx 2 new API for Fedora 31+
QubesOS/qubes-issues#5289
2019-09-06 16:24:30 +02:00
Frédéric Pierret (fepitre)
8639034bb6
qvm-device: prevent parser allowing abbreviations 2019-08-24 13:39:37 +02:00
Frédéric Pierret (fepitre)
41c6fbed72
qvm-device: handle 'list-device-classes' and 'list-classes'
QubesOS/qubes-issues#5213
2019-08-24 13:39:36 +02:00
Frédéric Pierret (fepitre)
3dce4e9742
qubesadmin: define methods list_vmclass and list_devicesclass
Adapt also previous direct calls of qubesdb

QubesOS/qubes-issues#5213
2019-08-10 22:08:31 +02:00
Frédéric Pierret (fepitre)
a982e1e538
qubesadmin: make PEP8 happy 2019-08-10 19:03:41 +02:00
Marek Marczykowski-Górecki
769f8a5ee8
Merge remote-tracking branch 'origin/pr/98'
* origin/pr/98:
  qvm-check: fix from Marek's comment
  qvm-check: refactor check mechanism and add filter for checking netvm
2019-08-08 15:35:34 +02:00
Frédéric Pierret (fepitre)
eb461e4c3b
qvm-check: fix from Marek's comment 2019-08-08 14:29:35 +02:00
Frédéric Pierret (fepitre)
7d93377b78
qvm-check: refactor check mechanism and add filter for checking netvm
Fix QubesOS/qubes-issues#3496
2019-08-08 14:26:05 +02:00
Marek Marczykowski-Górecki
489efce9cb
Merge remote-tracking branch 'origin/pr/99'
* origin/pr/99:
  devices: add missing docstring for _get_device_classes
  devices: make iteration device classes compatible with Python2
  tools/qvm-device: make PEP8 happy
  tests/devices: add test for handling listing device classes
  tests/devices: make PEP8 happy
  devices: handle listing of available device classes
  devices: make PEP8 happy
2019-08-08 14:13:38 +02:00
Frédéric Pierret (fepitre)
08d746e119
devices: add missing docstring for _get_device_classes 2019-08-07 22:10:37 +02:00
Frédéric Pierret (fepitre)
268d11fbff
devices: make iteration device classes compatible with Python2 2019-08-07 20:38:30 +02:00
Frédéric Pierret (fepitre)
174f155d31
tools/qvm-device: make PEP8 happy 2019-08-06 15:15:12 +02:00
Frédéric Pierret (fepitre)
4d0545405f
tests/devices: add test for handling listing device classes 2019-08-06 15:15:12 +02:00
Frédéric Pierret (fepitre)
07ca511bd7
tests/devices: make PEP8 happy 2019-08-06 15:15:03 +02:00
Frédéric Pierret (fepitre)
f633dd9b40
devices: handle listing of available device classes
QubesOS/qubes-issues#5213
2019-08-06 15:09:44 +02:00
Frédéric Pierret (fepitre)
1052217973
devices: make PEP8 happy 2019-08-06 15:07:33 +02:00
Marek Marczykowski-Górecki
e700af9eb2
tools/qvm-volume: add 'import' action
Add support for importing volume data with qvm-volume tool.
This could be also used to clear volume by issuing:

    qvm-volume import --no-resize some-vm:private /dev/null

QubesOS/qubes-issues#5192
2019-07-29 22:20:06 +02:00
Marek Marczykowski-Górecki
fdc632c959
tools: reset private volume when importing template over existing one
Reinstalling template is a recommended way to get it back to a clean
state after potential compromise. In that case it is essential to
discard any persistent storage of old template, as it could be used by
the attacker to re-compromise it after reinstall.
Do this similar as root volume is overridden - via volume import
function.

Fixes QubesOS/qubes-issues#5192
2019-07-29 22:20:06 +02:00
Marek Marczykowski-Górecki
21569b3a31
tests: fix events tests on python3.7
Fix syntax workaround for python3.7 + python2.7. This code can't use
'yield from' to be still importable on python2.7, but asyncio.sleep() is
no longer iterable on python3.7. Workaround it by manually calling
coroutine.send(None) in a loop - ugly as hell, but works. I can't wait
until we could drop python2 support...
2019-07-29 21:56:32 +02:00
Marek Marczykowski-Górecki
b8ddb39752
Allow app.domains[vm] where vm is a QubesVM object
Similar as in the core-admin API.
2019-06-25 06:28:00 +02:00
Malte Leip
32538fa5ec
qvm-prefs: add --hide-default option 2019-05-29 20:32:13 +02:00
Marek Marczykowski-Górecki
c1be4cd062
Merge remote-tracking branch 'qubesos/pr/92'
* qubesos/pr/92:
  Adds docs for clone ignore_errors, feeds linter
  Adds support for `--ignore-errors` to qvm-clone CLI
2019-04-17 00:30:10 +02:00
Marek Marczykowski-Górecki
37e4e4c6ca
tests: update yaml syntax in qvm-backup tests
yaml.safe_dump() finally produces canonical yaml output, not a
one-liner.
2019-04-16 23:10:38 +02:00
Conor Schaefer
40eeddbfc8
Adds docs for clone ignore_errors, feeds linter
Addresses review comments, specifically:

  * updates qvm-clone tests to handle ignore_errors option
  * adds manpage reference to new --ignore-errors option
  * trims line length in qvm-clone changes

All tests should now be passing, including the new qvm-clone
functionality.
2019-04-16 10:44:47 -07:00
Conor Schaefer
55a22e7955
Adds support for --ignore-errors to qvm-clone CLI
The underlying API already supported the `ignore_errors` param in
Python, now we just expose that option to the `qvm-clone` CLI.
2019-04-15 17:33:41 -07:00
Marek Marczykowski-Górecki
d7430d42ce
Make pylint happy
no-else-raise warning
2019-03-07 03:17:29 +01:00
Marek Marczykowski-Górecki
05f0d4ca0f
Make vm.run_with_args working on python2
This part of the qubesadmin module still needs to be compatible with
python2.
2019-03-07 03:05:09 +01:00
Marek Marczykowski-Górecki
fb910a71cf
tools/qvm-start: validate output of losetup command
QubesOS/qubes-issues#4860
2019-03-07 02:53:52 +01:00
Marek Marczykowski-Górecki
a2629b1239
tools/qvm-start: use vm.run_with_args to call losetup in the VM
This will fix handling filenames with spaces and shell special
characters.

Reported by @v6ak

Fixes QubesOS/qubes-issues#4860
2019-03-07 02:53:45 +01:00
Marek Marczykowski-Górecki
a35cb46342
tests: vm.run, vm.run_with_args
QubesOS/qubes-issues#4850
2019-02-28 05:34:42 +01:00
Marek Marczykowski-Górecki
0092c0ac99
vm: add run_with_args method to call a single command with specific arguments
The method will ensure the parameters are properly handled as actual
separate arguments, regardless of shell special characters in them.
For now implement this with shlex.quote, later dedicated qrexec service
could be used.

QubesOS/qubes-issues#4850
2019-02-28 04:37:59 +01:00
Marek Marczykowski-Górecki
88ab55b940
events: simplify handling qubesd reconnect
Retry connection on any OSError, instead of only few selected error
types. For example PermissionError may happen too (when socket is
created, but before it gets appropriate mode), which wasn't listed.
2019-02-24 05:32:51 +01:00
Marek Marczykowski-Górecki
1145f70334
tools/qvm-backup: allow to disable compression
Specify compression explicitly in the backup profile, not only when
requested with --compress or --compress-filter.
This will allow to disable compression with --no-compress option, as the
default if no compression is specified in the profile is to use gzip.

Fixes QubesOS/qubes-issues#4803
2019-02-24 05:32:46 +01:00
Marek Marczykowski-Górecki
5e4831ede4
backup: Do not try to attach devices during backup verification
The "restored" VM object will be discarded anyway, and it doesn't have
real 'attach' method.

Fixes QubesOS/qubes-issues#4830
2019-02-24 02:26:05 +01:00
Marek Marczykowski-Górecki
2ad9b57961
utils: include management_dispvm property when checking where VM is used
Fixes QubesOS/qubes-issues#4812
2019-02-24 02:26:05 +01:00
Marek Marczykowski-Górecki
b1a191446a
tools/qvm-run: do not color the output unless --pass-io is used
Since no output from VM is passed (and even if it would, it's redirected
to /dev/null), there is no need to switch output color.
This fixes the case when qvm-run is started in background - the color
change would affect further shell output.

Fixes QubesOS/qubes-issues#4808
2019-02-11 19:23:35 +01:00
Marek Marczykowski-Górecki
89ee4931fb
backup: do not fail restore if scrypt anticipate more time needed
For various reasons, decryption may be slower than initial encryption
(different hardware, different system load etc). Do not fail the restore
operation if scrypt anticipate it will take considerably more time or
memory, than while creating the backup (where the scrypt parameters were
originally set).

Thanks to @jharveyb for the report and suggested solution.
Fixes QubesOS/qubes-issues#4683
2019-01-09 19:09:44 +01:00
Marek Marczykowski-Górecki
a65ef425e1
tools/qvm-run: make -q affect 'command failed with code' message
Fixes QubesOS/qubes-issues#4616
2018-12-16 05:51:41 +01:00
Marek Marczykowski-Górecki
9061169f90
Merge branch 'devel-20181206' 2018-12-09 18:08:25 +01:00
Marek Marczykowski-Górecki
0bb35193b4
doc: fix rst syntax in documentation, including docstrings 2018-12-08 23:53:55 +01:00
Marek Marczykowski-Górecki
558c5d4a7c
Merge remote-tracking branch 'origin/pr/84'
* origin/pr/84:
  Fixed typo in documentation
  Documentation is important! Added shortcuts to manpages
  added tests for shortcuts
  diff doesn't set template
  added --standalone and --disp shortcuts
2018-12-08 15:10:42 +01:00
GammaSQ
b848625428
added tests for shortcuts 2018-12-08 12:38:26 +01:00
GammaSQ
ce2fb96a4b
diff doesn't set template 2018-12-08 12:29:48 +01:00
Marek Marczykowski-Górecki
4a727f1dfa
Merge remote-tracking branch 'origin/pr/85'
* origin/pr/85:
  isinstance instead of type
  reverted to comparing .index
  forgot braces
  forgot braces
  get_label now accepts integer and has correct return-type
2018-12-08 12:28:36 +01:00
GammaSQ
2c836c5adc
isinstance instead of type 2018-12-08 12:00:15 +01:00
Marek Marczykowski-Górecki
415f5faae9
tools/qvm-run: factor out actual process running into separate function
Pylint complains that main() is too long and that's indeed true. Factor
out single process call (together with all the prepartion) into separate
function and keep only common code in main().
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
5fe6ffc04c
tools/qvm-create: reject --root-{move,copy}-from with template-based qubes
Fixes QubesOS/qubes-issues#4424
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
f03937c51e
tools: remove extra debug 2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
e827e47926
Clone VM's volume into the same pool, unless overridden specifically
When cloning VM, create it in the same pool as the source one.
Previously it always used default pool, which means for example renaming
a VM in non-default pool moved it back to the default one.

Fixes QubesOS/qubes-issues#4145
Fixes QubesOS/qubes-issues#4523
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
bee55a3bce
tools/qvm-device: allow detaching all devices
QubesOS/qubes-issues#4530
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
954ffc4bf2
tools/qvm-run: add error message if command in VM failed
Print some error even without --pass-io, otherwise the only way to learn
the failure is checking $?, as no other visual sign is there.

Fixes QubesOS/qubes-issues#4533
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
4b00ef7ec7
tools/qvm-run: fix error reporting on qvm-run -n on halted qube
Fixes QubesOS/qubes-issues#4476
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
d34b1bfc4f
tools/qvm-run: factor out print_no_color function
Will be useful for other messages that should not be in red.
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
32cbc59ba9
tools/qvm-run: handle Ctrl+C nicely
Do not exit with ugly python backtrace, simply interrupt the command
(propagate SIGINT) and exit.

QubesOS/qubes-issues#4532
2018-12-08 00:44:17 +01:00
Marek Marczykowski-Górecki
9acce13a35
tools: fix qvm-run --pass-io --localcmd=... vmname command
qubes.VMShell service, used by qvm-run, expects the command on the first
input line. Previously, when --localcmd was used, the command wasn't
written anywhere and the local command was connected directly to
qubes.VMShell service. And the first line of its output was interpreted
as a command.

Fix this by starting the local command separately, after sending the
command to qubes.VMShell service.

While at it, unify handling shell command and service calls in the process.
vm.run_service(..., localcmd= ) isn't that useful in general case,
because for qubes.VMShell the caller first need to send the command
before starting local process. Since the qvm-run tool needs to implement
manual starting localcmd anyway, don't use localcmd= run_service's
argument at all to unify calling methods.

There is slight behavior change: previously localcmd was started only
after establishing service connection (for example only if qrexec policy
allows), now it is started in all the cases.

Fixes QubesOS/qubes-issues#4040
2018-12-08 00:44:02 +01:00
Marek Marczykowski-Górecki
e6202d496d
tests/tools: improve qvm-run tests
Fix most FD/process leaks, make qvm-run --passio tests working (as much
as possible).
2018-12-07 23:55:03 +01:00
Marek Marczykowski-Górecki
1d877742ae
tests/tools: extend vm.run_service() mockup
Handle stdout/stderr arguments and provide file-like objects when
requested.
2018-12-07 04:25:00 +01:00
GammaSQ
aebc944704
reverted to comparing .index 2018-12-04 09:40:54 +01:00
Marek Marczykowski-Górecki
28067f70ac
backup: simplify test expressions to make pylint happy 2018-12-03 23:27:10 +01:00
Marek Marczykowski-Górecki
377c80d33d
Remove useless 'pass' to make pylint happy 2018-12-03 23:22:50 +01:00
Marek Marczykowski-Górecki
86fe230092
Clarify QubesBase(), Qubes() and QubesLocal/QubesRemote usage
Add note in QubesBase docstring it shouldn't be used directly.
Additionally add base qubesd_call and run_service methods raising
NotImplementedError with helpful message. Lack of qubesd_call in
QubesBase leads to infinite recursion, because one in PropertyHolder
calls itself then.

Fixes QubesOS/qubes-issues#4568
2018-12-03 23:09:23 +01:00
GammaSQ
39c2c7bcd2
forgot braces 2018-12-03 15:55:21 +01:00
GammaSQ
2e637b5631
get_label now accepts integer and has correct return-type 2018-12-03 15:29:06 +01:00
GammaSQ
e55b530865
added --standalone and --disp shortcuts 2018-12-03 09:21:01 +01:00
Marek Marczykowski-Górecki
861e4fb04f
tools: clarify qvm-service --help 2018-10-29 05:27:05 +01:00
Marek Marczykowski-Górecki
3d53e7e310
Code style fix 2018-10-29 05:27:05 +01:00