Commit Graph

391 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
4b00ef7ec7
tools/qvm-run: fix error reporting on qvm-run -n on halted qube
Fixes QubesOS/qubes-issues#4476
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
d34b1bfc4f
tools/qvm-run: factor out print_no_color function
Will be useful for other messages that should not be in red.
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
32cbc59ba9
tools/qvm-run: handle Ctrl+C nicely
Do not exit with ugly python backtrace, simply interrupt the command
(propagate SIGINT) and exit.

QubesOS/qubes-issues#4532
2018-12-08 00:44:17 +01:00
Marek Marczykowski-Górecki
9acce13a35
tools: fix qvm-run --pass-io --localcmd=... vmname command
qubes.VMShell service, used by qvm-run, expects the command on the first
input line. Previously, when --localcmd was used, the command wasn't
written anywhere and the local command was connected directly to
qubes.VMShell service. And the first line of its output was interpreted
as a command.

Fix this by starting the local command separately, after sending the
command to qubes.VMShell service.

While at it, unify handling shell command and service calls in the process.
vm.run_service(..., localcmd= ) isn't that useful in general case,
because for qubes.VMShell the caller first need to send the command
before starting local process. Since the qvm-run tool needs to implement
manual starting localcmd anyway, don't use localcmd= run_service's
argument at all to unify calling methods.

There is slight behavior change: previously localcmd was started only
after establishing service connection (for example only if qrexec policy
allows), now it is started in all the cases.

Fixes QubesOS/qubes-issues#4040
2018-12-08 00:44:02 +01:00
Marek Marczykowski-Górecki
e6202d496d
tests/tools: improve qvm-run tests
Fix most FD/process leaks, make qvm-run --passio tests working (as much
as possible).
2018-12-07 23:55:03 +01:00
Marek Marczykowski-Górecki
1d877742ae
tests/tools: extend vm.run_service() mockup
Handle stdout/stderr arguments and provide file-like objects when
requested.
2018-12-07 04:25:00 +01:00
GammaSQ
aebc944704
reverted to comparing .index 2018-12-04 09:40:54 +01:00
Marek Marczykowski-Górecki
28067f70ac
backup: simplify test expressions to make pylint happy 2018-12-03 23:27:10 +01:00
Marek Marczykowski-Górecki
377c80d33d
Remove useless 'pass' to make pylint happy 2018-12-03 23:22:50 +01:00
Marek Marczykowski-Górecki
86fe230092
Clarify QubesBase(), Qubes() and QubesLocal/QubesRemote usage
Add note in QubesBase docstring it shouldn't be used directly.
Additionally add base qubesd_call and run_service methods raising
NotImplementedError with helpful message. Lack of qubesd_call in
QubesBase leads to infinite recursion, because one in PropertyHolder
calls itself then.

Fixes QubesOS/qubes-issues#4568
2018-12-03 23:09:23 +01:00
GammaSQ
39c2c7bcd2
forgot braces 2018-12-03 15:55:21 +01:00
GammaSQ
2e637b5631
get_label now accepts integer and has correct return-type 2018-12-03 15:29:06 +01:00
GammaSQ
e55b530865
added --standalone and --disp shortcuts 2018-12-03 09:21:01 +01:00
Marek Marczykowski-Górecki
861e4fb04f
tools: clarify qvm-service --help 2018-10-29 05:27:05 +01:00
Marek Marczykowski-Górecki
3d53e7e310
Code style fix 2018-10-29 05:27:05 +01:00
Marek Marczykowski-Górecki
67897e3f9f
Copy application menu on VM clone
The qubesd daemon have no information about clone source - from that
side it looks like a new VM.  This means application menu is created as
for a new VM.
To fix this re-initialize menu with --source option as part of the clone
operation. It will copy both list of available applications (if
applicable) and selected applications.

This fixes both qvm-clone case and rename.

Fixes QubesOS/qubes-issues#3902
Fixes QubesOS/qubes-issues#4124
2018-10-29 05:27:05 +01:00
Marek Marczykowski-Górecki
5078d75aa3
tools/qvm-create: fix handling invalid label 2018-10-18 03:24:24 +02:00
Marek Marczykowski-Górecki
759fafea63
tools/qvm-create: properly create template-based StandaloneVM
By definition StandaloneVM is not linked to the template. Creating one
from a template is a clone operation. It's already possible using
qvm-clone tool, but it's logical to do that using qvm-create tool too.
This was the case in R3.2 too.

While adding this special case, skip cloning private volume, to preserve
behaviour of TemplateBaseVMs which do not inherit private volume either.

Fixes QubesOS/qubes-issues#3793
2018-10-18 03:24:15 +02:00
Marek Marczykowski-Górecki
4ca6c32e6c
app: add option to ignore select volumes on app.clone_vm()
QubesOS/qubes-issues#3793
2018-10-18 02:43:09 +02:00
Patrik Hagara
4cd513757b
qvm-ls: add filtering by domain power state 2018-10-10 20:06:35 +02:00
Marek Marczykowski-Górecki
e7bc8f21d5
events: try reconnecting to qubesd also on BrokenPipeError
When qubesd exits while writing to it, not reading, the error is
BrokenPipe, not EOF. Handle it the same.
2018-10-07 02:46:28 +02:00
Marek Marczykowski-Górecki
0dfdab32e1
Merge branch 'devices-api'
* devices-api:
  devices: include devclass when comparing devices
  events: deserialize DeviceInfo class in device-* events
  devices: drop DeviceInfo.options
2018-10-07 02:46:14 +02:00
Marek Marczykowski-Górecki
5414739272
devices: include devclass when comparing devices 2018-09-29 14:13:06 +02:00
Marek Marczykowski-Górecki
907c8da005
Merge remote-tracking branch 'origin/pr/77'
* origin/pr/77:
  Exclude installed_by_rpm when Restoring VM from Backup
2018-09-27 14:57:08 +02:00
Marek Marczykowski-Górecki
5b02520ea4
Merge remote-tracking branch 'origin/pr/76'
* origin/pr/76:
  Fix typo
2018-09-27 14:56:25 +02:00
Marek Marczykowski-Górecki
4a1e90392b
events: deserialize DeviceInfo class in device-* events 2018-09-27 14:52:53 +02:00
jimtahu
5f868b4794
Exclude installed_by_rpm when Restoring VM from Backup
VMs listed as from an rpm when they were really from a backup
lead to a missleading error message when deleting.

Fixes QubesOS/qubes-issues#4192
2018-09-22 22:39:22 -05:00
Marek Marczykowski-Górecki
a8c24bee0f
devices: drop DeviceInfo.options
It's part of DeviceAssignment class, here it was leftover from draft
API.
2018-09-18 21:54:51 +02:00
Marek Marczykowski-Górecki
bf240cc928
devices: add DeviceInfo.devclass attribute
Have DeviceInfo also carry information about its class, to avoid the
need to iterate all the devices to get that information.
2018-09-18 16:50:00 +02:00
AJ Jordan
8b38a36de2
Fix typo 2018-09-14 14:48:22 -04:00
Marek Marczykowski-Górecki
0a8c6e2481
events: fix reconnecting to qubesd on its restart
Since now event listener reports proper QubesDaemonCommunicationError
exception instead of some form of IOError. Include it for automatic
reconnect logic.

Fixes a481490 "app: fix error reporting when connection to qubesd fails"
2018-09-13 14:29:27 +02:00
Marek Marczykowski-Górecki
fe73313da1
events: add support for wildcard event handlers
Port 5a39e777089d8bde6d0a620830a898c1cf3dd924 ("events: add support for
wildcard event handlers") from qubes-core-admin:

    Support registering handlers for more flexible wildcard events: not only
    '*', but also 'something*'. This allows to register handlers for
    'property-set:*' and such.
2018-09-07 18:59:05 +02:00
Marek Marczykowski-Górecki
045bad13e7
tools/qvm-create: resize root volume if needed before imporing data
If file to be imported is larger than the default root volume, resize
the volume first. It might be also a good idea to shrink it when needed,
but currently the backend refuse it.

Fixes QubesOS/qubes-issues#3422
2018-09-05 04:29:36 +02:00
Patrik Hagara
cd4424235b
qvm-ls: add filtering by tags 2018-08-26 22:08:32 +02:00
Marek Marczykowski-Górecki
584bd052ed
Adjust shell input for Windows shell, for qvm-run tool
Windows shell (cmd.exe) use '&' as a separator for multiple commands in
the same line.

Fixes QubesOS/qubes-issues#4165
2018-07-30 18:31:04 +02:00
Marta Marczykowska-Górecka
47b4e86736
Added test for qvm-remove dependency reporting 2018-07-20 01:35:00 +02:00
Marta Marczykowska-Górecka
ad2a6e3408
Better information on error in qvm-remove
If qvm-remove fails because the VM is in use, it will display
information about where it is used.

fixes QubesOS/qubes-issues#3193
2018-07-20 00:21:47 +02:00
Marta Marczykowska-Górecka
ca848ca7bd
Name change of vm dependencies helper function
As per @marmarek's request. Also docstrings.
2018-07-19 21:24:28 +02:00
Marta Marczykowska-Górecka
f9564ffa2b
Added tests and minor correction to vm_usage helper function
Now the function returns a list of tuples, not a list of lists.
2018-07-19 19:52:22 +02:00
Marta Marczykowska-Górecka
e6427f97dc
Helper function that lists where a given VM is used
A helper function to list vm's usage added to the utils.
2018-07-18 23:50:54 +02:00
Marek Marczykowski-Górecki
55c9a82c0a
Merge branch 'backup-logging-deadlock'
* backup-logging-deadlock:
  backup: avoid deadlock on logging during restore
2018-07-17 15:46:17 +02:00
Marek Marczykowski-Górecki
a7bfc03509
backup: avoid deadlock on logging during restore
When both threads and processes are used, logging module can deadlock.
Workaround this by re-initializating locks in logging module.
See https://bugs.python.org/issue6721 for details.
Revert this commit when the python issue is fixed (in the python version
used for backup restore).
2018-07-16 22:30:35 +02:00
Marek Marczykowski-Górecki
87122e54c9
Merge branch 'windows-tools'
* windows-tools:
  doc: add info what properties are inherited from template
  Add 'gui-emulated' feature
  qvm-start-gui: fix handlign rpc-clipboard feature
2018-07-16 22:06:44 +02:00
Marek Marczykowski-Górecki
4d61407f5d
Add 'gui-emulated' feature
Add an explicit method for forcing emulated VGA output. Previously it
was possible only by removing `gui` feature (setting it to false had a
different effect), or enabling debug mode.
Using lack of a feature as a third state was a bad idea.

QubesOS/qubes-issues#3585
2018-07-16 04:25:56 +02:00
Marek Marczykowski-Górecki
16064f6fb4
Fixes for pylint 2.0
Ignore most of them - we still support python 2.7 here.
Fix no-else-return.
2018-07-16 02:25:25 +02:00
Marek Marczykowski-Górecki
72a2fd646d
tools/qvm-start-gui: do not crash on qubesd restart
Do not crash if qubesd restart exactly the moment event handler was
called.
Event listener properly handle qubesd restarts (reconnects), but exists
if any handler raise an exception. Avoid this by logging such exceptions
but not propagating them.
2018-07-14 03:31:03 +02:00
Marek Marczykowski-Górecki
a481490d60
app: fix error reporting when connection to qubesd fails
Properly report QubesDaemonCommunicationError, instead of confusing
IOError or FileNotFoundError
2018-07-14 03:31:03 +02:00
Marek Marczykowski-Górecki
95ce30a9e3
qvm-start-gui: fix handlign rpc-clipboard feature
Pass -Q option to both stubdoman's gui daemon and actual VM's gui
daemon.

QubesOS/qubes-issues#3585
2018-07-11 23:39:37 +02:00
Marek Marczykowski-Górecki
d995e2b213
Merge remote-tracking branch 'qubesos/pr/68'
* qubesos/pr/68:
  Add test of qvm_prefs for specal case "None"
  Change "None" in qvm_prefs
  Change "None" to Property None

Fixes QubesOS/qubes-issues#3942
2018-06-27 03:02:34 +02:00
Marek Marczykowski-Górecki
b359f9ca5e
tools: don't fail qvm-template-postprocess on shutdown race condition
It may happen that shutdown timeout expires at exact time that template
has stopped.
2018-06-27 02:24:03 +02:00
Marek Marczykowski-Górecki
0caf6f735d
Fix reporting events even if its source is not available anymore
This is especially the case for domain-shutdown event for DispVM - when
handled, DispVM can be already removed. Give the handled VM object even
if actual VM is already removed. For this, avoid VM existence check by
using domains.get_blind() method.

And actually implement domains.get_blind() method - it was present only
in generic collection, but not VMCollection.
2018-06-27 02:24:02 +02:00
jimtahu
d1455ce6c3
Add test of qvm_prefs for specal case "None" 2018-06-13 23:38:25 -05:00
jimtahu
4156fa38a5
Change "None" in qvm_prefs
The string "none" is now changed to '' in qvm_prefs, and
only for certain properties (eg netvm).
2018-06-13 23:38:16 -05:00
jimtahu
c0373980af
Change "None" to Property None
This changes to accept the string none as the value None,
as if an empty string was entered. This allows setting the
netvm to "None" as described in QubesOS/qubes-issues#3942
2018-06-07 20:28:41 -05:00
Peter Gerber
b2a70f3794
tools/qvm-firewall: improve manpage and --help output 2018-05-03 01:03:35 +02:00
Peter Gerber
00fbfdd77a
tools/qvm-firewall: add dst4 and dst6 as synonyms for dsthost 2018-05-03 00:49:43 +02:00
Peter Gerber
ed9b42d5b4
tools/qvm-firewall: Show EXPIRE column in list output 2018-04-30 21:30:52 +02:00
Marek Marczykowski-Górecki
ab79bd2a44
tools/qvm-firewall: make 'list' a default action 2018-04-30 04:16:23 +02:00
Marek Marczykowski-Górecki
7f79075088
tools/qvm-firewall: add 'expire' rule support 2018-04-30 04:13:45 +02:00
Peter Gerber
e69b4fb1bd
tools/qvm-firewall: Describe available rules in --help output 2018-04-29 15:47:59 +02:00
Marek Marczykowski-Górecki
c75c0176dc
tools/qvm-template-postprocess: fix reinstall action
Do not remove template directory after reinstalling it in dom0.

Fixes QubesOS/qubes-issues#3169
2018-04-21 15:07:21 +02:00
Marek Marczykowski-Górecki
fbd5ca4150
tools/qvm-ls: add --kernel option for compatibility with R3.2
There was such option on Qubes 3.2, so add it here too. This is
especially useful for kernel package - preun script use it to verify if
given kernel isn't needed anymore.
2018-04-21 02:51:31 +02:00
Marek Marczykowski-Górecki
90df051f4f
tools/qvm-pool: add --set to modify pool properties
Fixes QubesOS/qubes-issues#3256
2018-04-13 00:26:25 +02:00
Marek Marczykowski-Górecki
471523167e
tools: fix error reporing on unknown storage pool 2018-04-12 23:19:34 +02:00
Marek Marczykowski-Górecki
efae2dfd38
Merge branch 'template-postprocess-fail'
* template-postprocess-fail:
  qvm-template-postprocess: do not remove VM on failed reinstall
  qvm-template-postprocess: resize volume only when needed
2018-03-20 19:22:33 +01:00
Marek Marczykowski-Górecki
a99acc68da
Merge branch 'improved-tools-messages'
* improved-tools-messages:
  tools: suppress full traceback in console tools
  tools: add SubParsersHelpAction, which include subcommands details in --help
2018-03-20 19:19:40 +01:00
Marek Marczykowski-Górecki
70b15c2eae
qvm-volume: refuse to shrink volume unless --force option is used
Right now Admin API backend will refuse to shrink volume anyway, but
we're planning to relax this restriction. Make sure the client side
(qvm-volume tool here, GUI VM settings already have this in place) will
employ appropriate safety check.

QubesOS/qubes-issues#3725
2018-03-20 17:53:36 +01:00
Marek Marczykowski-Górecki
4638a019e5
storage: make Pool.{size,usage} integers
And return None (instead of raising KeyError) when pool driver
does not provide such information.
2018-03-20 16:50:56 +01:00
Marek Marczykowski-Górecki
96a32bd096
qvm-template-postprocess: do not remove VM on failed reinstall
If root volume import fails on template reinstall, do not remove it -
keep it alone, with old volume content

QubesOS/qubes-issues#3169
2018-03-19 20:47:16 +01:00
Marek Marczykowski-Górecki
562137c36d
qvm-template-postprocess: resize volume only when needed
If needs to be extended - do it before import. If needs to be reduced -
after. This way, if data import fails for any reason, previous data
won't be destroyed (truncated).
Also, convert error on shrinking volume to a warning, as it doesn't break
the template (just leave it with bigger disk than needed). Currently all
storage pool implementations refuse to shrink a volume (but it may
change in the future).

QubesOS/qubes-issues#3169
2018-03-19 20:47:15 +01:00
Marek Marczykowski-Górecki
034e9b3a24
qvm-volume: add 'info' and 'config' actions
This allows to get and set volumes properties.

Fixes QubesOS/qubes-issues#3256
2018-03-18 23:38:10 +01:00
Marek Marczykowski-Górecki
6ca54e18a3
tools: suppress full traceback in console tools
QubesException class is used with meaningful messages and should be ok
to use it directly as error message. For other exceptions, still use
full traceback (most likely a bug somewhere, not user error).

Fixes QubesOS/qubes-issues#3610
2018-03-18 21:38:49 +01:00
Marek Marczykowski-Górecki
c70e440a6c
tools: add SubParsersHelpAction, which include subcommands details in --help 2018-03-18 21:38:31 +01:00
Marek Marczykowski-Górecki
b57b101b04
storage: add 'rw' and 'revision_to_keep' setters
QubesOS/qubes-issues#3256
2018-03-18 17:00:39 +01:00
Marek Marczykowski-Górecki
b1237bfe1f
tests: add tests for various 'qvm-prefs vm pref' cases 2018-03-04 03:43:07 +01:00
Marek Marczykowski-Górecki
2459b8108b
tools/qvm-prefs: fix printing None value
Don't print None value as 'None' string, but as empty one (same as at
API level). Otherwise it is indistinguishable from VM named 'None', or
same string property value.
2018-03-04 03:43:06 +01:00
Marek Marczykowski-Górecki
4a9b57f91f
qvm-template-postprocess: call fstrim after removing image file
This is especially important on LVM thin pool, where space after
removing the file needs to be given back to the pool, to be reused for
other volumes (for example this template).
2018-03-04 03:43:06 +01:00
Marek Marczykowski-Górecki
090cccd468
Merge remote-tracking branch 'qubesos/pr/56'
* qubesos/pr/56:
  qvm-backup-restore: also handle absent --ignore-missing
2018-02-27 02:32:37 +01:00
Marek Marczykowski-Górecki
9bf404f7bf
tools: terminate qvm-start-gui when X server is terminated
qvm-start-gui lifecycle should be bound to X server lifecycle. It should
be restarted when user logoff and login again, at least to start
gui-daemons again.
Do that by opening a connection to X server and reacting to breaking
that socket.

Fixes QubesOS/qubes-issues#3147
2018-02-26 22:11:25 +01:00
Rusty Bird
cf063a9638
qvm-backup-restore: also handle absent --ignore-missing
This was forgotten in 2d8bade.
2018-02-26 04:10:05 +00:00
Rusty Bird
0171269138
restore.py: add template-related debug log messages 2018-02-24 12:43:42 +00:00
Rusty Bird
ab65338cfb
restory.py: restore template_for_dispvms property early 2018-02-24 12:43:42 +00:00
Rusty Bird
c86b731d3e
restore.py: factor out _restore_property() helper 2018-02-24 12:43:42 +00:00
Rusty Bird
2ace32bb73
restore.py: deal with DispVM templates 2018-02-24 12:43:42 +00:00
Rusty Bird
68c8b7fafc
restore.py: sort into three tiers in _templates_first() 2018-02-24 12:43:42 +00:00
Rusty Bird
01ce5387d3
restore.py: don't try to restore dispid property 2018-02-23 17:15:20 +00:00
Rusty Bird
78571898da
qvm-backup: let backup core handle default VM selection
This takes the include_in_backups property into account, and does not
omit dom0.
2018-02-23 02:09:06 +00:00
Rusty Bird
2d8bade8b2
qvm-backup-restore: really pass options 2018-02-23 02:09:04 +00:00
Rusty Bird
a0d6327532
qvm-backup-restore: remove orphaned --replace-template option 2018-02-23 02:09:03 +00:00
Marek Marczykowski-Górecki
5a4577417f
Merge remote-tracking branch 'qubesos/pr/53'
* qubesos/pr/53:
  template-postprocess: fast path for file-reflink, like file
2018-02-14 04:45:59 +01:00
Marek Marczykowski-Górecki
d07475427f
Merge remote-tracking branch 'qubesos/pr/52'
* qubesos/pr/52:
  backup: don't crash when no 'qubes' group is present
  tests: dom0 backup restore, both v3 and v4
  backup: add support for openssl 1.1.0 options
  backup: skip dom0's properties while restoring core2 backup
  unused variable
  style issues
  Fix dom0 restore
  Fix dom0 handling
  Fix AdminVm class name
2018-02-14 04:45:12 +01:00
Marek Marczykowski-Górecki
56dc984507
backup: don't crash when no 'qubes' group is present
This is specifically the case on Travis-CI. But since dom0 backup is
restored into separate directory now, instead of directly overriding
user home, this check is much less relevant now.
2018-02-14 04:28:23 +01:00
Marek Marczykowski-Górecki
060171f19f
tests: dom0 backup restore, both v3 and v4
QubesOS/qubes-issues#3467
2018-02-14 04:28:23 +01:00
Marek Marczykowski-Górecki
a821034653
backup: add support for openssl 1.1.0 options
The command for listing available algorithms have changed:
old: openssl list-message-digest-algorithms
new: openssl list -digest-algorithms
2018-02-14 04:28:23 +01:00
Marek Marczykowski-Górecki
04635e7e80
backup: skip dom0's properties while restoring core2 backup
Fixes QubesOS/qubes-issues#3467
2018-02-14 04:28:22 +01:00
Marek Marczykowski-Górecki
24cfb4a581
vm: restore force_shutdown->kill deprecated alias
Apparently there are still users of this function - qubes manager.
2018-02-13 19:01:29 +01:00
Rusty Bird
e6466c3783
template-postprocess: fast path for file-reflink, like file
I don't know if any template currently hits this code path, even the
fedora-26-minimal root.img is large enough to be split into multiple
parts. Maybe Arch Linux?

Related to https://github.com/QubesOS/qubes-core-admin/pull/188
2018-02-12 21:51:01 +00:00
Christopher Laprise
e81c2f3cf6
unused variable 2018-02-01 23:20:31 -05:00
Christopher Laprise
e3d12fb96d
style issues 2018-02-01 23:00:06 -05:00
Christopher Laprise
30dd7acaa9
Fix dom0 restore 2018-02-01 22:21:24 -05:00