Commit Graph

175 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
090cccd468
Merge remote-tracking branch 'qubesos/pr/56'
* qubesos/pr/56:
  qvm-backup-restore: also handle absent --ignore-missing
2018-02-27 02:32:37 +01:00
Marek Marczykowski-Górecki
9bf404f7bf
tools: terminate qvm-start-gui when X server is terminated
qvm-start-gui lifecycle should be bound to X server lifecycle. It should
be restarted when user logoff and login again, at least to start
gui-daemons again.
Do that by opening a connection to X server and reacting to breaking
that socket.

Fixes QubesOS/qubes-issues#3147
2018-02-26 22:11:25 +01:00
Rusty Bird
cf063a9638
qvm-backup-restore: also handle absent --ignore-missing
This was forgotten in 2d8bade.
2018-02-26 04:10:05 +00:00
Rusty Bird
78571898da
qvm-backup: let backup core handle default VM selection
This takes the include_in_backups property into account, and does not
omit dom0.
2018-02-23 02:09:06 +00:00
Rusty Bird
2d8bade8b2
qvm-backup-restore: really pass options 2018-02-23 02:09:04 +00:00
Rusty Bird
a0d6327532
qvm-backup-restore: remove orphaned --replace-template option 2018-02-23 02:09:03 +00:00
Marek Marczykowski-Górecki
5a4577417f
Merge remote-tracking branch 'qubesos/pr/53'
* qubesos/pr/53:
  template-postprocess: fast path for file-reflink, like file
2018-02-14 04:45:59 +01:00
Rusty Bird
e6466c3783
template-postprocess: fast path for file-reflink, like file
I don't know if any template currently hits this code path, even the
fedora-26-minimal root.img is large enough to be split into multiple
parts. Maybe Arch Linux?

Related to https://github.com/QubesOS/qubes-core-admin/pull/188
2018-02-12 21:51:01 +00:00
Christopher Laprise
30dd7acaa9
Fix dom0 restore 2018-02-01 22:21:24 -05:00
Marek Marczykowski-Górecki
3d4f626515
Merge remote-tracking branch 'qubesos/pr/51'
* qubesos/pr/51:
  qvm-run: wait for X11 in --dispvm --gui case
  vm/DispVM: use 'qrexec_timeout' also for call connection timeout
2018-01-15 03:23:54 +01:00
Rusty Bird
c83deccdd3
qvm-run: wait for X11 in --dispvm --gui case
'qvm-run --dispvm' cannot easily make a separate qubes.WaitForSession
call. Instead, if --gui is active, pass the new WaitForSession argument
to qubes.VMShell, which will do the equivalent.

The unit tests have been copied (in slightly adapted form) from commit
a620f02e2a

Fixes QubesOS/qubes-issues#3012
Closes QubesOS/qubes-core-admin-client#49
2018-01-14 18:36:46 +00:00
Marek Marczykowski-Górecki
04f8d85819
base: make WrapperObjectsCollection behave like a dict
In core-admin matching collections are real dicts, so clone this API
behaviour here too. Specific changes:
 - iteration yields keys, not values
 - implement values and items methods

Additionally fix keys method, it was broken on python2 (list have no
copy method).
2018-01-12 05:29:40 +01:00
Marek Marczykowski-Górecki
8584ca4057
tools/qvm-ls: clarify that disk usage is in percents 2018-01-12 05:29:40 +01:00
Marek Marczykowski-Górecki
b16f3ab79b
tools/qvm-ls: add --network and --disk compatibility options
Shortcuts for new --format=...
2018-01-12 05:29:39 +01:00
Marek Marczykowski-Górecki
080b563ae2
tools/qvm-ls: add --raw-list option for compatibility with R3.2
There was an option to list just VM names, useful for scripting.
Preserve that for compatibility.
2018-01-12 05:29:39 +01:00
Marek Marczykowski-Górecki
0533703ddf
tools/qvm-ls: allow list only selected VMs
Restore possibility to list only some VMs, by naming them on command
line.
2018-01-12 05:29:39 +01:00
Marek Marczykowski-Górecki
4af13d5445
tools: fix handling optional domain list
It wasn't possible to use QubesArgumentParser(vmname_nargs=...) for
optional domain list - the option forced usage of either --all or
explicit domain list.
2018-01-12 05:27:57 +01:00
Marek Marczykowski-Górecki
9b5546f1b0
tools/qvm-start: add naive waiting for cdrom device
When starting a VM with --cdrom=some-vm:/some/path/to.iso, it can be
started only when loop device matching the path is available. For now,
add naive waiting (while ... sleep(1)) for it. Later it might worth
converting it to events handling.
2018-01-12 05:27:57 +01:00
Rusty Bird
58cecf6b3f
qvm-device/block: Alias '--ro' to '--option read-only=yes'
It's quicker to type and compatible with pre-R4.0 habits/scripts.
2018-01-10 19:30:30 +00:00
Marek Marczykowski-Górecki
51a89a9e77
Merge remote-tracking branch 'qubesos/pr/44'
* qubesos/pr/44:
  Fix style else-return
  tests: update qvm-template-process and qvm-remove tests
  Add --force to manpage.
  Avoid cloning installed_by_rpm
  Print vm list before prompt
  Use --force instead of --yes
  Toggle installed_by_rpm in template tool
  Fix error message grammar
  Add --yes option and confirm prompt.
2018-01-06 14:54:32 +01:00
Christopher Laprise
acd5f2e524
Fix style else-return 2018-01-05 23:13:24 -05:00
Christopher Laprise
8b03c9cc9c
Print vm list before prompt 2018-01-02 01:51:01 -05:00
Christopher Laprise
91928ae0c6
Use --force instead of --yes 2018-01-02 01:33:20 -05:00
Christopher Laprise
f7d27cdcdc
Toggle installed_by_rpm in template tool 2018-01-02 01:19:06 -05:00
Christopher Laprise
ac7461d1e8
Fix error message grammar 2018-01-02 00:42:47 -05:00
Christopher Laprise
ed11346977
Add --yes option and confirm prompt. 2017-12-29 14:17:30 -05:00
Marek Marczykowski-Górecki
917df5a8c5
tools/qvm-ls: fix handling columns with underscore
Mangling column type was inconsistent, resulting in
KeyError('VIRT_MODE').
2017-12-29 03:24:22 +01:00
Marek Marczykowski-Górecki
c2281fefc8
Merge remote-tracking branch 'qubesos/pr/43'
* qubesos/pr/43:
  Fix dom0-related issues
2017-12-22 17:19:59 +01:00
Christopher Laprise
c6bc4f05cb
Fix dom0-related issues 2017-12-11 10:16:53 -05:00
Marek Marczykowski-Górecki
bc9fd47aa0
Another approach for making qvm-shutdown nice for tests 2017-12-10 22:52:29 +01:00
Marek Marczykowski-Górecki
448fbb8309
make qvm-shutdown nice for unit tests
Make it deterministic.
2017-12-10 22:39:39 +01:00
Marek Marczykowski-Górecki
3541b7d883
Merge remote-tracking branch 'qubesos/pr/42'
* qubesos/pr/42:
  verify-only avoid dom0 handler
  Fix verify-only
  Issue #3303
2017-12-10 22:24:55 +01:00
Christopher Laprise
f86c640d65
Fix verify-only 2017-12-06 21:31:30 -05:00
Marek Marczykowski-Górecki
cb069367d5
Merge remote-tracking branch 'qubesos/pr/41'
* qubesos/pr/41:
  Remove dead code
2017-12-05 22:23:02 +01:00
Jean-Philippe Ouellet
123aa28e87
Remove dead code 2017-11-30 06:57:03 -05:00
Jean-Philippe Ouellet
16ae4a64b6
Disable spinner if --raw-data 2017-11-30 06:48:03 -05:00
Jean-Philippe Ouellet
b52371f614
Fix qvm-ls --raw-data 2017-11-29 21:33:50 -05:00
Marek Marczykowski-Górecki
afe4eef15f
Merge remote-tracking branch 'qubesos/pr/38'
* qubesos/pr/38:
  Restore correct colors in verbose mode
  Fix setting stderr default coloring policy
2017-11-20 13:04:42 +01:00
Jean-Philippe Ouellet
6afe5a98c2
Restore correct colors in verbose mode
Restore stderr color when printing to stderr, not stdout color
2017-11-20 00:45:53 -05:00
Jean-Philippe Ouellet
57bf88aa30
Fix setting stderr default coloring policy
stderr default should be set if stderr is None, not if stdout is None
2017-11-20 00:44:07 -05:00
qubesuser
dbd2f0ee06 remove format_doc and docutils import
This wastes 100-200ms loading and removing markup every run and none
of the strings contain any markup anyway...
2017-11-10 16:31:45 +01:00
Marek Marczykowski-Górecki
dc5afa56bf
Merge remote-tracking branch 'qubesos/pr/35'
* qubesos/pr/35:
  make qvm-run work for non-blocking stdin
2017-11-07 22:22:29 +01:00
qubesuser
cb644eb174 make qvm-run work for non-blocking stdin
The main process sometimes sets fd 1 to O_NONBLOCK, and since in the
terminal case fd 0 and 1 are the same fd, this also results in fd 0
being non-blocking, causing qvm-run to crash with EAGAIN.

So just make the code work for both blocking and non-blocking stdin.
2017-11-07 20:43:16 +01:00
Marek Marczykowski-Górecki
7bcab46f96
tools/qvm-shutdown: fix handling shutdown timeout for multiple VMs
When some VM timeout on shutdown, the tool will try to kill all of them,
but at this point some of them may be already powered off (not all
hanged during shutdown, but only some). Handle this
situation instead of crashing. And add appropriate test.
2017-10-28 22:40:24 +02:00
Marek Marczykowski-Górecki
2fac77da6f
Document device attach options in man page
Fixes QubesOS/qubes-issues#3204
2017-10-24 03:24:43 +02:00
Marek Marczykowski-Górecki
8981e71b88
tools: fix handling qvm-run --no-autostart
Fixes QubesOS/qubes-issues#3083
2017-10-19 14:55:33 +02:00
Marek Marczykowski-Górecki
cbdc2170b4
tools: clarify that qvm-backup needs absolute path
Fixes QubesOS/qubes-issues#3066
2017-10-16 03:09:10 +02:00
Marek Marczykowski-Górecki
0187588531
tools: adjust qvm-start message and logic, update tests
QubesOS/qubes-issues#3130
2017-10-08 18:17:03 +02:00
Marek Marczykowski-Górecki
bf4c9bacb4
Merge remote-tracking branch 'qubesos/pr/28'
* qubesos/pr/28:
  qvm-start: exit if domain running and no --skip-if-running
2017-10-08 17:14:47 +02:00
Marek Marczykowski-Górecki
559cc9b0d6
tools/qvm-start: fix starting a VM with cdrom image
1. Output of `losetup` command contains `\n` - strip it.
2. Provide read-only option - if device info hasn't propagated to qubesd
yet, it will not be set automatically.

Fixes QubesOS/qubes-issues#3146
2017-10-04 15:18:28 +02:00
Marek Marczykowski-Górecki
8476afc306
tools: finish 'qvm-volume revert' implementation, add tests
It wasn't aware of snapshot identifier, fix that. Also update to use
VM:VOLUME syntax, not POOL:VID.
2017-10-02 21:12:17 +02:00
Marek Marczykowski-Górecki
edcaed537a
Always use QubesVM objects, instead of AppVM/TemplateVM etc
Very few calls at client side really needs VM class name. So, even in
non-blind mode use just QubesVM class, to avoid strange cases depending
on blind mode being enabled or not. Then, have VM class name in 'klass'
property. If known at object creation time, cache it, otherwise query
qubesd at first access.
2017-10-02 21:12:16 +02:00
Mindy Preston
76c4a6421b qvm-start: exit if domain running and no --skip-if-running 2017-09-27 17:15:52 -05:00
Marek Marczykowski-Górecki
ab9a57a544
tools: fix qvm-run --help
QubesOS/qubes-issues#
2017-09-20 22:53:30 +02:00
Marek Marczykowski-Górecki
5e2638ab5e
tools: restore qvm-start --cdrom and similar options
Booting a VM from cdrom require attaching the device before VM startup,
which is possible only in persistent mode. But for qvm-start --cdrom
adding a cdrom only temporarily, use new update_persistence() function
to convert the assignment to temporary one.

Fixes QubesOS/qubes-issues#3055
2017-09-05 08:33:47 +02:00
Marek Marczykowski-Górecki
b895015274
tools: cleanup temporary files after qvm-template-postprocess 2017-09-05 03:57:55 +02:00
Marek Marczykowski-Górecki
336807e8da
tools: fix too long lines
QubesOS/qubes-issues#2983
2017-09-04 03:07:03 +02:00
Marek Marczykowski-Górecki
533fccfc10
Merge remote-tracking branch 'qubesos/pr/24'
* qubesos/pr/24:
  Fixed https://github.com/QubesOS/qubes-issues/issues/2983
2017-09-04 03:02:49 +02:00
Marek Marczykowski-Górecki
0fa374264f
tools: add qvm-create --help-classes
List available VM classes

QubesOS/qubes-issues#3017
2017-08-30 16:54:24 +02:00
Marek Marczykowski-Górecki
0464a3ebfc
tools: implement qvm-create --root-move-from/--root-copy-from 2017-08-30 16:54:23 +02:00
Marek Marczykowski-Górecki
a260685bd1
tools: drop --delete and --unset aliases for --default option
This may be confiusing, for example one may think that
`qvm-prefs --unset vmname netvm` will make vmname network-disconnected.
This type of mistakes may have severe security consequence, so better
drop those option names.

QubesOS/qubes-issues#3002

cc @rootkovska
2017-08-12 22:33:36 +02:00
Marek Marczykowski-Górecki
596eb3d96c
tools/qvm-run: ignore --filter-esc/--no-filter-esc unless --pass-io is given
If no VM output is printed on terminal, those options doesn't make sense
anyway.

Fixes QubesOS/qubes-issues#3013
2017-08-12 14:19:41 +02:00
blackpit
6534f8c191 Fixed https://github.com/QubesOS/qubes-issues/issues/2983
qvm-create -l yellow backuptest-1-vm
qvm-create -l yellow backuptest-2-vm
echo -n passphrase >test.passphrase

qvm-backup --passphrase-file test.passphrase --yes `pwd` backuptest-1-vm backuptest-2-vm

qvm-remove backuptest-1-vm1 || true
qvm-remove backuptest-2-vm1 || true
qvm-backup-restore --rename-conflicting --passphrase-file test.passphrase qubes-2017-08-09T165253 backuptest-1-vm

qvm-remove backuptest-1-vm1 || true
qvm-remove backuptest-2-vm1 || true
qvm-backup-restore --rename-conflicting --passphrase-file test.passphrase qubes-2017-08-09T165253 backuptest-2-vm
2017-08-09 19:43:53 +02:00
Marek Marczykowski-Górecki
414be0945d
tools/qvm-device: list attach options 2017-08-09 04:04:27 +02:00
Marek Marczykowski-Górecki
430ff342d4
tools/qvm-run: do not use os.set_blocking
It's available only in python >= 3.5, but we do support tools on python
3.4 (Debian jessie).
2017-08-09 04:04:26 +02:00
Marek Marczykowski-Górecki
37ae76823b
tools: add qvm-run --dispvm option
Add option to uniformly start new DispVM from either VM or Dom0. This
use DispVMWrapper, which translate it to either qrexec call to $dispvm,
or (in dom0) to appropriate Admin API call to create fresh DispVM
first.
This require abandoning registering --all and --exclude by
QubesArgumentParser, because we need to add --dispvm mutually exclusive
with those two. But actually handling those two options is still done by
QubesArgumentParser.

This also updates man page and tests.

Fixes QubesOS/qubes-issues#2974
2017-08-06 20:44:55 +02:00
Marek Marczykowski-Górecki
50bd9f5fab
tools/qvm-run: fix handling small data blocks
When data block is smaller than 4096 (and no EOF is reached), python's
io.read() will call read(2) again to get more data. This may deadlock if
the other end of connection will write anything only after receiveing
data (which is the case for qubes.Filecopy).
Disable this buffering by using syscall wrappers directly. To not affect
performance that much, increase buffer size to 64k.

Fixes QubesOS/qubes-issues#2948
2017-08-02 02:43:03 +02:00
Marek Marczykowski-Górecki
c4460751a9
tools/qvm-start-gui: do not start stubdomain GUI for VMs with gui-agent 2017-07-30 18:54:08 +02:00
Marek Marczykowski-Górecki
f48321dd98
Merge branch 'devel-6'
* devel-6:
  qvm-ls: fix total VM size reporting
  doc: update manpage of qvm-service
  tools: qvm-service tool
  tests: too much copy&paste
  features: serialize True as '1'
  tools/qvm-start-gui: add --force-stubdomain options
  tools/qvm-shutdown: fix help message
  tools/qvm-shutdown: drop --force option, it isn't supported anymore
2017-07-29 05:28:57 +02:00
Marek Marczykowski-Górecki
9210048673
backup/restore: add option to ignore size limit
Allow to restore backup which have miscalculated VMs size, but otherwise
is good.
2017-07-29 05:13:46 +02:00
Marek Marczykowski-Górecki
7db2ed82ea
qvm-ls: fix total VM size reporting
There is no vm.storage object in qubesadmin module.
2017-07-29 04:56:20 +02:00
Marek Marczykowski-Górecki
18153652f3
tools: qvm-service tool
This really use features, but keep compatibility with Qubes 3.x

Fixes QubesOS/qubes-issues#1227
2017-07-28 14:24:04 +02:00
Marek Marczykowski-Górecki
416ee0bd22
tools/qvm-start-gui: add --force-stubdomain options
Sometimes it's useful to access emulated VGA, without rebooting the VM
in debug mode
2017-07-27 19:17:47 +02:00
Marek Marczykowski-Górecki
5b0c8e84d8
tools/qvm-shutdown: fix help message 2017-07-27 19:17:47 +02:00
Marek Marczykowski-Górecki
5d0bd4f6dc
tools/qvm-shutdown: drop --force option, it isn't supported anymore
Admin API does not allow this action.
2017-07-27 19:17:47 +02:00
Marek Marczykowski-Górecki
2d5d9d6d7d
tools: add qvm-backup tool
New qvm-backup tool can either use pre-existing backup profile
(--profile), or - when running in dom0 - can create new one based on
used options (--save-profile).

This commit add a tool itself, update its man page, and add tests for
it.

Fixes QubesOS/qubes-issues#2931
2017-07-21 03:58:18 +02:00
Marek Marczykowski-Górecki
d8af76ed60
backup: move BackupRestore class and helpers to 'restore' submodule
This breaks cyclic imports and also allow cleaner separation between
backup make and restore code.

No functional change.
2017-07-21 03:54:04 +02:00
Marek Marczykowski-Górecki
ea47701fe6
tools/qvm-backup-restore: fix restoring only specific VMs
Related to QubesOS/qubes-issues#2924
2017-07-21 03:54:04 +02:00
Marek Marczykowski-Górecki
c50fc21e44
tests: qvm-backup-restore tool tests
Very simple one, but also fix reporting errors detected by tests.
2017-07-21 03:54:04 +02:00
Marek Marczykowski-Górecki
3c9fb8dbac
tools/qvm-volume: fix default action handling 2017-07-21 03:10:02 +02:00
Marek Marczykowski-Górecki
ae0729fb53
storage: drop 'internal' volume property
Since external block devices use Devices API now, it isn't useful
anymore.

QubesOS/qubes-issues#2256
2017-07-21 03:10:01 +02:00
Marek Marczykowski-Górecki
c736395432
tools/qvm-backup-restore: fix default list of VMs to restore
By default restore all of them, not only the ones named "[" or "]"
(which are invalid name).

Fixes QubesOS/qubes-issues#2924
2017-07-19 23:52:11 +02:00
Marek Marczykowski-Górecki
756235d4f3
Change 'hvm' property to 'virt_mode'
QubesOS/qubes-issues#2912
2017-07-18 04:22:34 +02:00
Marek Marczykowski-Górecki
0c0b625a70
Merge branch 'devel-backup' 2017-07-18 01:49:37 +02:00
Marek Marczykowski-Górecki
f058c48c92
Merge branch 'devel-2-qvm-run-1'
* devel-2-qvm-run-1:
  Make pylint happy
  tools/qvm-run: fix handling EOF
  tests: mark qvm-run tests with "expected failure"
  tools/qvm-run: fix handling copying stdin to the process
2017-07-18 01:49:16 +02:00
Marek Marczykowski-Górecki
5178029a3c
Make pylint happy 2017-07-18 01:32:06 +02:00
Marek Marczykowski-Górecki
706cecd60d
tools/qvm-run: fix handling EOF 2017-07-18 01:12:43 +02:00
Marek Marczykowski-Górecki
e7ee06936a
tools/qvm-backup-restore: handle VMs selection, not only exclusion
QubesOS/qubes-issues#1214
2017-07-17 23:34:03 +02:00
Marek Marczykowski-Górecki
f0151d73b3
tools: add qvm-backup-restore
Frontend tool for backup restore code.

Fixes QubesOS/qubes-issues#1214
2017-07-17 20:28:22 +02:00
Marek Marczykowski-Górecki
941b553b81
tools/qvm-run: fix handling copying stdin to the process
Launch stdin copy loop in a separate process (multiprocessing.Process)
and terminate it when target process is terminated.
Another idea here was threads, but there is no API to kill a thread
waiting on read().
2017-07-08 00:11:15 +02:00
Marek Marczykowski-Górecki
389252f386
tools: handle shutdown order in qvm-shutdown
VMs can have runtime dependencies - for example it isn't possible to
shutdown netvm used by some other running VM(s). Since client-side tools
may not have full knowledge about rules enforcing those dependencies
(for example may not have access to 'netvm' property), implement
best-effort approach:
1. Try to shutdown all requested VMs
2. For those where shutdown request succeed, wait for actual shutdown
3. For others - go back to step 1

And loop until all VMs are shutdown, or all shutdown requests fails.
2017-07-08 00:08:19 +02:00
Marek Marczykowski-Górecki
2052b32202
events: simplify wait_for_domain_shutdown coroutine
1. Handle timeout externally - using asyncio.wait_for.
2. Add support for waiting for multiple VMs.
2017-07-06 22:01:17 +02:00
Marek Marczykowski-Górecki
d8f018eb95
tools: clarify help in qvm-template-postprocess 2017-07-05 14:16:31 +02:00
Marek Marczykowski-Górecki
7c9699cd87
tools: split calling qubes.PostInstall service to a separate function
The code is long enough to warrant separate function (suggested by
pylint).
2017-07-05 14:16:31 +02:00
Marek Marczykowski-Górecki
627aebf2cd
tools: remove policy handling from qvm-firewall tool
Follow the API removal

QubesOS/qubes-issues#2869
2017-07-05 14:16:30 +02:00
Marek Marczykowski-Górecki
5430e04e1c
tools: move event loop creation/closing to main function
Do not close event loop in utility function - handle it only in main().
For this reason, change appropriate functions to coroutines.

Fixes QubesOS/qubes-issues#2865
2017-06-25 20:09:10 +02:00
Marek Marczykowski-Górecki
0012eb3ac6
tools/qvm-template-postprocess: improve error handling
Don't fail the whole process when "just" appmenus import fails.
But if data import fails, remove the VM

Also update for vm.run_service_for_stdio raising CalledProcessError.
2017-06-25 18:22:06 +02:00
Marek Marczykowski-Górecki
3cf5840d7a
Merge branch 'devel-4'
* devel-4:
  tools/qvm-start-gui: multiple fixes
  vm: raise CalledProcessError instead of QubesVMError on failed service call
  events: improve handling qubesd restart
2017-06-25 13:16:50 +02:00
Marek Marczykowski-Górecki
cef80a76e4
Merge branch 'devel-3'
* devel-3:
  Implement VM clone as create + copy data+metadata
  storage: make Volumes sortable
  base: add PropertyHolder.clone_properties
  doc: minor fixes to man pages
  storage: add volume clone method
  doc: fix skel-manpage tool
  tools: add qvm-tags tool
  tags support
2017-06-25 13:16:22 +02:00
Marek Marczykowski-Górecki
64377207a8
tools/qvm-start-gui: multiple fixes
Don't start GUI daemon for given VM twice when qvm-start-gui was started
during VM startup (while waiting for qrexec startup). This is especially
common while running tests.

Report failed qubes.SetMonitorLayout as warning (instead of unhandled
exception).

Clear VM cache on qubesd reconnect.

Fix logging.
2017-06-21 06:01:53 +02:00