Commit Graph

447 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
48ad8325d0
tests: check rejecting/accepting compression filters
3 new tests:
- custom (common) compression filter
- custom (uncommon) compression filter - should be rejected
- custom (uncommon) compression filter forced - accepted
2019-09-10 15:24:15 +02:00
Marek Marczykowski-Górecki
14f77860bf
backup/restore: add option to use uncommon compression filter anyway
Previous commit introduced protection against uncommon (potentially
malicious) compression filters. This breaks restoring backups made with
a custom compression filter. Add an option to override this check, by
naming compression filter to use explicitly.
2019-09-10 15:24:15 +02:00
Marek Marczykowski-Górecki
10f15e6669
backup/restore: reject uncommon compression filters and improve header validation
Compression filter named in a backup header is executed in restore
environment (commonly dom0). While this field is properly authenticated,
there may be cases where backup archive comes from less
trusted source, like migrating from potentially compromised
system.

Modify backup header parsing code to add field specific validators.
Whitelist only know crypto, hmac and compression algorithms.

Based on a patch by Jean-Philippe Ouellet <jpo@vt.edu>
Reported-by: Jean-Philippe Ouellet <jpo@vt.edu>
2019-09-10 15:24:15 +02:00
Frédéric Pierret (fepitre)
3ddeb2046a
dochelpers: make PEP8 happier 2019-09-06 16:29:56 +02:00
Frédéric Pierret (fepitre)
6df676c259
dochelpers: handle legacy sphinx library 2019-09-06 16:29:52 +02:00
Frédéric Pierret (fepitre)
ab8f487b50
Fix Sphinx 2 new API for Fedora 31+
QubesOS/qubes-issues#5289
2019-09-06 16:24:30 +02:00
Frédéric Pierret (fepitre)
8639034bb6
qvm-device: prevent parser allowing abbreviations 2019-08-24 13:39:37 +02:00
Frédéric Pierret (fepitre)
41c6fbed72
qvm-device: handle 'list-device-classes' and 'list-classes'
QubesOS/qubes-issues#5213
2019-08-24 13:39:36 +02:00
Frédéric Pierret (fepitre)
3dce4e9742
qubesadmin: define methods list_vmclass and list_devicesclass
Adapt also previous direct calls of qubesdb

QubesOS/qubes-issues#5213
2019-08-10 22:08:31 +02:00
Frédéric Pierret (fepitre)
a982e1e538
qubesadmin: make PEP8 happy 2019-08-10 19:03:41 +02:00
Marek Marczykowski-Górecki
769f8a5ee8
Merge remote-tracking branch 'origin/pr/98'
* origin/pr/98:
  qvm-check: fix from Marek's comment
  qvm-check: refactor check mechanism and add filter for checking netvm
2019-08-08 15:35:34 +02:00
Frédéric Pierret (fepitre)
eb461e4c3b
qvm-check: fix from Marek's comment 2019-08-08 14:29:35 +02:00
Frédéric Pierret (fepitre)
7d93377b78
qvm-check: refactor check mechanism and add filter for checking netvm
Fix QubesOS/qubes-issues#3496
2019-08-08 14:26:05 +02:00
Marek Marczykowski-Górecki
489efce9cb
Merge remote-tracking branch 'origin/pr/99'
* origin/pr/99:
  devices: add missing docstring for _get_device_classes
  devices: make iteration device classes compatible with Python2
  tools/qvm-device: make PEP8 happy
  tests/devices: add test for handling listing device classes
  tests/devices: make PEP8 happy
  devices: handle listing of available device classes
  devices: make PEP8 happy
2019-08-08 14:13:38 +02:00
Frédéric Pierret (fepitre)
08d746e119
devices: add missing docstring for _get_device_classes 2019-08-07 22:10:37 +02:00
Frédéric Pierret (fepitre)
268d11fbff
devices: make iteration device classes compatible with Python2 2019-08-07 20:38:30 +02:00
Frédéric Pierret (fepitre)
174f155d31
tools/qvm-device: make PEP8 happy 2019-08-06 15:15:12 +02:00
Frédéric Pierret (fepitre)
4d0545405f
tests/devices: add test for handling listing device classes 2019-08-06 15:15:12 +02:00
Frédéric Pierret (fepitre)
07ca511bd7
tests/devices: make PEP8 happy 2019-08-06 15:15:03 +02:00
Frédéric Pierret (fepitre)
f633dd9b40
devices: handle listing of available device classes
QubesOS/qubes-issues#5213
2019-08-06 15:09:44 +02:00
Frédéric Pierret (fepitre)
1052217973
devices: make PEP8 happy 2019-08-06 15:07:33 +02:00
Marek Marczykowski-Górecki
e700af9eb2
tools/qvm-volume: add 'import' action
Add support for importing volume data with qvm-volume tool.
This could be also used to clear volume by issuing:

    qvm-volume import --no-resize some-vm:private /dev/null

QubesOS/qubes-issues#5192
2019-07-29 22:20:06 +02:00
Marek Marczykowski-Górecki
fdc632c959
tools: reset private volume when importing template over existing one
Reinstalling template is a recommended way to get it back to a clean
state after potential compromise. In that case it is essential to
discard any persistent storage of old template, as it could be used by
the attacker to re-compromise it after reinstall.
Do this similar as root volume is overridden - via volume import
function.

Fixes QubesOS/qubes-issues#5192
2019-07-29 22:20:06 +02:00
Marek Marczykowski-Górecki
21569b3a31
tests: fix events tests on python3.7
Fix syntax workaround for python3.7 + python2.7. This code can't use
'yield from' to be still importable on python2.7, but asyncio.sleep() is
no longer iterable on python3.7. Workaround it by manually calling
coroutine.send(None) in a loop - ugly as hell, but works. I can't wait
until we could drop python2 support...
2019-07-29 21:56:32 +02:00
Marek Marczykowski-Górecki
b8ddb39752
Allow app.domains[vm] where vm is a QubesVM object
Similar as in the core-admin API.
2019-06-25 06:28:00 +02:00
Malte Leip
32538fa5ec
qvm-prefs: add --hide-default option 2019-05-29 20:32:13 +02:00
Marek Marczykowski-Górecki
c1be4cd062
Merge remote-tracking branch 'qubesos/pr/92'
* qubesos/pr/92:
  Adds docs for clone ignore_errors, feeds linter
  Adds support for `--ignore-errors` to qvm-clone CLI
2019-04-17 00:30:10 +02:00
Marek Marczykowski-Górecki
37e4e4c6ca
tests: update yaml syntax in qvm-backup tests
yaml.safe_dump() finally produces canonical yaml output, not a
one-liner.
2019-04-16 23:10:38 +02:00
Conor Schaefer
40eeddbfc8
Adds docs for clone ignore_errors, feeds linter
Addresses review comments, specifically:

  * updates qvm-clone tests to handle ignore_errors option
  * adds manpage reference to new --ignore-errors option
  * trims line length in qvm-clone changes

All tests should now be passing, including the new qvm-clone
functionality.
2019-04-16 10:44:47 -07:00
Conor Schaefer
55a22e7955
Adds support for --ignore-errors to qvm-clone CLI
The underlying API already supported the `ignore_errors` param in
Python, now we just expose that option to the `qvm-clone` CLI.
2019-04-15 17:33:41 -07:00
Marek Marczykowski-Górecki
d7430d42ce
Make pylint happy
no-else-raise warning
2019-03-07 03:17:29 +01:00
Marek Marczykowski-Górecki
05f0d4ca0f
Make vm.run_with_args working on python2
This part of the qubesadmin module still needs to be compatible with
python2.
2019-03-07 03:05:09 +01:00
Marek Marczykowski-Górecki
fb910a71cf
tools/qvm-start: validate output of losetup command
QubesOS/qubes-issues#4860
2019-03-07 02:53:52 +01:00
Marek Marczykowski-Górecki
a2629b1239
tools/qvm-start: use vm.run_with_args to call losetup in the VM
This will fix handling filenames with spaces and shell special
characters.

Reported by @v6ak

Fixes QubesOS/qubes-issues#4860
2019-03-07 02:53:45 +01:00
Marek Marczykowski-Górecki
a35cb46342
tests: vm.run, vm.run_with_args
QubesOS/qubes-issues#4850
2019-02-28 05:34:42 +01:00
Marek Marczykowski-Górecki
0092c0ac99
vm: add run_with_args method to call a single command with specific arguments
The method will ensure the parameters are properly handled as actual
separate arguments, regardless of shell special characters in them.
For now implement this with shlex.quote, later dedicated qrexec service
could be used.

QubesOS/qubes-issues#4850
2019-02-28 04:37:59 +01:00
Marek Marczykowski-Górecki
88ab55b940
events: simplify handling qubesd reconnect
Retry connection on any OSError, instead of only few selected error
types. For example PermissionError may happen too (when socket is
created, but before it gets appropriate mode), which wasn't listed.
2019-02-24 05:32:51 +01:00
Marek Marczykowski-Górecki
1145f70334
tools/qvm-backup: allow to disable compression
Specify compression explicitly in the backup profile, not only when
requested with --compress or --compress-filter.
This will allow to disable compression with --no-compress option, as the
default if no compression is specified in the profile is to use gzip.

Fixes QubesOS/qubes-issues#4803
2019-02-24 05:32:46 +01:00
Marek Marczykowski-Górecki
5e4831ede4
backup: Do not try to attach devices during backup verification
The "restored" VM object will be discarded anyway, and it doesn't have
real 'attach' method.

Fixes QubesOS/qubes-issues#4830
2019-02-24 02:26:05 +01:00
Marek Marczykowski-Górecki
2ad9b57961
utils: include management_dispvm property when checking where VM is used
Fixes QubesOS/qubes-issues#4812
2019-02-24 02:26:05 +01:00
Marek Marczykowski-Górecki
b1a191446a
tools/qvm-run: do not color the output unless --pass-io is used
Since no output from VM is passed (and even if it would, it's redirected
to /dev/null), there is no need to switch output color.
This fixes the case when qvm-run is started in background - the color
change would affect further shell output.

Fixes QubesOS/qubes-issues#4808
2019-02-11 19:23:35 +01:00
Marek Marczykowski-Górecki
89ee4931fb
backup: do not fail restore if scrypt anticipate more time needed
For various reasons, decryption may be slower than initial encryption
(different hardware, different system load etc). Do not fail the restore
operation if scrypt anticipate it will take considerably more time or
memory, than while creating the backup (where the scrypt parameters were
originally set).

Thanks to @jharveyb for the report and suggested solution.
Fixes QubesOS/qubes-issues#4683
2019-01-09 19:09:44 +01:00
Marek Marczykowski-Górecki
a65ef425e1
tools/qvm-run: make -q affect 'command failed with code' message
Fixes QubesOS/qubes-issues#4616
2018-12-16 05:51:41 +01:00
Marek Marczykowski-Górecki
9061169f90
Merge branch 'devel-20181206' 2018-12-09 18:08:25 +01:00
Marek Marczykowski-Górecki
0bb35193b4
doc: fix rst syntax in documentation, including docstrings 2018-12-08 23:53:55 +01:00
Marek Marczykowski-Górecki
558c5d4a7c
Merge remote-tracking branch 'origin/pr/84'
* origin/pr/84:
  Fixed typo in documentation
  Documentation is important! Added shortcuts to manpages
  added tests for shortcuts
  diff doesn't set template
  added --standalone and --disp shortcuts
2018-12-08 15:10:42 +01:00
GammaSQ
b848625428
added tests for shortcuts 2018-12-08 12:38:26 +01:00
GammaSQ
ce2fb96a4b
diff doesn't set template 2018-12-08 12:29:48 +01:00
Marek Marczykowski-Górecki
4a727f1dfa
Merge remote-tracking branch 'origin/pr/85'
* origin/pr/85:
  isinstance instead of type
  reverted to comparing .index
  forgot braces
  forgot braces
  get_label now accepts integer and has correct return-type
2018-12-08 12:28:36 +01:00
GammaSQ
2c836c5adc
isinstance instead of type 2018-12-08 12:00:15 +01:00
Marek Marczykowski-Górecki
415f5faae9
tools/qvm-run: factor out actual process running into separate function
Pylint complains that main() is too long and that's indeed true. Factor
out single process call (together with all the prepartion) into separate
function and keep only common code in main().
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
5fe6ffc04c
tools/qvm-create: reject --root-{move,copy}-from with template-based qubes
Fixes QubesOS/qubes-issues#4424
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
f03937c51e
tools: remove extra debug 2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
e827e47926
Clone VM's volume into the same pool, unless overridden specifically
When cloning VM, create it in the same pool as the source one.
Previously it always used default pool, which means for example renaming
a VM in non-default pool moved it back to the default one.

Fixes QubesOS/qubes-issues#4145
Fixes QubesOS/qubes-issues#4523
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
bee55a3bce
tools/qvm-device: allow detaching all devices
QubesOS/qubes-issues#4530
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
954ffc4bf2
tools/qvm-run: add error message if command in VM failed
Print some error even without --pass-io, otherwise the only way to learn
the failure is checking $?, as no other visual sign is there.

Fixes QubesOS/qubes-issues#4533
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
4b00ef7ec7
tools/qvm-run: fix error reporting on qvm-run -n on halted qube
Fixes QubesOS/qubes-issues#4476
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
d34b1bfc4f
tools/qvm-run: factor out print_no_color function
Will be useful for other messages that should not be in red.
2018-12-08 00:44:18 +01:00
Marek Marczykowski-Górecki
32cbc59ba9
tools/qvm-run: handle Ctrl+C nicely
Do not exit with ugly python backtrace, simply interrupt the command
(propagate SIGINT) and exit.

QubesOS/qubes-issues#4532
2018-12-08 00:44:17 +01:00
Marek Marczykowski-Górecki
9acce13a35
tools: fix qvm-run --pass-io --localcmd=... vmname command
qubes.VMShell service, used by qvm-run, expects the command on the first
input line. Previously, when --localcmd was used, the command wasn't
written anywhere and the local command was connected directly to
qubes.VMShell service. And the first line of its output was interpreted
as a command.

Fix this by starting the local command separately, after sending the
command to qubes.VMShell service.

While at it, unify handling shell command and service calls in the process.
vm.run_service(..., localcmd= ) isn't that useful in general case,
because for qubes.VMShell the caller first need to send the command
before starting local process. Since the qvm-run tool needs to implement
manual starting localcmd anyway, don't use localcmd= run_service's
argument at all to unify calling methods.

There is slight behavior change: previously localcmd was started only
after establishing service connection (for example only if qrexec policy
allows), now it is started in all the cases.

Fixes QubesOS/qubes-issues#4040
2018-12-08 00:44:02 +01:00
Marek Marczykowski-Górecki
e6202d496d
tests/tools: improve qvm-run tests
Fix most FD/process leaks, make qvm-run --passio tests working (as much
as possible).
2018-12-07 23:55:03 +01:00
Marek Marczykowski-Górecki
1d877742ae
tests/tools: extend vm.run_service() mockup
Handle stdout/stderr arguments and provide file-like objects when
requested.
2018-12-07 04:25:00 +01:00
GammaSQ
aebc944704
reverted to comparing .index 2018-12-04 09:40:54 +01:00
Marek Marczykowski-Górecki
28067f70ac
backup: simplify test expressions to make pylint happy 2018-12-03 23:27:10 +01:00
Marek Marczykowski-Górecki
377c80d33d
Remove useless 'pass' to make pylint happy 2018-12-03 23:22:50 +01:00
Marek Marczykowski-Górecki
86fe230092
Clarify QubesBase(), Qubes() and QubesLocal/QubesRemote usage
Add note in QubesBase docstring it shouldn't be used directly.
Additionally add base qubesd_call and run_service methods raising
NotImplementedError with helpful message. Lack of qubesd_call in
QubesBase leads to infinite recursion, because one in PropertyHolder
calls itself then.

Fixes QubesOS/qubes-issues#4568
2018-12-03 23:09:23 +01:00
GammaSQ
39c2c7bcd2
forgot braces 2018-12-03 15:55:21 +01:00
GammaSQ
2e637b5631
get_label now accepts integer and has correct return-type 2018-12-03 15:29:06 +01:00
GammaSQ
e55b530865
added --standalone and --disp shortcuts 2018-12-03 09:21:01 +01:00
Marek Marczykowski-Górecki
861e4fb04f
tools: clarify qvm-service --help 2018-10-29 05:27:05 +01:00
Marek Marczykowski-Górecki
3d53e7e310
Code style fix 2018-10-29 05:27:05 +01:00
Marek Marczykowski-Górecki
67897e3f9f
Copy application menu on VM clone
The qubesd daemon have no information about clone source - from that
side it looks like a new VM.  This means application menu is created as
for a new VM.
To fix this re-initialize menu with --source option as part of the clone
operation. It will copy both list of available applications (if
applicable) and selected applications.

This fixes both qvm-clone case and rename.

Fixes QubesOS/qubes-issues#3902
Fixes QubesOS/qubes-issues#4124
2018-10-29 05:27:05 +01:00
Marek Marczykowski-Górecki
5078d75aa3
tools/qvm-create: fix handling invalid label 2018-10-18 03:24:24 +02:00
Marek Marczykowski-Górecki
759fafea63
tools/qvm-create: properly create template-based StandaloneVM
By definition StandaloneVM is not linked to the template. Creating one
from a template is a clone operation. It's already possible using
qvm-clone tool, but it's logical to do that using qvm-create tool too.
This was the case in R3.2 too.

While adding this special case, skip cloning private volume, to preserve
behaviour of TemplateBaseVMs which do not inherit private volume either.

Fixes QubesOS/qubes-issues#3793
2018-10-18 03:24:15 +02:00
Marek Marczykowski-Górecki
4ca6c32e6c
app: add option to ignore select volumes on app.clone_vm()
QubesOS/qubes-issues#3793
2018-10-18 02:43:09 +02:00
Patrik Hagara
4cd513757b
qvm-ls: add filtering by domain power state 2018-10-10 20:06:35 +02:00
Marek Marczykowski-Górecki
e7bc8f21d5
events: try reconnecting to qubesd also on BrokenPipeError
When qubesd exits while writing to it, not reading, the error is
BrokenPipe, not EOF. Handle it the same.
2018-10-07 02:46:28 +02:00
Marek Marczykowski-Górecki
0dfdab32e1
Merge branch 'devices-api'
* devices-api:
  devices: include devclass when comparing devices
  events: deserialize DeviceInfo class in device-* events
  devices: drop DeviceInfo.options
2018-10-07 02:46:14 +02:00
Marek Marczykowski-Górecki
5414739272
devices: include devclass when comparing devices 2018-09-29 14:13:06 +02:00
Marek Marczykowski-Górecki
907c8da005
Merge remote-tracking branch 'origin/pr/77'
* origin/pr/77:
  Exclude installed_by_rpm when Restoring VM from Backup
2018-09-27 14:57:08 +02:00
Marek Marczykowski-Górecki
5b02520ea4
Merge remote-tracking branch 'origin/pr/76'
* origin/pr/76:
  Fix typo
2018-09-27 14:56:25 +02:00
Marek Marczykowski-Górecki
4a1e90392b
events: deserialize DeviceInfo class in device-* events 2018-09-27 14:52:53 +02:00
jimtahu
5f868b4794
Exclude installed_by_rpm when Restoring VM from Backup
VMs listed as from an rpm when they were really from a backup
lead to a missleading error message when deleting.

Fixes QubesOS/qubes-issues#4192
2018-09-22 22:39:22 -05:00
Marek Marczykowski-Górecki
a8c24bee0f
devices: drop DeviceInfo.options
It's part of DeviceAssignment class, here it was leftover from draft
API.
2018-09-18 21:54:51 +02:00
Marek Marczykowski-Górecki
bf240cc928
devices: add DeviceInfo.devclass attribute
Have DeviceInfo also carry information about its class, to avoid the
need to iterate all the devices to get that information.
2018-09-18 16:50:00 +02:00
AJ Jordan
8b38a36de2
Fix typo 2018-09-14 14:48:22 -04:00
Marek Marczykowski-Górecki
0a8c6e2481
events: fix reconnecting to qubesd on its restart
Since now event listener reports proper QubesDaemonCommunicationError
exception instead of some form of IOError. Include it for automatic
reconnect logic.

Fixes a481490 "app: fix error reporting when connection to qubesd fails"
2018-09-13 14:29:27 +02:00
Marek Marczykowski-Górecki
fe73313da1
events: add support for wildcard event handlers
Port 5a39e777089d8bde6d0a620830a898c1cf3dd924 ("events: add support for
wildcard event handlers") from qubes-core-admin:

    Support registering handlers for more flexible wildcard events: not only
    '*', but also 'something*'. This allows to register handlers for
    'property-set:*' and such.
2018-09-07 18:59:05 +02:00
Marek Marczykowski-Górecki
045bad13e7
tools/qvm-create: resize root volume if needed before imporing data
If file to be imported is larger than the default root volume, resize
the volume first. It might be also a good idea to shrink it when needed,
but currently the backend refuse it.

Fixes QubesOS/qubes-issues#3422
2018-09-05 04:29:36 +02:00
Patrik Hagara
cd4424235b
qvm-ls: add filtering by tags 2018-08-26 22:08:32 +02:00
Marek Marczykowski-Górecki
584bd052ed
Adjust shell input for Windows shell, for qvm-run tool
Windows shell (cmd.exe) use '&' as a separator for multiple commands in
the same line.

Fixes QubesOS/qubes-issues#4165
2018-07-30 18:31:04 +02:00
Marta Marczykowska-Górecka
47b4e86736
Added test for qvm-remove dependency reporting 2018-07-20 01:35:00 +02:00
Marta Marczykowska-Górecka
ad2a6e3408
Better information on error in qvm-remove
If qvm-remove fails because the VM is in use, it will display
information about where it is used.

fixes QubesOS/qubes-issues#3193
2018-07-20 00:21:47 +02:00
Marta Marczykowska-Górecka
ca848ca7bd
Name change of vm dependencies helper function
As per @marmarek's request. Also docstrings.
2018-07-19 21:24:28 +02:00
Marta Marczykowska-Górecka
f9564ffa2b
Added tests and minor correction to vm_usage helper function
Now the function returns a list of tuples, not a list of lists.
2018-07-19 19:52:22 +02:00
Marta Marczykowska-Górecka
e6427f97dc
Helper function that lists where a given VM is used
A helper function to list vm's usage added to the utils.
2018-07-18 23:50:54 +02:00
Marek Marczykowski-Górecki
55c9a82c0a
Merge branch 'backup-logging-deadlock'
* backup-logging-deadlock:
  backup: avoid deadlock on logging during restore
2018-07-17 15:46:17 +02:00
Marek Marczykowski-Górecki
a7bfc03509
backup: avoid deadlock on logging during restore
When both threads and processes are used, logging module can deadlock.
Workaround this by re-initializating locks in logging module.
See https://bugs.python.org/issue6721 for details.
Revert this commit when the python issue is fixed (in the python version
used for backup restore).
2018-07-16 22:30:35 +02:00
Marek Marczykowski-Górecki
87122e54c9
Merge branch 'windows-tools'
* windows-tools:
  doc: add info what properties are inherited from template
  Add 'gui-emulated' feature
  qvm-start-gui: fix handlign rpc-clipboard feature
2018-07-16 22:06:44 +02:00
Marek Marczykowski-Górecki
4d61407f5d
Add 'gui-emulated' feature
Add an explicit method for forcing emulated VGA output. Previously it
was possible only by removing `gui` feature (setting it to false had a
different effect), or enabling debug mode.
Using lack of a feature as a third state was a bad idea.

QubesOS/qubes-issues#3585
2018-07-16 04:25:56 +02:00