Commit Graph

5379 Commits

Author SHA1 Message Date
Rusty Bird
60bf68a748
storage/reflink: add _path_import (don't reuse _path_dirty)
Import volume data to a new _path_import (instead of _path_dirty) before
committing to _path_clean. In case the computer crashes while an import
operation is running, the partially written file should not be attached
to Xen on the next volume startup.

Use <name>-import.img as the filename like 'file' does, to be compatible
with qubes.tests.api_admin/TC_00_VMs/test_510_vm_volume_import.
2018-09-09 20:01:18 +00:00
Rusty Bird
d301aa2e50
storage/reflink: delete stale tempfiles on start and remove
When the AT_REPLACE flag for linkat() finally lands in the Linux kernel,
_replace_file() can be modified to use unnamed (O_TMPFILE) tempfiles.
Until then, make sure stale tempfiles from previous crashes can't hang
around for too long.
2018-09-09 20:01:17 +00:00
Rusty Bird
75a4a1340e
storage/reflink: don't recompute static properties per call 2018-09-09 20:01:15 +00:00
Rusty Bird
ef2698adb4
storage/reflink: make revisions() more readable, use iglob 2018-09-09 20:01:14 +00:00
Rusty Bird
18f9356c2c
storage/reflink: refuse to revert() dirty volume 2018-09-09 20:01:13 +00:00
Rusty Bird
677183d8a6
storage/reflink: add revision even if empty
It's sort of useful to be able to revert a volume that has only ever
been started once to its empty state. And the lvm_thin driver allows it
too, so why not.
2018-09-09 20:01:12 +00:00
Rusty Bird
850778b52a
storage/reflink: remove redundant format specifiers 2018-09-09 20:01:11 +00:00
Marek Marczykowski-Górecki
a371375f1d
version 4.0.29 2018-09-09 06:38:16 +02:00
Marek Marczykowski-Górecki
b2cc605f4b
tests: clean local variables from traceback objects
System tests are fragile for any object leaks, especially those holding
open files. Instead of wrapping all tests with try/finally removing
those local variables (as done in qubes.tests.integ.backup for example),
apply generic solution: clean all traceback objects from local
variables. Those aren't used to generate text report by either test
runner (qubes.tests.run and nose2). If one wants to break into debugger
and inspect tracebacks interactively, needs to comment out call to
cleanup_traceback.
2018-09-09 02:43:17 +02:00
Marek Marczykowski-Górecki
c102fa3d68
tests: add basic audio play/rec tests
QubesOS/qubes-issues#4204
2018-09-08 04:13:24 +02:00
Marek Marczykowski-Górecki
e8d2c4e232
Merge remote-tracking branch 'qubesos/pr/213'
* qubesos/pr/213:
  Fix from Marek's revies
  doc: loading graph
2018-09-07 21:35:44 +02:00
Wojtek Porczyk
b200dc8df2
Fix from Marek's revies
- add load_extras()
- change callouts to arrows for draw.io limitations

Cc: @marmarek
2018-09-07 19:44:29 +02:00
Marek Marczykowski-Górecki
8ce3433406
tests: drop sudo in tests already running as root
Don't spam already trashed log.
2018-09-07 15:12:05 +02:00
Marek Marczykowski-Górecki
1b99bd9ab3
Merge branch 'tests-dispvm-fix'
* tests-dispvm-fix:
  tests: fix DispVM related tests
2018-09-07 01:57:19 +02:00
Marek Marczykowski-Górecki
6a3ecd0170
Merge remote-tracking branch 'qubesos/pr/229'
* qubesos/pr/229:
  Order qubesd before systemd-user-sessions
2018-09-07 01:07:27 +02:00
Marek Marczykowski-Górecki
7f1e2741ec
Merge remote-tracking branch 'qubesos/pr/228'
* qubesos/pr/228:
  storage/lvm: filter out warning about intended over-provisioning
  tests: fix getting kernel package version inside VM
  tests/extra: add start_guid option to VMWrapper
  vm/qubesvm: fire 'domain-start-failed' event even if fail was early
  vm/qubesvm: check if all required devices are available before start
  storage/lvm: fix reporting lvm command error
  storage/lvm: save pool's revision_to_keep property
2018-09-07 01:06:59 +02:00
Rusty Bird
d3a5799245
Order qubesd before systemd-user-sessions
qubes-vm@.service would already cause this ordering, but not every user
has any autostart=True VMs.

Also needed to maybe f*x QubesOS/qubes-issues#3149 at some point.
2018-09-06 16:23:24 +00:00
Marek Marczykowski-Górecki
e1378f70fb
doc: fix libvirt config path 2018-09-06 01:16:50 +02:00
Marek Marczykowski-Górecki
ee25f7c7bb
vm: fix error reporting on PVH without kernel set
Fixes QubesOS/qubes-issues#4254
2018-09-03 00:23:05 +02:00
Marek Marczykowski-Górecki
62ac42f06a
Merge remote-tracking branch 'qubesos/pr/220'
* qubesos/pr/220:
  code style fixes
  Add domain-paused/-unpaused events
2018-09-02 05:34:24 +02:00
Marek Marczykowski-Górecki
895c8d2414
Merge remote-tracking branch 'qubesos/pr/225'
* qubesos/pr/225:
  tools/qvm-sync-clock: don't start clockvm
2018-09-02 03:38:10 +02:00
Marek Marczykowski-Górecki
b72b81fb9e
Merge remote-tracking branch 'qubesos/pr/224'
* qubesos/pr/224:
  Fix error on non ASCII PCI IDs upon qvm-device list

Fixed code style.
2018-09-02 03:36:00 +02:00
Marek Marczykowski-Górecki
57c9b2edf7
code style fixes 2018-09-02 03:27:14 +02:00
Marek Marczykowski-Górecki
890df9ba03
qubespolicy: ease testing by calling str(target) only once
Don't call it multiple times depending on number of registered loggers.
2018-09-01 19:51:03 +02:00
Rusty Bird
8b2b26134e
tools/qvm-sync-clock: don't start clockvm
Since bda9264, a qubes.GetDate call from a VM will not cause clockvm
startup. Also avoid causing it with /etc/cron.d/qubes-sync-clock.cron.

Fixes QubesOS/qubes-issues#3588
2018-08-27 23:31:16 +00:00
Galland
6f04c8d65b
Fix error on non ASCII PCI IDs upon qvm-device list
solves https://github.com/QubesOS/qubes-issues/issues/4229
2018-08-21 03:08:17 +02:00
Patrick Schleizer
fbdf460db8
comments 2018-08-08 09:38:45 +00:00
Marek Marczykowski-Górecki
a121ecfeb4
Merge remote-tracking branch 'qubesos/pr/222'
* qubesos/pr/222:
  add Whonix defaults
2018-08-07 20:47:51 +02:00
Patrick Schleizer
57424ef50b
add Whonix defaults 2018-08-08 01:31:32 +08:00
Patrick Schleizer
65341a3468
$tag:anon-vm $anyvm deny 2018-08-08 00:29:07 +08:00
Jean-Philippe Ouellet
e95ef5f61d
Add domain-paused/-unpaused events
Needed for event-driven domains-tray UI updating and anti-GUI-DoS
usability improvements.

Catches errors from event handlers to protect libvirt, and logs to
main qubesd logger singleton (by default meaning systemd journal).
2018-08-01 05:41:50 -04:00
Marek Marczykowski-Górecki
ecaee10f80
Merge remote-tracking branch 'qubesos/pr/219'
* qubesos/pr/219:
  remove Whonix specific exceptions
2018-07-20 19:38:22 +02:00
Patrick Schleizer
6f311ee1ff
remove Whonix specific exceptions
Whonix will use `qvm-open-in-dvm` so no more exceptions required.
2018-07-20 23:39:00 +08:00
Patrick Schleizer
b6a015b329
remove Whonix specific exceptions
Whonix will use `qvm-open-in-dvm` so no more exceptions required.
2018-07-20 23:38:59 +08:00
Marek Marczykowski-Górecki
e7f84dedd1
version 4.0.28 2018-07-19 15:21:21 +02:00
Marek Marczykowski-Górecki
0e089ca38d
ext/windows: copy private.img on windows TemplateBasedVM creation
This is a workaround for missing private.img initialization in Qubes
Windows Tools.

QubesOS/qubes-issues#3585
2018-07-17 05:25:31 +02:00
Marek Marczykowski-Górecki
e6edbabf94
tests: exclude windows templates from linux tests 2018-07-16 22:02:59 +02:00
Marek Marczykowski-Górecki
e51efcf980
vm: document domain-start-failed event 2018-07-16 22:02:59 +02:00
Marek Marczykowski-Górecki
af2435c0d4
Make some properties default to template's value (if any)
Multiple properties are related to system installed inside the VM, so it
makes sense to have them the same for all the VMs based on the same
template. Modify default value getter to first try get the value from a
template (if any) and only if it fails, fallback to original default
value.
This change is made to those properties:
 - default_user (it was already this way)
 - kernel
 - kernelopts
 - maxmem
 - memory
 - qrexec_timeout
 - vcpus
 - virt_mode

This is especially useful for manually installed templates (like
Windows).

Related to QubesOS/qubes-issues#3585
2018-07-16 22:02:58 +02:00
Marek Marczykowski-Górecki
af7d54d388
Update windows-related feature requests
Handle 'os' feature - if it's Windows, then set rpc-clipboard feature.
Handle 'gui-emulated' feature - request for specifically stubdomain GUI.
With 'gui' feature it is only possible to enable gui-agent based on, or
disable GUI completely.
Handle 'default-user' - verify it for weird characters and set
'default_user' property (if wasn't already set).

QubesOS/qubes-issues#3585
2018-07-16 22:02:58 +02:00
Marek Marczykowski-Górecki
ce87451c73
Merge branch 'lvm-snapshots'
* lvm-snapshots:
  tests: fix handling app.pools iteration
  storage/lvm: add repr(ThinPool) for more meaningful test reports
  tests: adjust for variable volume path
  api/admin: expose volume path in admin.vm.volume.Info
  tests: LVM: import, list_volumes, volatile volume, snapshot volume
  tests: collect all SIGCHLD before cleaning event loop
  storage/lvm: use temporary volume for data import
  tests: ThinVolume.revert()
  tests: LVM volume naming migration, and new naming in general
  storage/lvm: improve handling interrupted commit
2018-07-16 16:40:35 +02:00
Marek Marczykowski-Górecki
be2465c1f9
Fix issues found by pylint 2.0
Resolve:
 - no-else-return
 - useless-object-inheritance
 - useless-return
 - consider-using-set-comprehension
 - consider-using-in
 - logging-not-lazy

Ignore:
 - not-an-iterable - false possitives for asyncio coroutines

Ignore all the above in qubespolicy/__init__.py, as the file will be
moved to separate repository (core-qrexec) - it already has a copy
there, don't desynchronize them.
2018-07-15 23:51:15 +02:00
Marek Marczykowski-Górecki
c688641363
tests: fix DispVM related tests
- fix regex for editor window search - 'disp*' matches 'disk_space.py'
(a dom0 local widget...)
- increase a timeout for automatic DispVM cleanup
2018-07-15 22:29:22 +02:00
Marek Marczykowski-Górecki
f8d17012c3
tests: fix loading tests for templates
Fix #209
2018-07-15 21:57:34 +02:00
Marek Marczykowski-Górecki
69e3018b94
tests: fix handling app.pools iteration 2018-07-15 21:31:48 +02:00
Marek Marczykowski-Górecki
2af1815ab7
storage/lvm: add repr(ThinPool) for more meaningful test reports 2018-07-15 21:30:04 +02:00
Marek Marczykowski-Górecki
e644378f18
tests: adjust for variable volume path
LVM volumes now have variable volume path. Compare strip path before
comparing content's hash in tests.
2018-07-12 16:52:48 +02:00
Marek Marczykowski-Górecki
d211a2771a
api/admin: expose volume path in admin.vm.volume.Info
Since (for LVM at least) path is dynamic now, add information about it
to volume info. This is not very useful outside of dom0, but in dom0 it
can be very useful for various scripts.
This will disclose current volume revision id, but it is already
possible to deduce it from snapshots list.
2018-07-12 16:52:48 +02:00
Marek Marczykowski-Górecki
4282a41fcb
tests: LVM: import, list_volumes, volatile volume, snapshot volume 2018-07-12 16:52:47 +02:00
Marek Marczykowski-Górecki
76c872a43a
tests: collect all SIGCHLD before cleaning event loop
On python 3.6.4 apparently it requires two callbacks runs to cleanup
stale SIGCHLD handlers.
2018-07-12 16:52:47 +02:00