Commit Graph

5695 Commits

Author SHA1 Message Date
Rusty Bird
d181bf1aa4
storage: factor out _wait_and_reraise(); fix clone/create
_wait_and_reraise() is similar to asyncio.gather(), but it preserves the
current behavior of waiting for all futures and only _then_ reraising
the first exception (if there is any) in line.

Also switch Storage.create() and Storage.clone() to _wait_and_reraise().
Previously, they called asyncio.wait() and implicitly swallowed all
exceptions.
2018-09-11 23:50:22 +00:00
Rusty Bird
d33bd3f2b6
storage: fix search_pool_containing_dir()
Canonicalize both directories, resolving symlink components. Compare
with commonpath() instead of startswith(), because /foo doesn't contain
/foobar.
2018-09-11 23:50:21 +00:00
Rusty Bird
f0ee73e63f
storage: remove broken default parameter from isodate()
With that syntax, the default timestamp would have been from the time of
the function's definition (not invocation). But all callers are passing
an explicit timestamp anyway.
2018-09-11 23:50:20 +00:00
Rusty Bird
44ca78523f
storage: insert missing NotImplementedError in Volume.stop() 2018-09-11 23:50:19 +00:00
Rusty Bird
266d90c2f9
storage: fix docstrings 2018-09-11 23:50:18 +00:00
Rusty Bird
c75fe09814
storage/reflink: is_reflink_supported() -> is_supported() 2018-09-11 23:50:16 +00:00
Rusty Bird
1889c9b75f
storage/reflink: run synchronous volume methods in executor
Convert create(), verify(), remove(), start(), stop(), revert(),
resize(), and import_volume() into coroutine methods, via a decorator
that runs them in the event loop's thread-based default executor.

This reduces UI hangs by unblocking the event loop, and can e.g. speed
up VM starts by starting multiple volumes in parallel.
2018-09-11 23:50:15 +00:00
Rusty Bird
3d986be02a
storage/reflink: native FICLONE in _copy_file() happy path
Avoid a subprocess launch, and distinguish reflink vs. fallback copy in
the log.
2018-09-11 23:50:14 +00:00
Rusty Bird
edda3a1734
storage/reflink: factor out _ficlone() 2018-09-11 23:50:13 +00:00
Rusty Bird
69af0a48ec
storage/reflink: inline and simplify _cmd() 2018-09-11 23:50:12 +00:00
Rusty Bird
fb06a8089a
storage/reflink: _update_loopdev_sizes() without losetup
Factor out a function, and use the LOOP_SET_CAPACITY ioctl instead of
going through losetup.
2018-09-11 23:50:10 +00:00
xaki23
ce794f33d8
add missing /sbin/ to hwclock call (so it will work as cronjob) 2018-09-10 17:24:35 +02:00
Rusty Bird
385ba91772
storage/reflink: resize(): don't look for loopdevs if clean 2018-09-09 20:01:22 +00:00
Rusty Bird
e7b7c253ac
storage/reflink: inline _require_self_on_stop() 2018-09-09 20:01:20 +00:00
Rusty Bird
6e8d7d4201
storage/reflink: no-op import_volume() if not save_on_stop
Instead of raising a NotImplementedError, just return self like 'file'
and lvm_thin. This is needed when Storage.clone() is modified in another
commit* to no longer swallow exceptions.

* "storage: factor out _wait_and_reraise(); fix clone/create"
2018-09-09 20:01:19 +00:00
Rusty Bird
60bf68a748
storage/reflink: add _path_import (don't reuse _path_dirty)
Import volume data to a new _path_import (instead of _path_dirty) before
committing to _path_clean. In case the computer crashes while an import
operation is running, the partially written file should not be attached
to Xen on the next volume startup.

Use <name>-import.img as the filename like 'file' does, to be compatible
with qubes.tests.api_admin/TC_00_VMs/test_510_vm_volume_import.
2018-09-09 20:01:18 +00:00
Rusty Bird
d301aa2e50
storage/reflink: delete stale tempfiles on start and remove
When the AT_REPLACE flag for linkat() finally lands in the Linux kernel,
_replace_file() can be modified to use unnamed (O_TMPFILE) tempfiles.
Until then, make sure stale tempfiles from previous crashes can't hang
around for too long.
2018-09-09 20:01:17 +00:00
Rusty Bird
75a4a1340e
storage/reflink: don't recompute static properties per call 2018-09-09 20:01:15 +00:00
Rusty Bird
ef2698adb4
storage/reflink: make revisions() more readable, use iglob 2018-09-09 20:01:14 +00:00
Rusty Bird
18f9356c2c
storage/reflink: refuse to revert() dirty volume 2018-09-09 20:01:13 +00:00
Rusty Bird
677183d8a6
storage/reflink: add revision even if empty
It's sort of useful to be able to revert a volume that has only ever
been started once to its empty state. And the lvm_thin driver allows it
too, so why not.
2018-09-09 20:01:12 +00:00
Rusty Bird
850778b52a
storage/reflink: remove redundant format specifiers 2018-09-09 20:01:11 +00:00
Marek Marczykowski-Górecki
a371375f1d
version 4.0.29 2018-09-09 06:38:16 +02:00
Marek Marczykowski-Górecki
b2cc605f4b
tests: clean local variables from traceback objects
System tests are fragile for any object leaks, especially those holding
open files. Instead of wrapping all tests with try/finally removing
those local variables (as done in qubes.tests.integ.backup for example),
apply generic solution: clean all traceback objects from local
variables. Those aren't used to generate text report by either test
runner (qubes.tests.run and nose2). If one wants to break into debugger
and inspect tracebacks interactively, needs to comment out call to
cleanup_traceback.
2018-09-09 02:43:17 +02:00
Marek Marczykowski-Górecki
c102fa3d68
tests: add basic audio play/rec tests
QubesOS/qubes-issues#4204
2018-09-08 04:13:24 +02:00
Marek Marczykowski-Górecki
e8d2c4e232
Merge remote-tracking branch 'qubesos/pr/213'
* qubesos/pr/213:
  Fix from Marek's revies
  doc: loading graph
2018-09-07 21:35:44 +02:00
Wojtek Porczyk
b200dc8df2
Fix from Marek's revies
- add load_extras()
- change callouts to arrows for draw.io limitations

Cc: @marmarek
2018-09-07 19:44:29 +02:00
Marek Marczykowski-Górecki
8ce3433406
tests: drop sudo in tests already running as root
Don't spam already trashed log.
2018-09-07 15:12:05 +02:00
Marek Marczykowski-Górecki
1b99bd9ab3
Merge branch 'tests-dispvm-fix'
* tests-dispvm-fix:
  tests: fix DispVM related tests
2018-09-07 01:57:19 +02:00
Marek Marczykowski-Górecki
6a3ecd0170
Merge remote-tracking branch 'qubesos/pr/229'
* qubesos/pr/229:
  Order qubesd before systemd-user-sessions
2018-09-07 01:07:27 +02:00
Marek Marczykowski-Górecki
7f1e2741ec
Merge remote-tracking branch 'qubesos/pr/228'
* qubesos/pr/228:
  storage/lvm: filter out warning about intended over-provisioning
  tests: fix getting kernel package version inside VM
  tests/extra: add start_guid option to VMWrapper
  vm/qubesvm: fire 'domain-start-failed' event even if fail was early
  vm/qubesvm: check if all required devices are available before start
  storage/lvm: fix reporting lvm command error
  storage/lvm: save pool's revision_to_keep property
2018-09-07 01:06:59 +02:00
Rusty Bird
d3a5799245
Order qubesd before systemd-user-sessions
qubes-vm@.service would already cause this ordering, but not every user
has any autostart=True VMs.

Also needed to maybe f*x QubesOS/qubes-issues#3149 at some point.
2018-09-06 16:23:24 +00:00
Marek Marczykowski-Górecki
e1378f70fb
doc: fix libvirt config path 2018-09-06 01:16:50 +02:00
Marek Marczykowski-Górecki
ee25f7c7bb
vm: fix error reporting on PVH without kernel set
Fixes QubesOS/qubes-issues#4254
2018-09-03 00:23:05 +02:00
Marek Marczykowski-Górecki
62ac42f06a
Merge remote-tracking branch 'qubesos/pr/220'
* qubesos/pr/220:
  code style fixes
  Add domain-paused/-unpaused events
2018-09-02 05:34:24 +02:00
Marek Marczykowski-Górecki
895c8d2414
Merge remote-tracking branch 'qubesos/pr/225'
* qubesos/pr/225:
  tools/qvm-sync-clock: don't start clockvm
2018-09-02 03:38:10 +02:00
Marek Marczykowski-Górecki
b72b81fb9e
Merge remote-tracking branch 'qubesos/pr/224'
* qubesos/pr/224:
  Fix error on non ASCII PCI IDs upon qvm-device list

Fixed code style.
2018-09-02 03:36:00 +02:00
Marek Marczykowski-Górecki
57c9b2edf7
code style fixes 2018-09-02 03:27:14 +02:00
Marek Marczykowski-Górecki
890df9ba03
qubespolicy: ease testing by calling str(target) only once
Don't call it multiple times depending on number of registered loggers.
2018-09-01 19:51:03 +02:00
Rusty Bird
8b2b26134e
tools/qvm-sync-clock: don't start clockvm
Since bda9264, a qubes.GetDate call from a VM will not cause clockvm
startup. Also avoid causing it with /etc/cron.d/qubes-sync-clock.cron.

Fixes QubesOS/qubes-issues#3588
2018-08-27 23:31:16 +00:00
Galland
6f04c8d65b
Fix error on non ASCII PCI IDs upon qvm-device list
solves https://github.com/QubesOS/qubes-issues/issues/4229
2018-08-21 03:08:17 +02:00
Patrick Schleizer
fbdf460db8
comments 2018-08-08 09:38:45 +00:00
Marek Marczykowski-Górecki
a121ecfeb4
Merge remote-tracking branch 'qubesos/pr/222'
* qubesos/pr/222:
  add Whonix defaults
2018-08-07 20:47:51 +02:00
Patrick Schleizer
57424ef50b
add Whonix defaults 2018-08-08 01:31:32 +08:00
Patrick Schleizer
65341a3468
$tag:anon-vm $anyvm deny 2018-08-08 00:29:07 +08:00
Jean-Philippe Ouellet
e95ef5f61d
Add domain-paused/-unpaused events
Needed for event-driven domains-tray UI updating and anti-GUI-DoS
usability improvements.

Catches errors from event handlers to protect libvirt, and logs to
main qubesd logger singleton (by default meaning systemd journal).
2018-08-01 05:41:50 -04:00
Marek Marczykowski-Górecki
ecaee10f80
Merge remote-tracking branch 'qubesos/pr/219'
* qubesos/pr/219:
  remove Whonix specific exceptions
2018-07-20 19:38:22 +02:00
Patrick Schleizer
6f311ee1ff
remove Whonix specific exceptions
Whonix will use `qvm-open-in-dvm` so no more exceptions required.
2018-07-20 23:39:00 +08:00
Patrick Schleizer
b6a015b329
remove Whonix specific exceptions
Whonix will use `qvm-open-in-dvm` so no more exceptions required.
2018-07-20 23:38:59 +08:00
Marek Marczykowski-Górecki
e7f84dedd1
version 4.0.28 2018-07-19 15:21:21 +02:00