Commit Graph

2104 Commits

Author SHA1 Message Date
3hhh
9e26bfd432
tests/lvm & callback: remove explicit class references 2020-08-05 16:56:02 +02:00
3hhh
b95339ea27
storage/callback: remove the "word of caution"
As discussed in the PR, sync code will not be interrupted when run from
async code as long as Qubes OS doesn't run dedicated threads for async
& sync code. So there's simply no issue to be expected and thus no special
caution required.
2020-08-01 10:04:27 +02:00
3hhh
b9b86976f3
storage/callback: comment fixes 2020-07-29 19:45:35 +02:00
3hhh
2487d86c72
storage/callback: add the config ID as callback argument 2020-07-29 17:30:47 +02:00
3hhh
536e12d80c
storage/callback: some callbacks added & removed
Added:
post_volume_create & post_volume_import as requested by Marek

Removed:
post_ctor as this wasn't really useful anyway, but required a lot of
sync code. Without it, some refactoring & potential async improvements
became possible.
2020-07-29 17:06:23 +02:00
3hhh
fd3a56e0cb
tests/lvm & callback: Refactoring
Mostly to avoid re-writing storage_lvm globals in storage_callback
tests.
2020-07-28 18:44:16 +02:00
3hhh
3db5e9f8bf
Revert "storage/callback: do not run sync code async"
This reverts commit 287a4a0429.

As Marek correctly pointed out, sync functions cannot be run async against one another even if run inside an async function
(the python interpreter will remain active until the next yield and that's at the end of the sync func / inside the async function).
--> So there's no need for a lock.

I still cannot protect against assumptions made by sync code authors about blocking the Qubes OS main loop. Those will be broken.

Moreover the code of this commit was botched anyway.
2020-07-28 18:42:02 +02:00
3hhh
fdceb064fe
tests/callback: ensure missing conf causes errors 2020-07-19 09:06:23 +02:00
3hhh
287a4a0429
storage/callback: do not run sync code async 2020-07-18 12:47:22 +02:00
3hhh
a53781b114
tests/callback: added callback-specific tests
This involved some further generalisation of the lvm tests.
2020-07-17 14:38:06 +02:00
3hhh
56c8d9d039
storage/callback: async Volume.export() & added Volume.export_end()
Fixes QubesOS/qubes-issues#5935
2020-07-16 17:02:19 +02:00
3hhh
42d62bb47e
storage/lvm: make the "hack" work with CallbackPool instances
CallbackPool instances are no ThinPool instances, but behave
identically, if their backend driver is a ThinPool instance.
2020-07-16 14:31:04 +02:00
3hhh
409ea88a66
storage/callback: add the backend_class property
This should be useful for devs to inspect the Callback* classes.
2020-07-16 14:31:04 +02:00
3hhh
e5838dbd97
storage/callback: various fixes
- Removed all own class attributes to avoid name clashes with delegated
class attributes.
- Implemented the previously missing Pool.usage_details property.
- Shadowed all class attributes as instance properties. This is required
as the parent classes enforce the class attributes upon the
CallbackPool & CallbackVolume classes, but they need to be delegated to
the class of the _cb_impl object. We also cannot implement them as class
attributes in CallbackVolume & CallbackPool as they need to work for
arbitrary backend drivers and two backend drivers must not interfere with
each other. Possible alternative: One could dynamically create classes.
2020-07-16 14:31:03 +02:00
3hhh
d9f1bced22
tests/callback: add rudimentary tests for the callback driver 2020-07-16 14:31:03 +02:00
3hhh
fe27b2a1eb
tests/lvm: make the tests re-usable for other drivers
in particular for the callback driver tests
2020-07-16 14:31:03 +02:00
3hhh
43fca80a5b
storage/callback: fix issues detected by pylint 2020-07-16 14:31:03 +02:00
3hhh
529e4bfbbf
storage/callback: volume callbacks now also rceive the source volume
as argument (if there's any)

This is useful for disposable VMs to identify from which template they
originate.
2020-07-16 14:31:03 +02:00
3hhh
57e7a02912
storage/callback: add a post_volume_start callback 2020-07-16 14:31:03 +02:00
3hhh
bf8ece8a0c
storage/callback: more succinct callback names
[pre|post]_[operation] should be more clear than
on_[operation]
2020-07-16 14:31:03 +02:00
3hhh
9de54ab242
storage/callback: make CallbackVolume a Volume
Unfortunately this appears to be necessary due to
various Qubes OS `assert` checks and to get `__str__()` et al
from the super class. It also means that we have to implement
all methods of the super class (in the future as well).
2020-07-16 14:31:03 +02:00
3hhh
178d4dd997
storage/callback: enforce CallbackPool as the pool attribute of delegated volumes
This fixes a bug preventing the use of the callback pool driver
with disposable VMs.
2020-07-16 14:31:03 +02:00
3hhh
caddc1c499
storage/callback: pylint: disable line-too-long 2020-07-16 14:31:03 +02:00
3hhh
889c9238fe
storage/callback: asyncio implementation 2020-07-16 14:31:03 +02:00
3hhh
170e5f5d7a
storage/callback: fix the rpm build 2020-07-16 14:31:03 +02:00
3hhh
eee800366d
storage/tests: expect the callback pool driver 2020-07-16 14:31:03 +02:00
3hhh
dab41ddcf7
storage/callback: comments 2020-07-16 14:31:03 +02:00
3hhh
a00b2d563a
storage/callback: use Qubes exceptions 2020-07-16 14:31:03 +02:00
3hhh
49dd8250c5
storage/callback: added sphinx attribute comments 2020-07-16 14:31:03 +02:00
3hhh
bbb596e3ee
storage/callback: initialize logger in __init__ 2020-07-16 14:31:02 +02:00
3hhh
5530265b27
storage/callback: make pylint happy 2020-07-16 14:31:02 +02:00
3hhh
efa0d7c257
storage/callback: more readable bash invocation 2020-07-16 14:31:02 +02:00
3hhh
746697ad2c
storage: added the callback pool driver 2020-07-16 14:31:02 +02:00
Marek Marczykowski-Górecki
784878f1f7
Merge remote-tracking branch 'origin/pr/359'
* origin/pr/359:
  Add tests for vm.volume.Clear.
  Use self.dest.storage.import* wrappers instead.
  Add admin.vm.volume.Clear call (QubesOS/qubes-issues#5946)
2020-07-16 03:44:46 +02:00
WillyPillow
be69d8ddb7
Add tests for vm.volume.Clear. 2020-07-16 00:39:16 +08:00
WillyPillow
56fbf108f8
Use self.dest.storage.import* wrappers instead. 2020-07-16 00:38:02 +08:00
Marek Marczykowski-Górecki
f30eebc40e
Merge branch 'devel20200705'
* devel20200705:
  tests: skip gnome-terminal on xfce template flavor
  tests: fix FD leak in qrexec test
  tests: switch default LVM pool to qubes_dom0/vm-pool
  backup: fix error handler for scrypt errors
  Adjust code for possibly coroutine Volume.export() and Volume.export_end()
  storage: add Volume.export_end() function
  backup: add support for calling a function after backing up a file/volume
  backup: call volume.export() just before actually extracting it
  vm/dispvm: place all volumes in the same pool as DispVM's template
  tests: extend TestPool storage driver to make create_on_disk working
  storage: pass a copy of volume_config to pool.init_volume
  tests: cleanup properly in wait_on_fail decorator
2020-07-15 16:22:08 +02:00
Marek Marczykowski-Górecki
01b33e58d7
Merge remote-tracking branch 'origin/pr/350'
* origin/pr/350:
  Changed feature keyboard_layout to a property
2020-07-15 16:21:38 +02:00
Marek Marczykowski-Górecki
6adf56f4ba
Merge remote-tracking branch 'origin/pr/355'
* origin/pr/355:
  Removed unused vm.icon_path property
2020-07-15 16:19:53 +02:00
Marek Marczykowski-Górecki
4c0f8bc24c
tests: skip gnome-terminal on xfce template flavor
It isn't installed there.
2020-07-14 20:17:41 +02:00
Marta Marczykowska-Górecka
bed8e578d7
Removed unused vm.icon_path property
The property was not used for anything, but caused numerous
problems due to symlinks.

fixes QubesOS/qubes-issues#5934
2020-07-14 17:52:09 +02:00
Marta Marczykowska-Górecka
06e2d14a97
Changed feature keyboard_layout to a property
Purpose: make it easier to implement more robust keyboard layout tools and
propagation.

references QubesOS/qubes-issues#1396
references QubesOS/qubes-issues#4294
2020-07-14 01:32:55 +02:00
Marek Marczykowski-Górecki
80c0a0caa8
tests: fix FD leak in qrexec test
Terminate dangling process to avoid FD leak (detected by test cleanup
code).
2020-07-14 01:15:45 +02:00
WillyPillow
5c7b57e690
Add admin.vm.volume.Clear call (QubesOS/qubes-issues#5946) 2020-07-14 01:52:50 +08:00
Marek Marczykowski-Górecki
e13fde07fd
Merge remote-tracking branch 'origin/pr/358'
* origin/pr/358:
  vm: emit property-reset:stubdom_xid event on domain start/stop too
2020-07-10 11:06:00 +02:00
Marek Marczykowski-Górecki
7dfaef35cd
Merge remote-tracking branch 'origin/pr/352'
* origin/pr/352:
  audio: set sink volume to workaround alsa save/restore issue
  audio: increase timeout to match hvm loading
  audio: auxiliary pauses should be harmless now, place them back just in case
  audio: add silence threshold
  audio: unload guest' module-vchan-sink in hvm tests
  audio: fix prepare_audio_vm
  audio: do not use pacat on copying audio_in.raw
  Audio: rework audio tests
2020-07-10 11:05:46 +02:00
Marek Marczykowski-Górecki
4a2e0bc734
tests: switch default LVM pool to qubes_dom0/vm-pool
This is now the installer default. Anyway, it is still possible to
override it with an environment variable.
2020-07-09 02:52:59 +02:00
Marek Marczykowski-Górecki
bd6d71a555
vm: emit property-reset:stubdom_xid event on domain start/stop too
Similart to property-reset:xid, emit property-reset:stubdom_xid when
domain is started/stopped. This allows client side of the Admin API
(qubes-core-admin-client) to invalidate the cache when necessary.

Found by audio tests: #352
2020-07-08 12:54:35 +02:00
Marek Marczykowski-Górecki
8a04abe4b1
backup: fix error handler for scrypt errors
process started via asyncio needs also stderr access via asyncio.
2020-07-08 12:50:10 +02:00
Marek Marczykowski-Górecki
0bccddf1f5
Adjust code for possibly coroutine Volume.export() and Volume.export_end()
Now Volume.export() may be a coroutine and also may be accompanied by
Volume.export_end() cleaning up after it.

See previous commits for building blocks for this.

This commit adjusts usage of Volume.export() and adds matching
Volume.export_end() throughout the code base.

Fixes QubesOS/qubes-issues#5935
2020-07-08 12:50:10 +02:00