Commit Graph

6136 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
4a2e0bc734
tests: switch default LVM pool to qubes_dom0/vm-pool
This is now the installer default. Anyway, it is still possible to
override it with an environment variable.
2020-07-09 02:52:59 +02:00
Marek Marczykowski-Górecki
bd6d71a555
vm: emit property-reset:stubdom_xid event on domain start/stop too
Similart to property-reset:xid, emit property-reset:stubdom_xid when
domain is started/stopped. This allows client side of the Admin API
(qubes-core-admin-client) to invalidate the cache when necessary.

Found by audio tests: #352
2020-07-08 12:54:35 +02:00
Marek Marczykowski-Górecki
8a04abe4b1
backup: fix error handler for scrypt errors
process started via asyncio needs also stderr access via asyncio.
2020-07-08 12:50:10 +02:00
Marek Marczykowski-Górecki
0bccddf1f5
Adjust code for possibly coroutine Volume.export() and Volume.export_end()
Now Volume.export() may be a coroutine and also may be accompanied by
Volume.export_end() cleaning up after it.

See previous commits for building blocks for this.

This commit adjusts usage of Volume.export() and adds matching
Volume.export_end() throughout the code base.

Fixes QubesOS/qubes-issues#5935
2020-07-08 12:50:10 +02:00
Dmitry Fedorov
0cf5fe55c2
audio: set sink volume to workaround alsa save/restore issue 2020-07-08 13:18:38 +03:00
Dmitry Fedorov
2fe57b6281
audio: increase timeout to match hvm loading 2020-07-08 13:02:51 +03:00
Marek Marczykowski-Górecki
d96480719f
storage: add Volume.export_end() function
This is a counterpart to Volume.export(). Up until now, no driver needed
any cleanup after exporting data, but it doesn't mean there won't be
any. This is especially relevant because Volume.export() is supposed to
return a path of a snapshot from before VM start - which may be a
different one than currently active one.

QubesOS/qubes-issues#5935
2020-07-08 06:05:02 +02:00
Marek Marczykowski-Górecki
f48327f636
backup: add support for calling a function after backing up a file/volume
When Volume.export is called late and can be also a coroutine, it may
make sense to also have a cleanup function for changes made by it.
This commit only adjust backup code internals, but doesn't call
appropriate Volume function yet.

QubesOS/qubes-issues#5935
2020-07-08 06:05:02 +02:00
Marek Marczykowski-Górecki
ebd0ca7e79
backup: call volume.export() just before actually extracting it
There are two reasons for this:
 - call it from a coroutine, allowing export() itself be a coroutine
 - avoid calling export() when only collecting preliminary backup
   summary

Both needs some more changes in other parts of the codebase to be useful
(see next commits).
This will be especially useful when export() will need to make some
changes (like, create a snapshot, mount something etc).

QubesOS/qubes-issues#5935
2020-07-08 06:05:02 +02:00
Marek Marczykowski-Górecki
8b760451a6
vm/dispvm: place all volumes in the same pool as DispVM's template
Make all volume's pool controlled by DisposableVM Template. This
specifically makes DispVM's volatile volume to be placed directly in the
same pool as its template.

Fixes QubesOS/qubes-issues#5933
2020-07-08 06:05:01 +02:00
Marek Marczykowski-Górecki
410a0728cc
tests: extend TestPool storage driver to make create_on_disk working
Add dummy TestVolume with empty create() method. Other core code
requires also TestPool.get_volume implemented, so add that too (naive
version remembering instances returned from TestPool.init_volume).
2020-07-08 06:05:01 +02:00
Marek Marczykowski-Górecki
d9d55b0586
storage: pass a copy of volume_config to pool.init_volume
Avoid local modification in a pool's init_volume influence
vm.volume_config. Currently every pool driver replaces
volume_config['pool'] with a pool object (instead of name) and it leads
to confusing cases where depending on start stage, it is sometimes an
object and sometimes a string.
Additionally, some pool drivers may modify volume_config in unexpected
way - for example test pool driver removes 'pool' entry entirely. Avoid
this fragile interface by giving pool driver a copy of volume_config,
instead of vm.volume_config directly.

Note one side effect is that 'vid' (and other pool-specific parameters)
is not set into vm.volume_config directly after creating a VM, but
possibly only after loading from XML. This should not be an issue in
theory (no core code should expect it), but if some place use
volume_config instead of Volume instance for getting pool-specific
options, it should be fixed.
2020-07-08 06:05:01 +02:00
Marek Marczykowski-Górecki
e38265501c
tests: cleanup properly in wait_on_fail decorator
Close transport used to wait for user input, otherwise all further tests
would fail on cleanup (FD leak detected). This in practice is only
useful when using wait_on_fail decorator without --failfast option.
2020-07-08 06:05:01 +02:00
Rusty Bird
1b09528740
storage/reflink: clarify comment 2020-07-07 16:25:48 +00:00
Rusty Bird
e188b93c95
storage: move @locked from lvm to Volume base class
And use it in reflink, instead of a synchronous lock.
2020-07-07 15:39:08 +00:00
Rusty Bird
a1b5262426
storeage/reflink: unlock size getter
Don't update _size in the getter, so it can be unlocked (which is
helpful for QubesOS/qubes-issues#5935).

!!! If cherry-picking for release4.0, also adjust import_data() to !!!
!!! use self.size (no underscore) instead of self._get_size()      !!!
2020-07-07 15:39:06 +00:00
Rusty Bird
b98c1814ee
storage/reflink: update _size during volatile volume stop 2020-07-07 15:39:05 +00:00
Marek Marczykowski-Górecki
d0f619d3c6
Merge branch 'tests-extra-fix' 2020-06-24 18:02:35 +02:00
Dmitry Fedorov
a27a8edf88
audio: auxiliary pauses should be harmless now, place them back just in case 2020-06-24 15:22:34 +03:00
Dmitry Fedorov
a68144e688
audio: add silence threshold 2020-06-24 15:03:10 +03:00
Dmitry Fedorov
362cf006eb
audio: unload guest' module-vchan-sink in hvm tests 2020-06-24 14:59:24 +03:00
Dmitry Fedorov
314e3c5d45
audio: fix prepare_audio_vm 2020-06-24 14:57:37 +03:00
Dmitry Fedorov
79a05dcceb
audio: do not use pacat on copying audio_in.raw 2020-06-24 14:51:46 +03:00
Marek Marczykowski-Górecki
1ebbfa8dbc
tests: fix extra tests loader
Fix load error reporting - make sure 'err' variable is transferred into
'runTest' function scope.
Then, relax test loading requirements - use 'resolve' instead of 'load',
to bypass dependencies check (defined in setup.py of the package). The
required dependencies should be handled by RPM already, and in some
cases may not match those in python package. An example is PDF
converter, where dependencies at python level are set for the actual
converter, which is irrelevant for running tests from dom0 (tests will
interact with PDF converter inside a VM).
2020-06-24 01:27:57 +02:00
Dmitry Fedorov (easydozen)
fd7d1267d7
Audio: rework audio tests
Replace constant audio bytearray with generated sine wave sample
to get more robust results across test environments.

Use zero-crossings as an audio fingerprint
and compare it with sine wave frequency.
2020-06-23 16:12:09 +03:00
Rusty Bird
5ed762d152
storage: clean up __init__() of Pool() and subclasses
- Remove **kwargs
- Make all arguments keyword-only
- Use required arguments instead of asserts

QubesOS/qubes-issues#5895
2020-06-22 14:03:21 +00:00
Rusty Bird
652bf3b6d9
storage: modernize super() syntax 2020-06-22 14:03:20 +00:00
Rusty Bird
c2ce28936e
storage/reflink: prefer canonical values for setup_check 2020-06-22 14:03:19 +00:00
Rusty Bird
6f2d82fd20
storage/kernels: fix pool revisions_to_keep setter message 2020-06-22 14:03:18 +00:00
Rusty Bird
3ab789f5d8
storage/file: remove redundant _revisions_to_keep init
FilePool's revisions_to_keep setter - called in the next line via
Pool.__init__() - initializes the variable.
2020-06-22 14:03:17 +00:00
Rusty Bird
8f1d59e9af
api/admin: report unexpected driver options in admin.pool.Add
Fixes QubesOS/qubes-issues#5895
2020-06-22 14:03:16 +00:00
Marek Marczykowski-Górecki
c4739b7ed0
travis: don't clone qubes-builder twice
It is already part of the included base config.
2020-06-22 15:52:51 +02:00
Marek Marczykowski-Górecki
a230a7236b
version 4.1.13 2020-06-13 16:38:41 +02:00
Marek Marczykowski-Górecki
bda82c894d
libvirt xen: Add just device_video
Re-introduce part of "libvirt xen: Add gfx_passthru and device_video"
commit by Artur Puzio <contact@puzio.waw.pl>
2020-06-10 16:32:48 +02:00
Marek Marczykowski-Górecki
4f04a1db91
Revert "libvirt xen: Add gfx_passthru and device_video"
Libxl now automatically enables the gfx_passthru when IGD is attached.

This reverts commit 3e8f5a7a57.
2020-06-10 15:40:28 +02:00
Marek Marczykowski-Górecki
dc1b3b4d86
Do not announce RDRAND instruction on Ivy Bridge
XSA-320 / CVE-2020-0543 affects Ivy Bridge and later platforms, but a
fix (microcode update) won't be available for Ivy Bridge. Disable
affected instruction (do not announce it in CPUID - complying software
should not use it then).
2020-06-10 06:12:46 +02:00
Marek Marczykowski-Górecki
47d4030bdb
tests: move pure-qrexec tests into separate module
Move qrexec (as a mechianism) tests into qubes.tests.integ.qrexec.
Yet anoter split for better openQA jobs scheduling.
2020-06-06 04:07:55 +02:00
Marek Marczykowski-Górecki
60b8b40317
tests: move IPv6 tests into separate file
Move to qubes.tests.integ.network_ipv6.
Again, to allow scheduling as a separate openQA job.
2020-06-06 04:07:55 +02:00
Marek Marczykowski-Górecki
586526c072
tests: move vm update test to a seprate file
This allows scheduling them in a separate job on openQA
2020-06-06 04:07:54 +02:00
Dmitry Fedorov
01fc33e4df
add audio-model feature 2020-05-29 12:09:40 +03:00
Marek Marczykowski-Górecki
719eb5c2a2
Fix revisions_to_keep volume property when creating VM in a specific pool
The revisions_to_keep should be inherited from the pool by default (or
whatever else logic is in the storage pool driver). When creating VM in
a specific pool, volumes config is re-initialized to include right
defaults. But the config cleaning logic in `_clean_volume_config()`
failed to remove revisions_to_keep property initialized by the default
pool driver. This prevented new pool driver to apply its own default
logic.

An extreme result was inability to create a VM in 'file' pool at all,
because it refuses any revisions_to_keep > 1, and the default LVM
pool has revisions_to_keep = 2.
2020-05-28 01:45:18 +02:00
Marek Marczykowski-Górecki
4e6472b018
tests: switch simple http server to python3
Python2 is phased out and not available anymore in some templates.
2020-05-27 13:41:02 +02:00
Marek Marczykowski-Górecki
096b7d611a
tests: adjust top.enable salt test
Don't assume any other enabled top. If there is none, 'base'
(environment name) also isn't listed.
2020-05-27 13:36:30 +02:00
Marek Marczykowski-Górecki
b889bf98f4
rpm: adjust dependency to prevent mismatching qubes-core-admin-client
Actually, make the dependency on the python module package, which is the
important part.
2020-05-26 02:30:24 +02:00
Marek Marczykowski-Górecki
ba89ad469c
version 4.1.12 2020-05-24 04:37:53 +02:00
Marek Marczykowski-Górecki
aebb1594d7
Merge branch 'qubesd-socket'
* qubesd-socket:
  rpm: update dependency on qrexec
  Remove qubesd-query-fast
  Make qubesd connected directly as an socket qrexec service
2020-05-24 03:26:04 +02:00
Marek Marczykowski-Górecki
7193fad5f5
Merge branch 'policy-convert'
* policy-convert:
  Avoid moving old, user-modified qrexec policy to .rpmsave files
  Convert default qrexec policy to the new format
2020-05-24 02:23:06 +02:00
Marek Marczykowski-Górecki
657b6e4806
Avoid moving old, user-modified qrexec policy to .rpmsave files
Keep it at original files, to still load it using compat rules.
This way the update should not break user's policies.

Note the unchanged policy files are still going to be removed - meaning
those calls will use the new policy.

QubesOS/qubes-issues#4370
2020-05-24 02:22:37 +02:00
Marek Marczykowski-Górecki
9cc6050e7c
Convert default qrexec policy to the new format
QubesOS/qubes-issues#4370
2020-05-24 02:22:36 +02:00
Marek Marczykowski-Górecki
64edbdf7d3
Merge remote-tracking branch 'origin/pr/344'
* origin/pr/344:
  travis: pip -> pip3
  Update .travis.yml
  Drop initial root thin pool definition
  Prevent double hyphens in thin_pool parsing
  Rename default root thin pool from 'lvm' to 'root'
2020-05-24 02:19:37 +02:00