Commit Graph

6183 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
399d2138bd
tests: remove old hardware.py tests, migrated to devices_pci.py 2018-10-10 00:08:57 +02:00
Marek Marczykowski-Górecki
1245606453
tests: fix cleanup of dom0_update tests
Reset updatevm to None before removing VMs, otherwise removing updatevm
will fail.
2018-10-10 00:08:57 +02:00
Marek Marczykowski-Górecki
8dab298b89
tests: create testcases on module import if environment variable is set
If QUBES_TEST_TEMPLATES or QUBES_TEST_LOAD_ALL is set, create testcases
on modules import, instead of waiting until `load_tests` is called.
The `QUBES_TEST_TEMPLATES` doesn't require `qubes.xml` access, so it
should be safe to do regardless of the environment. The
`QUBES_TEST_LOAD_ALL` force loading tests (and reading `qubes.xml`)
regardless.

This is useful for test runners not supporting load_tests protocol. Or
with limited support - for example both default `unittest` runner and
`nose2` can either use load_tests protocol _or_ select individual tests.
Setting any of those variable allow to run a single test with those
runners.

With this feature used together load_tests protocol, tests could be
registered twice. Avoid this by not listing already defined test classes
in create_testcases_for_templates (according to load_tests protocol,
those should already be registered).
2018-10-10 00:08:29 +02:00
Marek Marczykowski-Górecki
c8929cfee9
tests: improve handling backups in core3 2018-10-07 19:51:55 +02:00
Marek Marczykowski-Górecki
7a607e3731
tests: add QUBES_TEST_TEMPLATES env variable
Allow easily list templates to be tested, without enumerating all the
test classes. This is especially useful with nose2 runner which can't
use load tests protocol _and_ select subset of tests.
2018-10-07 19:51:55 +02:00
Marek Marczykowski-Górecki
35c66987ab
tests: improve clearing tracebacks from Qubes* objects
Clear also tracebacks of chained exceptions.
2018-10-07 15:46:52 +02:00
Marek Marczykowski-Górecki
f33eca1e3f
tests: remove old 'hvm' test file
Those cases are already covered with basic core3 tests or are irrelevant
now.
2018-10-07 15:46:52 +02:00
Marek Marczykowski-Górecki
7c91e82365
tests: handle KWrite editor in DispVM tests 2018-10-07 15:46:52 +02:00
Marek Marczykowski-Górecki
02f9661169
tests: migrate mime handlers test to core3 2018-10-07 15:46:52 +02:00
Marek Marczykowski-Górecki
5bc0baeafa
tests: do not leak objects in object leaks checking function
If any object is leaked, QubesTestCase.cleanup_gc() raises an exception,
which have leaked objects list referenced in its traceback. This happens
after cleanup_traceback(), so isn't cleaned, causing cleanup_gc() fail
for all the further tests in the same test run.

Avoid this, by dropping list just before checking if any object is
leaked.
2018-09-29 02:40:43 +02:00
Marek Marczykowski-Górecki
b88fa39894
Fix mock-based build 2018-09-29 02:40:28 +02:00
Marek Marczykowski-Górecki
8b75b264ae
Merge remote-tracking branch 'origin/pr/235'
* origin/pr/235:
  Add default_qrexec_timeout to qubes-prefs
2018-09-19 06:03:10 +02:00
Marek Marczykowski-Górecki
1efac373dd
version 4.0.30 2018-09-19 05:46:07 +02:00
Marek Marczykowski-Górecki
b2387389d0
Update documentation for device-attach event 2018-09-19 05:44:02 +02:00
Rusty Bird
bee69a98b9
Add default_qrexec_timeout to qubes-prefs
When a VM (or its template) does not explicitly set a qrexec_timeout,
fall back to a global default_qrexec_timeout (with default value 60),
instead of hardcoding the fallback value to 60.

This makes it easy to set a higher timeout for the whole system, which
helps users who habitually launch applications from several (not yet
started) VMs at the same time. 60 seconds can be too short for that.
2018-09-16 18:42:48 +00:00
Marek Marczykowski-Górecki
eed2076722
Merge branch 'tests-20180913'
* tests-20180913:
  tests: fix time sync test
  tests: wait for DispVM's qubes.VMShell exit
  tests: exclude whonixcheck and NetworkManager from editor window search
  tests: reenable some qrexec tests, convert them to py3k/asyncio
  tests: skip tests not relevant on Whonix
  tests: improve shutdown timeout handling
  tests: drop qvm-prefs tests
2018-09-16 05:24:50 +02:00
Marek Marczykowski-Górecki
7feed2f680
Handle qubes.skip_autostart option on kernel command line
This allows to prevent automatically starting VMs at boot, mostly for
troubleshooting.

Fixes QubesOS/qubes-issues#4312
2018-09-16 05:22:30 +02:00
Marek Marczykowski-Górecki
e26655bc82
tests: fix time sync test
qvm-sync-clock no longer fetches time from the network, by design.
So, lets not break clockvm's time and check only if everything else
correctly synchronize with it.
2018-09-16 04:43:50 +02:00
Marek Marczykowski-Górecki
c4a84b3298
tests: wait for DispVM's qubes.VMShell exit
It isn't enough to wait for window to disappear, the service may still
be running. And if it is, test cleanup logic will complain about FD
leak.
To avoid deadlock on some test failure, do it with a timeout.
2018-09-16 04:43:50 +02:00
Marek Marczykowski-Górecki
240b1dd75e
tests: exclude whonixcheck and NetworkManager from editor window search
Those may pop up before actual editor is found, which fails the test as
it can't handle such "editor".
2018-09-16 04:43:50 +02:00
Marek Marczykowski-Górecki
ac8b8a3ad4
tests: reenable some qrexec tests, convert them to py3k/asyncio 2018-09-16 04:43:50 +02:00
Marek Marczykowski-Górecki
576bcb158e
tests: skip tests not relevant on Whonix 2018-09-15 05:12:41 +02:00
Marek Marczykowski-Górecki
556a08cb78
tests: improve shutdown timeout handling
Instead of waiting 1sec, wait up to 5sec but skip when vm is shut off.
This fix tests on slow machines, including openQA nested virt.
2018-09-15 05:12:41 +02:00
Marek Marczykowski-Górecki
5a1bf11d0d
tests: drop qvm-prefs tests
Those are moved to qubes-core-admin-client repository.
2018-09-15 05:12:41 +02:00
Rusty Bird
5756e870bd
storage/reflink: use context managers in is_supported()
Don't rely on garbage collection to close and remove the tempfiles.
2018-09-13 19:46:48 +00:00
Rusty Bird
867baf47d1
api/admin: fix typo 2018-09-13 19:46:46 +00:00
Rusty Bird
b3983f5ef8
'except FileNotFoundError' instead of ENOENT check 2018-09-13 19:46:45 +00:00
AJ Jordan
5aa35a1208
Make log location more explicit in error message
See https://github.com/QubesOS/qubes-issues/issues/4224#issuecomment-414513721.
2018-09-12 22:40:21 -04:00
Marek Marczykowski-Górecki
4dab769934
Merge remote-tracking branch 'qubesos/pr/231'
* qubesos/pr/231: (30 commits)
  tests/app: test varlibqubes pool driver selection
  tests/storage_reflink: test some file-reflink helpers
  tests/integ/storage: add file-reflink integration tests
  tests/integ/basic: use export() in get_rootimg_checksum()
  tests/integ/backupcompatibility: Storage.verify() is a coro
  tests: delete orphaned Makefile
  app: create /var/lib/qubes as file-reflink if supported
  app: uncouple pool setup from loading initial configuration
  tools/qubes-create: fix docstring
  storage: factor out _wait_and_reraise(); fix clone/create
  storage: fix search_pool_containing_dir()
  storage: remove broken default parameter from isodate()
  storage: insert missing NotImplementedError in Volume.stop()
  storage: fix docstrings
  storage/reflink: is_reflink_supported() -> is_supported()
  storage/reflink: run synchronous volume methods in executor
  storage/reflink: native FICLONE in _copy_file() happy path
  storage/reflink: factor out _ficlone()
  storage/reflink: inline and simplify _cmd()
  storage/reflink: _update_loopdev_sizes() without losetup
  ...
2018-09-12 02:42:15 +02:00
Rusty Bird
cf1ea5cee1
tests/app: test varlibqubes pool driver selection 2018-09-11 23:50:33 +00:00
Rusty Bird
797bbc43a0
tests/storage_reflink: test some file-reflink helpers
Tested:

    - _copy_file()
    - _create_sparse_file()
    - _resize_file()
    - _update_loopdev_sizes()

Smoke tested by calls from the functions above:

    - _replace_file()
    - _rename_file()
    - _make_dir()
    - _fsync_dir()
2018-09-11 23:50:32 +00:00
Rusty Bird
b82e739346
tests/integ/storage: add file-reflink integration tests 2018-09-11 23:50:31 +00:00
Rusty Bird
8c117549ad
tests/integ/basic: use export() in get_rootimg_checksum()
volume.path and volume.export() refer to the same thing in lvm_thin and
'file', but not in file-reflink (where volume.path is the -dirty.img,
which doesn't exist if the volume is not started).
2018-09-11 23:50:30 +00:00
Rusty Bird
49e7ce025f
tests/integ/backupcompatibility: Storage.verify() is a coro 2018-09-11 23:50:28 +00:00
Rusty Bird
bc30c6f3e8
tests: delete orphaned Makefile 2018-09-11 23:50:27 +00:00
Rusty Bird
8d1913a8cc
app: create /var/lib/qubes as file-reflink if supported
Use the file-reflink storage driver if /var/lib/qubes is on a filesystem
that supports reflinks, e.g. when the btrfs layout was selected in
Anaconda. If it doesn't support reflinks (or if detection fails, e.g. in
an unprivileged test environment), use 'file' as before.
2018-09-11 23:50:26 +00:00
Rusty Bird
53ef5ed431
app: uncouple pool setup from loading initial configuration
And ensure that setup is called on every type of these pools, not just
lvm_thin.
2018-09-11 23:50:25 +00:00
Rusty Bird
8eb9c64f20
tools/qubes-create: fix docstring 2018-09-11 23:50:24 +00:00
Rusty Bird
d181bf1aa4
storage: factor out _wait_and_reraise(); fix clone/create
_wait_and_reraise() is similar to asyncio.gather(), but it preserves the
current behavior of waiting for all futures and only _then_ reraising
the first exception (if there is any) in line.

Also switch Storage.create() and Storage.clone() to _wait_and_reraise().
Previously, they called asyncio.wait() and implicitly swallowed all
exceptions.
2018-09-11 23:50:22 +00:00
Rusty Bird
d33bd3f2b6
storage: fix search_pool_containing_dir()
Canonicalize both directories, resolving symlink components. Compare
with commonpath() instead of startswith(), because /foo doesn't contain
/foobar.
2018-09-11 23:50:21 +00:00
Rusty Bird
f0ee73e63f
storage: remove broken default parameter from isodate()
With that syntax, the default timestamp would have been from the time of
the function's definition (not invocation). But all callers are passing
an explicit timestamp anyway.
2018-09-11 23:50:20 +00:00
Rusty Bird
44ca78523f
storage: insert missing NotImplementedError in Volume.stop() 2018-09-11 23:50:19 +00:00
Rusty Bird
266d90c2f9
storage: fix docstrings 2018-09-11 23:50:18 +00:00
Rusty Bird
c75fe09814
storage/reflink: is_reflink_supported() -> is_supported() 2018-09-11 23:50:16 +00:00
Rusty Bird
1889c9b75f
storage/reflink: run synchronous volume methods in executor
Convert create(), verify(), remove(), start(), stop(), revert(),
resize(), and import_volume() into coroutine methods, via a decorator
that runs them in the event loop's thread-based default executor.

This reduces UI hangs by unblocking the event loop, and can e.g. speed
up VM starts by starting multiple volumes in parallel.
2018-09-11 23:50:15 +00:00
Rusty Bird
3d986be02a
storage/reflink: native FICLONE in _copy_file() happy path
Avoid a subprocess launch, and distinguish reflink vs. fallback copy in
the log.
2018-09-11 23:50:14 +00:00
Rusty Bird
edda3a1734
storage/reflink: factor out _ficlone() 2018-09-11 23:50:13 +00:00
Rusty Bird
69af0a48ec
storage/reflink: inline and simplify _cmd() 2018-09-11 23:50:12 +00:00
Rusty Bird
fb06a8089a
storage/reflink: _update_loopdev_sizes() without losetup
Factor out a function, and use the LOOP_SET_CAPACITY ioctl instead of
going through losetup.
2018-09-11 23:50:10 +00:00
xaki23
ce794f33d8
add missing /sbin/ to hwclock call (so it will work as cronjob) 2018-09-10 17:24:35 +02:00