Commit Graph

55 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
8420adf973
tests: improve audio tests
- wait for the client be listed in dom0
- report parecord stderr
- allow up to 20ms to be missing, to account for potentially suspended
  device initially
2020-04-09 05:56:12 +02:00
Marek Marczykowski-Górecki
3066190283
tests: fix qvm-copy-to-vm test
Make the check if remote file wasn't removed meaningful. Previously the
user didn't have permission to remote the source file, so even if the
tool would try, it would fail.
2020-04-01 17:57:12 +02:00
Marek Marczykowski-Górecki
1b7e2a5cbf
tests: ensure proper cleanup of auxiliary process
Various qrexec tests create auxiliary process (service_proc) as a local
variable. In case of test failure, process cleanup isn't called and may
lead to FD leaks and breaking subsequent tests.

Fix this by always saving such process instance in self.service_proc and
cleaning it up in self.tearDown() (this code is already there).

Add also waiting (and in case of timeout - killing) of a service call
process too.
2020-04-01 17:35:57 +02:00
Marek Marczykowski-Górecki
f62a861140
tests: fix test_053_qrexec_vm_service_eof_reverse
and not mark it as expected failure anymore. Note the removal of the
expected failure isn't just about the changes here, but also about the
actual fix on the qrexec side (ffafd01 "Fix not closed file descriptors in
qubes-rpc-multiplexer" commit in core-qrexec repository).
2020-03-15 02:55:18 +01:00
Marek Marczykowski-Górecki
5423ead27a
tests/vm_qrexec_gui: do not swallow stderr on failure
QubesVM.run_for_stdio() by default captures stderr. In case of call fail
(non-zero return code), captured stderr is included in the exception
object, but isn't printed by default CalledProcessError message.
Make it visible by:
 - handling CalledProcessError and including in the test failure message
   (when exception is captured already)
 - not capturing stderr (if no exception handling is present in the
   test)
2020-03-14 03:43:22 +01:00
Marek Marczykowski-Górecki
d033457f3d
tests: adjust for qrexec-client-vm exit code
It now consistently returns remote exit code, not a local one.
Check 'cat' exit code instead.
2020-03-14 03:43:17 +01:00
Marek Marczykowski-Górecki
3ce4e5eaa5
tests: adjust SystemTestCase.create_remote_file to create executables
The main use case for this function is to create qrexec services in VMs.
Since qrexec now require service scripts to be executable, make
create_remote_file() adjust permissions.
2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
2460adbdef
tests: socket-based qrexec services
QubesOS/qubes-issues#3912
2020-03-10 11:56:47 +01:00
Marek Marczykowski-Górecki
ec88796129
tests: make audio tests less racy
Wait specifically for pulseaudio to start in the VM, instead of just
waiting few seconds.
Also, improve failure message to distinguish total lack of audio from
just missing few samples.
2020-03-01 22:36:54 +01:00
Pawel Marczewski
e69bdc5efa
Test file copy hang when target machine qrexec is disabled
See QubesOS/qubes-issues#5347.

Fix in QubesOS/qubes-core-qrexec#15.
2020-01-14 15:07:42 +01:00
Marek Marczykowski-Górecki
cc1ac0b859
tests: log stderr of paplay command 2019-09-29 06:43:34 +02:00
Marek Marczykowski-Górecki
273238bd2a
tests: fix qrexec abort test
Exit code depends on when exactly the other end was terminated.
2019-09-22 23:30:38 +02:00
Marek Marczykowski-Górecki
75916dca5d
tests: fix waiting for audio recording to end 2019-04-05 03:16:55 +02:00
Marek Marczykowski-Górecki
186702646e
tests: drop expectedFailure from qubes_desktop_run test 2019-01-12 02:41:22 +01:00
Marek Marczykowski-Górecki
fb14f589cb
tests: wait for full user session before doing rest of the test
Clean VM shutdown may timeout if its initiated before full startup, so
make sure the full startup is completed first.
2018-10-27 16:44:53 +02:00
Marek Marczykowski-Górecki
9e81087b25
tests: use improved wait_for_window in various tests
Replace manual `xdotool search calls` with wait_for_window(), where
compatible.
2018-10-15 05:16:29 +02:00
Marek Marczykowski-Górecki
3b6703f2bd
tests: fix race condition in gui_memory_pinning test
Don't rely on top update timing, pause it updates for taking
screenshots.
2018-10-14 05:48:25 +02:00
Marek Marczykowski-Górecki
8dab298b89
tests: create testcases on module import if environment variable is set
If QUBES_TEST_TEMPLATES or QUBES_TEST_LOAD_ALL is set, create testcases
on modules import, instead of waiting until `load_tests` is called.
The `QUBES_TEST_TEMPLATES` doesn't require `qubes.xml` access, so it
should be safe to do regardless of the environment. The
`QUBES_TEST_LOAD_ALL` force loading tests (and reading `qubes.xml`)
regardless.

This is useful for test runners not supporting load_tests protocol. Or
with limited support - for example both default `unittest` runner and
`nose2` can either use load_tests protocol _or_ select individual tests.
Setting any of those variable allow to run a single test with those
runners.

With this feature used together load_tests protocol, tests could be
registered twice. Avoid this by not listing already defined test classes
in create_testcases_for_templates (according to load_tests protocol,
those should already be registered).
2018-10-10 00:08:29 +02:00
Marek Marczykowski-Górecki
e26655bc82
tests: fix time sync test
qvm-sync-clock no longer fetches time from the network, by design.
So, lets not break clockvm's time and check only if everything else
correctly synchronize with it.
2018-09-16 04:43:50 +02:00
Marek Marczykowski-Górecki
ac8b8a3ad4
tests: reenable some qrexec tests, convert them to py3k/asyncio 2018-09-16 04:43:50 +02:00
Marek Marczykowski-Górecki
576bcb158e
tests: skip tests not relevant on Whonix 2018-09-15 05:12:41 +02:00
Marek Marczykowski-Górecki
c102fa3d68
tests: add basic audio play/rec tests
QubesOS/qubes-issues#4204
2018-09-08 04:13:24 +02:00
Wojtek Porczyk
843bbdb2c5 qubes/tests: change globals= to module= and fix syntax errors 2018-04-24 18:30:55 +02:00
Wojtek Porczyk
dfe7688158 qubes/tests: use loadTestsFromNames for nose2 compat 2018-03-30 03:05:01 +02:00
Marek Marczykowski-Górecki
4d59f883a0
tests: minor fixes
- FD leak
- switch to xterm to test also on minimal template
2018-01-16 21:41:38 +01:00
Marek Marczykowski-Górecki
19a1579a99
tests: fix deadlock in filecopy test
Error window (where test need to send Enter key) is opened while
qvm-move-to-vm is still running.
2017-12-05 17:39:32 +01:00
Marek Marczykowski-Górecki
42f8619559
tests: fix asyncio usage some more places
gui_memory_pinning tests finally works. Also, fix DispVM and private
volume resize tests.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
2164a8d7b8
Change license to LGPL v2.1+
See this thread for reasoning and acceptance from contributors:
https://groups.google.com/d/topic/qubes-devel/G7KzrfU0lWY/discussion
"Changing qubes-core-admin license to LGPL v2.1+"
2017-10-12 00:11:50 +02:00
Marek Marczykowski-Górecki
cbc5dbb9b1
tests: various fixes
- Use proper features/services names (updates proxy test).
- Fix logic error in wait_for_window.
- Fix test for qvm-sync-clock (first sync clockvm, then dom0), also fix
  cleanup (unset clockvm before removing it)
- More fixes for asyncio usage
2017-10-04 15:24:52 +02:00
Marek Marczykowski-Górecki
687ca4092c
tests: various fixes to integration tests
Mostly add missing self.loop.run_until_complete().
2017-09-05 02:01:58 +02:00
Marek Marczykowski-Górecki
d676576600
tests: fix qvm-sync-clock test for asyncio and new time sync API
qvm-sync-clock in dom0 now synchronize only dom0 time. For VM time,
qvm-sync-clock needs to be called in VM. Also, both will communicate
with qubesd, so must be called asynchronously from tests.
2017-09-05 02:01:58 +02:00
Wojtek Porczyk
25670fbd61 qubes.tests.list_templates(), and .close() the app 2017-08-31 20:30:23 +02:00
Bahtiar `kalkin-` Gadimov
3be7b23f46
When running tests clean up and remove qubesd.socket
Changed the inheritance hierarchy:

1. Renamed `SystemTestsMixin` to `SystemTestCase`
2. `SystemTestCase` is a child of `QubesTestCase`
3. All classes extending the prior `SystemTestsMixin` now just extend `object`
2017-07-14 14:54:04 +02:00
Wojtek Porczyk
b04f612374 qubes: have "service" features' keys separated by period 2017-07-05 04:16:16 +02:00
Marek Marczykowski-Górecki
ea0cbe3a56
tests: improve tests for qrexe exit code handling
Check if exit code retrieved from dom0 is really the one expected.

Fix typo in test_065_qrexec_exit_code_vm (testvm1/testvm2), adjust for
reporing remote exit code and remove expectedFailure.

QubesOS/qubes-issues#2861
2017-06-21 05:23:35 +02:00
Marek Marczykowski-Górecki
a73dcf6016
tests: wait for session in tests requiring running GUI
Since tests expose qubesd socket, qvm-start-gui should handle starting
GUI daemons (so, GUI session inside VM). Add synchronization with it
using qubes.WaitForSession service.
2017-06-21 04:45:46 +02:00
Marek Marczykowski-Górecki
376ac4b32d
tests: fix vm.run_for_stdio in some more places
When test expect to wait for remote process, use vm.run_for_stdio.
Additionally, when the call fail, (stdout, stderr) is not assigned - use
the one attached to exception object instead.
2017-06-21 04:33:10 +02:00
Marek Marczykowski-Górecki
a0f616f14e
tests: fix checking exit code
Since run_for_stdio raise an exception for non-zero exit code, it isn't
ignored anymore. So, check if qrexec-client-vm return expected value,
instead of keep ignoring it.

QubesOS/qubes-issues#2861
2017-06-21 00:17:43 +02:00
Marek Marczykowski-Górecki
72240c13b6
tests: fix vm_qrexec_gui/TC_10_Generic/test_000_anyvm_deny_dom0
When method (as expected) raise an exception, service output would not be
assigned. Extract it from exception object.
2017-06-21 00:17:43 +02:00
Marek Marczykowski-Górecki
51a17ba749
tests: do not reload qubes.xml
In core3 this isn't needed anymore (and unit tests already check if
that's really true).
2017-06-21 00:17:42 +02:00
Marek Marczykowski-Górecki
984a070f3e
tests: move create_*_file to SystemTestsMixin 2017-06-21 00:17:40 +02:00
Wojtek Porczyk
f56f7d13fb
tests/integ/vm_qrexec_gui: skip test_051_qrexec_simple_eof_reverse
QubesOS/qubes-issues#2851
2017-06-21 00:17:39 +02:00
Wojtek Porczyk
139f18fa1d
qubes/tests/integ/vm_qrexec_gui: some fixes 2017-06-21 00:17:39 +02:00
Wojtek Porczyk
0c0b0ea6ef
qubes/tests/integ/vm_qrexec_gui: change time.sleep to asyncio.sleep 2017-06-21 00:17:38 +02:00
Wojtek Porczyk
858e547525
qubes: reorganise API protocols
Now instantiating API servers is handled by common function. This is,
among other reasons, for creating ad-hoc sockets for tests.
2017-06-20 13:00:20 +02:00
Wojtek Porczyk
d9f5192cc2 last fixes (thanks, @marmarek!) 2017-06-05 14:34:06 +02:00
Wojtek Porczyk
dc793be81f Fixes from marmarek's review 2017-06-01 12:31:14 +02:00
Wojtek Porczyk
42cbd9ff68 qubes.tests asyncio, part 2
QubesOS/qubes-issues#2622
2017-05-29 19:08:38 +02:00
Wojtek Porczyk
df03800278 Fix miscellaneous warnings
Among them:
- explicitly close files (possibly using with: syntax)
- use non-deprecated methods
2017-05-29 17:34:15 +02:00
Wojtek Porczyk
b256af3bfb qubes.tests asyncio
QubesOS/qubes-issues#2622
2017-05-29 17:34:15 +02:00