Commit Graph

906 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
55208a0c7c
Merge branch 'dispvm-policy' 2017-09-14 00:15:29 +02:00
Marek Marczykowski-Górecki
0d59965a7b
Merge branch 'cdrom-boot'
* cdrom-boot:
  devices: fix error reporting
  api/admin: implement admin.vm.device....Set.persistent
  devices: implement DeviceCollection.update_persistent()
  devices: move DeviceInfo definition earlier
  api: do not fail events when listener is cancelled in the meantime
2017-09-07 13:02:04 +02:00
Marek Marczykowski-Górecki
99bd193688
Rename 'dispvm_allowed' to 'template_for_dispvms'
'dispvm_allowed' name was confusing, because it suggested being able to
spawn new DispVMs, not being a template for DispVM.

Fixes QubesOS/qubes-issues#3047
2017-09-05 02:07:26 +02:00
Marek Marczykowski-Górecki
c247ddff72
dispvm: fix setting up new DispVM
Clone properties from DispVM template after setting base properties
(qid, name, uuid). This means we can use standard clone_properties()
function. Otherwise various setters may fail - for example
netvm setter require uuid property initialized (for VM lookup in VM
collection).
Also, make dispvm_allowed check more robust - include direct creation of
DispVM, and also check just before VM startup (if property was changed
in the meantime).

Fixes QubesOS/qubes-issues#3057
2017-09-05 02:07:26 +02:00
Marek Marczykowski-Górecki
687ca4092c
tests: various fixes to integration tests
Mostly add missing self.loop.run_until_complete().
2017-09-05 02:01:58 +02:00
Marek Marczykowski-Górecki
d676576600
tests: fix qvm-sync-clock test for asyncio and new time sync API
qvm-sync-clock in dom0 now synchronize only dom0 time. For VM time,
qvm-sync-clock needs to be called in VM. Also, both will communicate
with qubesd, so must be called asynchronously from tests.
2017-09-05 02:01:58 +02:00
Marek Marczykowski-Górecki
cea70748a6
api/admin: implement admin.vm.device....Set.persistent
This will allow converting persistent device (used to boot VM for
example) to non-persistent.

QubesOS/qubes-issues#3055
2017-09-05 01:33:47 +02:00
Marek Marczykowski-Górecki
9d062c4c66
devices: implement DeviceCollection.update_persistent()
Allow attached device to be converted from persistent to non-persistent
and the other way around.
This is to allow starting a VM with some device attached temporarily.
When VM is not running, it is possible to attach device only
persistently, so this change will allow to do that, then, after starting
the VM, change it to non-persistent - so it will not be attached again
at further startups.

QubesOS/qubes-issues#3055
2017-09-05 01:33:47 +02:00
Wojtek Porczyk
6ff1bfdc16 qubes/tests: convert some tearDown into addCleanup
This is because .tearDown() is not executed if the exception occurs in
setUp() [for example self.skipTest() raises an exception]. The lower
levels of .tearDown() being executed are critical to not leaking file
descriptors.
2017-08-31 20:30:24 +02:00
Wojtek Porczyk
d0f2fdba55 qubes/tests: fix some of the fd leaks
This is WIP, there are more fds leaking.
2017-08-31 20:30:24 +02:00
Wojtek Porczyk
f1334e5095 qubes/tests/run: --break-to-repl 2017-08-31 20:30:24 +02:00
Wojtek Porczyk
25670fbd61 qubes.tests.list_templates(), and .close() the app 2017-08-31 20:30:23 +02:00
Wojtek Porczyk
8547d06cc9 qubes/tests: skip env-dependent tests using decorator
Gtk segfaults without X11 display. Some tests need dedicated PCI device.
2017-08-31 20:29:34 +02:00
Marek Marczykowski-Górecki
7a9e5c3650
tests: qrexec-policy cli tool tests 2017-08-14 02:24:32 +02:00
Marek Marczykowski-Górecki
971c7d4ac9
api/admin: add admin.vm.CreateDisposable in place of internal.vm.Create.DispVM
Add public Admin API call to create Disposable VM that would be
automatically destroyed after shutdown. Do not keep this functionality
for qrexec-policy tool only.
Also, use admin.vm.Start there, instead of internal.vm.Start and
admin.vm.Kill instead of internal.vm.CleanupDispVM (this is enough,
because DispVM now have auto_cleanup property).

QubesOS/qubes-issues#2974
2017-08-06 20:54:10 +02:00
Marek Marczykowski-Górecki
691a6f4d8c
vm/dispvm: add auto_cleanup property, unify creating new DispVM
Add auto_cleanup property, which remove DispVM after its shutdown
- this is to unify DispVM handling - less places needing special
handling after DispVM shutdown.

New DispVM inherit all settings from respective AppVM. Move this from
classmethod `DispVM.from_appvm()`, to DispVM constructor. This unify
creating new DispVM with any other VM class.
Notable exception are attached devices - because only one running VM can
have a device attached, this would prevent second DispVM started from
the same AppVM. If one need DispVM with some device attached, one can
create DispVM with auto_cleanup=False. Such DispVM will still not have
persistent storage (as any other DispVM).

Tests included.

QubesOS/qubes-issues#2974
2017-08-06 20:54:09 +02:00
Marek Marczykowski-Górecki
36f1a3abaf
Merge branch 'services'
* services:
  tests: check clockvm-related handlers
  doc: include list of extensions
  qubesvm: fix docstring
  ext/services: move exporting 'service.*' features to extensions
  app: update handling features/service os ClockVM
2017-07-29 05:09:32 +02:00
Marek Marczykowski-Górecki
34ff40c721
tests: fix AdminVM test 2017-07-29 05:07:45 +02:00
Marek Marczykowski-Górecki
c8c32d4956
Merge branch 'stats'
* stats:
  tests: admin.vm.Stats
  api: fix handling interrupted calls
  api/admin: implement admin.vm.Stats
  app: refresh getting VM statistics, rename to QubesHost.get_vm_stats
  api: use str(subject) instead of explicit subject.name
2017-07-29 05:05:32 +02:00
Marek Marczykowski-Górecki
76640df091
Merge branch 'tests-storage'
* tests-storage:
  tests: register libvirt events
  tests: even more agressive cleanup in tearDown
  app: do not wrap libvirt_conn.close() in auto-reconnect wrapper
  api: keep track of established connections
  tests: drop VM cleanup from tearDownClass, fix asyncio usage in tearDown
  storage: fix Storage.clone and Storage.clone_volume
  tests: more tests fixes
  firewall: raise ValueError on invalid hostname in dsthost=
  qmemman: don't load qubes.xml
  tests: fix AdminVM test
  tests: create temporary files in /tmp
  tests: remove renaming test - it isn't supported anymore
  tests: various fixes for storage tests
  tests: fix removing LVM volumes
  tests: fix asyncio usage in some tests
  tests: minor fixes to api/admin tests
  storage/file: create -cow.img only when needed
  storage: move volume_config['source'] filling to one place
  app: do not create 'default' storage pool
  app: add missing setters for default_pool* global properties
2017-07-29 05:01:42 +02:00
Marek Marczykowski-Górecki
639fa26079
Merge branch 'qdb-watch'
* qdb-watch:
  tests: add qdb_watch test
  ext/block: make use of QubesDB watch
  vm: add API for watching changes in QubesDB
  vm: optimize imports
  api/admin: don't send internal events in admin.Events
  Add explanation why admin.vm.volume.Import is a custom script
  Follow change of qubesdb path return type
  Rename vm.qdb to vm.untrusted_qdb
2017-07-29 05:01:13 +02:00
Marek Marczykowski-Górecki
1556814f59
backup: fix handling already encoded passphrase
When passphrase is retrieved from VM, it is already encoded. Do not try
to encode it again.

QubesOS/qubes-issues#2931
2017-07-29 04:47:43 +02:00
Marek Marczykowski-Górecki
73fb16dbda
tests: check clockvm-related handlers 2017-07-28 16:32:48 +02:00
Marek Marczykowski-Górecki
9bbefa0edb
tests: admin.vm.Stats
QubesOS/qubes-issues#853
2017-07-28 03:02:33 +02:00
Marek Marczykowski-Górecki
e8b875f552
app: refresh getting VM statistics, rename to QubesHost.get_vm_stats
Get a VM statistics once. If previous measurements are provided,
calculate difference too. This is backend part of upcoming
admin.vm.Stats service.

QubesOS/qubes-issues#853
2017-07-27 22:16:03 +02:00
Marek Marczykowski-Górecki
0d9574d9fc
api: use str(subject) instead of explicit subject.name
This allows better flexibility, when subject is not necessary a VM
object.
2017-07-27 22:15:02 +02:00
Marek Marczykowski-Górecki
6a85592b51
tests: register libvirt events
This is especially needed to have proper domain-shutdown event.
2017-07-26 03:27:00 +02:00
Marek Marczykowski-Górecki
c53582b285
tests: even more agressive cleanup in tearDown
Remove some more references to objects holding (possibly indirectly)
reference to libvirt connection:
 - local variables in tearDown function
 - running Admin API calls (especially admin.Events)
 - vmm._libvirt_conn directly, in case some reference to Qubes()
   is still there
 - any instance attribute that is an object from 'qubes' python package
   (instead of just those descending from BaseVM)
 - do not create new Qubes() instance for removing VMs - if we already
   have one in self.app

Then trigger garbage collector to really cleanup those objects (and
close relevant file descriptors). It's important do do this before
closing event loop, because some of descructors may try to use it (for
example remove registered handlers).
2017-07-26 03:27:00 +02:00
Marek Marczykowski-Górecki
4be325a383
tests: drop VM cleanup from tearDownClass, fix asyncio usage in tearDown
When tearDownClass is executed, event loop is already closed. Since no
test really need it right now, drop support for test class-wide VMs and
convert those methods back to instance methods.
Also put coroutines (vm.remove_from_disk, vm.kill) onto event loop.
2017-07-26 03:26:59 +02:00
Marek Marczykowski-Górecki
1af7034d57
tests: more tests fixes 2017-07-26 03:26:58 +02:00
Marek Marczykowski-Górecki
214c646417
firewall: raise ValueError on invalid hostname in dsthost=
...instead of AssertionError
2017-07-26 03:26:58 +02:00
Marek Marczykowski-Górecki
5d9bc00885
tests: fix AdminVM test
Since it is no longer child of QubesVM, constructor do not take 'qid'
and 'name' arguments.
Also:
- remove other dropped properties tests (netvm, storage related)
- make the test working in non-dom0
2017-07-26 03:26:40 +02:00
Marek Marczykowski-Górecki
a9934316c1
tests: create temporary files in /tmp 2017-07-26 03:26:39 +02:00
Marek Marczykowski-Górecki
8bae2d39c2
tests: remove renaming test - it isn't supported anymore
QubesOS/qubes-issues#2868
2017-07-26 03:26:39 +02:00
Marek Marczykowski-Górecki
8b2db94b41
tests: various fixes for storage tests
- improve TestPool mock - init_volume now return appropriate mock type,
   instead of TestPool
 - improve patching base directory (/var/lib/qubes) - it is stored in
   more than one place...
 - fix inheritance in TC_01_ThinPool class
 - fix expected LVM volume names ('vm-' prefix)
 - fix cleanup after FilePool tests - remove temporary qubes.xml
 - asyncio usage
 - better reporting in integ.storage - include error message in the
   report, not only as a comment in code
2017-07-26 03:26:13 +02:00
Marek Marczykowski-Górecki
a8e2f3111d
tests: fix removing LVM volumes
VM volumes have 'vm-' name prefix now.
This is continuation of fd5386c storage/lvm: prefix VM LVM volumes with 'vm-'
2017-07-26 03:26:12 +02:00
Marek Marczykowski-Górecki
75394a1348
tests: fix asyncio usage in some tests 2017-07-26 03:26:12 +02:00
Marek Marczykowski-Górecki
ba2b034843
tests: minor fixes to api/admin tests
Drop debug print, fix mocking Storage.remove (it is a coroutine).
2017-07-26 03:26:11 +02:00
Marek Marczykowski-Górecki
4208897146
storage/file: create -cow.img only when needed
Do not create it at volume creation time. It it needed only when VM is
running, so create it just before startup only.

QubesOS/qubes-issues#2256
2017-07-26 03:26:11 +02:00
Marek Marczykowski-Górecki
c5667791e8
storage: move volume_config['source'] filling to one place
Don't set 'source' volume in various places (each VM class constructor
etc), do it as part of volume initialization. And when it needs to be
re-calculated, call storage.init_volume again.

This code was duplicated, and as usual in such a case, those copies
were different - one have set 'size', the other one not.

QubesOS/qubes-issues#2256
2017-07-26 03:26:11 +02:00
Marek Marczykowski-Górecki
3a21e1f1b3
app: do not create 'default' storage pool
Since we have app.default_pool* properties, create appropriately named
pool and let those properties choose the right pool. This also means we
don't need to specify pool name in default volume config anymore

QubesOS/qubes-issues#2256
2017-07-25 06:20:42 +02:00
Marek Marczykowski-Górecki
1933a720d0
tests: add qdb_watch test
Use dom0 instance of QubesDB, to be independent of working (or not) VMs
at this stage.

QubesOS/qubes-issues#2940
2017-07-25 05:20:39 +02:00
Marek Marczykowski-Górecki
0fb7c1fbed
Follow change of qubesdb path return type
It's str/unicode, not bytes now.

QubesOS/qubes-issues#2937
2017-07-25 05:20:38 +02:00
Marek Marczykowski-Górecki
1759bca00f
Rename vm.qdb to vm.untrusted_qdb
QubesDB can be freely modified by a VM, so one should take care when
reading any data retrieved from it.

Fixes QubesOS/qubes-issues#2934
2017-07-24 13:01:55 +02:00
Marek Marczykowski-Górecki
397a8263bd
tarwriter: use new PAX format, instead of old GNU for sparse files
The old format have many issues and is discouraged by tar developers. In
this case the most important one is header with possible non-ASCII
characters, which will result in UnicodeDecodeError (tarfile module
require header parts in utf-8).
PAX format is much cleaner, as it use standard mechanism for extended
headers.
2017-07-21 03:14:06 +02:00
Marek Marczykowski-Górecki
dbea03ad1f
tests: admin.backup.*
Those tests are only about Admin API integration, not about actual backup
backend code

QubesOS/qubes-issues#2931
2017-07-20 12:19:01 +02:00
Marek Marczykowski-Górecki
801d6acf5c
Rename 'net/*' features to 'net.*'
Same reasoning as with 'service/*' - '/' is not allowed in qrexec (so
Admin API) call argument.

Related to QubesOS/qubes-issues#2906
2017-07-18 03:50:02 +02:00
Marek Marczykowski-Górecki
9e4f2d0907
Merge remote-tracking branch 'qubesos/pr/131'
* qubesos/pr/131:
  When running tests clean up and remove qubesd.socket
  Do not display gi Gtk warning when running tests
2017-07-18 02:19:34 +02:00
Marek Marczykowski-Górecki
c6e09b5a99
storage: fix resize and revert handling
- fix coroutine calling from Admin API
 - fix off-by-one error in resize
 - adjust tests
2017-07-18 01:42:37 +02:00
Marek Marczykowski-Górecki
4037bf9abc
vm/qubesvm: do not assign all host vCPUs to a qube
Fixes QubesOS/qubes-issues#2900
2017-07-17 12:33:52 +02:00
Marek Marczykowski-Górecki
ef756eb05d
tests: libvirt xml rendering unit test 2017-07-17 12:28:24 +02:00
Marek Marczykowski-Górecki
9ba84ecdad
Add code and test for migrating vm.hvm into vm.virt_mode
This will allow to load old qubes.xml - otherwise previous change render
the system unusable (until manual qubes.xml edit).

QubesOS/qubes-issues#2912
2017-07-17 12:27:17 +02:00
Marek Marczykowski-Górecki
21940bef90
Change vm.hvm into vm.virt_mode
Fixes QubesOS/qubes-issues#2912
2017-07-17 12:26:10 +02:00
Marek Marczykowski-Górecki
cbc7241a93
api/admin: rename mgmt-permission:* event to admin-permission:*
Be more consistent with Admin API name and method names.
2017-07-17 02:41:42 +02:00
Bahtiar `kalkin-` Gadimov
3be7b23f46
When running tests clean up and remove qubesd.socket
Changed the inheritance hierarchy:

1. Renamed `SystemTestsMixin` to `SystemTestCase`
2. `SystemTestCase` is a child of `QubesTestCase`
3. All classes extending the prior `SystemTestsMixin` now just extend `object`
2017-07-14 14:54:04 +02:00
Marek Marczykowski-Górecki
a42e500108
Merge branch 'tests-fixes-1'
* tests-fixes-1:
  api: extract function to make pylint happy
  tests/vm: simplify AppVM storage test
  storage: do not use deepcopy on volume configs
  api: cleanup already started servers when some later failed
  tests: fix block devices tests when running on real system
  tests: fix some FD leaks
2017-07-14 01:05:21 +02:00
Marek Marczykowski-Górecki
2df81adab1
tests/vm: simplify AppVM storage test
Use minimal TestPool(), instead of Mock().
This allow effectively compare Volume instances (init_volume with the
same parameters return equal Volume instance).
2017-07-12 21:33:14 +02:00
Marek Marczykowski-Górecki
d7b968d867
tests: fix block devices tests when running on real system 2017-07-12 21:33:14 +02:00
Marek Marczykowski-Górecki
7aa8d74ad4
tests: fix some FD leaks
There are still much more of them...
2017-07-12 21:33:14 +02:00
Wojtek Porczyk
b04f612374 qubes: have "service" features' keys separated by period 2017-07-05 04:16:16 +02:00
Marek Marczykowski-Górecki
dddd94b339
Merge branch 'core3-storage3' 2017-07-04 16:44:53 +02:00
Marek Marczykowski-Górecki
6db39345fb
Merge remote-tracking branch 'origin/core3-quick-fixes-20170703' 2017-07-04 16:11:18 +02:00
Marek Marczykowski-Górecki
51022cada5
Merge remote-tracking branch 'origin/policy-adminvm' 2017-07-04 15:51:30 +02:00
Marek Marczykowski-Górecki
0e554296e3
storage: drop 'internal' and 'removable' volume properties
Since dynamic volumes (qvm-block) are moved to devices API, those two
are not needed anymore.

QubesOS/qubes-issues#2256
2017-07-04 14:25:08 +02:00
Marek Marczykowski-Górecki
075d991114
config: eliminate duplicated qubes_base_dir
Remove it from system_path dict, have it in one place.
2017-07-04 14:25:07 +02:00
Marek Marczykowski-Górecki
317d140f46
storage/file: major FilePool/FileVolume cleanup and documentation
This driver isn't used in default Qubes 4.0 installation, but if we do
have it, let it follow defined API and its own documentation. And also
explicitly reject not supported operations:
 - support only revisions_to_keep<=1, but do not support revert() anyway
 (implemented version were wrong on so many levels...)
 - use 'save_on_stop'/'snap_on_start' properties directly instead of
 obsolete volume types
 - don't call sudo - qubesd is running as root
 - consistently use path, path_cow, path_source, path_source_cow

Also, add tests for BlockDevice instance returned by
FileVolume.block_device().

QubesOS/qubes-issues#2256
2017-07-04 14:25:07 +02:00
Marek Marczykowski-Górecki
8318038a73
tests: workaround asyncio.iscouroutinefunction(Mock)=True
It's already fixed in Python 3.5.3, but on Travis we have Python 3.5.2.
https://bugs.python.org/issue28703
2017-07-04 13:04:24 +02:00
Marek Marczykowski-Górecki
d00e4538bf
devices: make attach/detach related events async
This will allow starting processes and calling RPC services in those
events. This if required for usb devices, which are attached using RPC
services.
Intentionally keep device listing events synchronous only - to
discourage putting long-running actions there.

This change also require some not-async attach method version for
loading devices from qubes.xml - have `load_persistent` for this.
2017-07-04 13:04:23 +02:00
Marek Marczykowski-Górecki
ea1a04cb19
events: add support for async event handlers
See documentation for details.
2017-07-04 13:04:22 +02:00
Marek Marczykowski-Górecki
6238254f49
events: merge fire_event and fire_event_pre functions
Those functions really do very similar things - lets merge them and add
simple parameter.
2017-07-04 13:04:21 +02:00
Marek Marczykowski-Górecki
9d17c52a8d
tests: add missing import 2017-07-04 05:22:39 +02:00
Marek Marczykowski-Górecki
f393ce62dc
Fix docstrings formatting
Fix warnings/errors reported by Sphinx
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
dd1e05dc83
vm: drop rename related methods
Since VM name in immutable now, drop rename-related methods.

QubesOS/qubes-issues#2868
2017-07-04 04:11:13 +02:00
Marek Marczykowski-Górecki
2dbb51a38d
exc/block: fix attaching block device exposed by dom0
Don't set <backenddomain> element in libvirt XML when device is provided
by dom0.
2017-07-04 04:09:28 +02:00
Marek Marczykowski-Górecki
3748eb3e2b
Merge branch 'core3-firewall2' 2017-07-04 03:38:59 +02:00
Marek Marczykowski-Górecki
f83c516082
Merge branch 'two-stage-clone' 2017-07-04 03:37:41 +02:00
Marek Marczykowski-Górecki
26a9974432
api/admin: split vm.volume.Clone to CloneFrom and CloneTo
The first operation returns a token, which can be passed to the second
one to actually perform clone operation. This way the caller needs have
power over both source and destination VMs (or at least appropriate
volumes), so it's easier to enforce appropriate qrexec policy.

The pending tokens are stored on Qubes() instance (as QubesAdminAPI is
not persistent). It is design choice to keep them in RAM only - those
are one time use and this way restarting qubesd is a simple way to
invalidate all of them. Otherwise we'd need some additional calls like
CloneCancel or such.

QubesOS/qubes-issues#2622
2017-06-26 13:27:48 +02:00
Marek Marczykowski-Górecki
3dcd29afea
api/admin: remove admin.vm.Clone operation
The same can be achieved with Create+volume.Clone

QubesOS/qubes-issues#2622
2017-06-26 13:10:00 +02:00
Marek Marczykowski-Górecki
a748b393f4
storage: move remove() to Volume
This is continuation of 0f12870 "storage: use direct object references,
not only identifiers".

QubesOS/qubes-issues#2256
2017-06-26 13:09:26 +02:00
Marek Marczykowski-Górecki
0200fdadcb
api/admin: firewall-related methods
In the end firewall is implemented as .Get and .Set rules, with policy
statically set to 'drop'. This way allow atomic firewall updates.

Since we already have appropriate firewall format handling in
qubes.firewall module - reuse it from there, but adjust the code to be
prepared for potentially malicious input. And also mark such variables
with untrusted_ prefix.

There is also third method: .Reload - which cause firewall reload
without making any change.

QubesOS/qubes-issues#2622
Fixes QubesOS/qubes-issues#2869
2017-06-26 13:08:27 +02:00
Marek Marczykowski-Górecki
65d15e6040
api/admin: skip firewall in vm.Clone
This operation is going to be removed, so apply a quick fix for tests.

QubesOS/qubes-issues#2622
2017-06-26 13:07:19 +02:00
Marek Marczykowski-Górecki
842efb577d
firewall: always use policy 'drop'
There is a problem with having separate default action ("policy") and
rules because it isn't possible to set both of them atomically at the
same time.
To solve this problem, always have policy 'drop' (as a safe default),
but by default have a single rule with action 'accept'

Fixes QubesOS/qubes-issues#2869
2017-06-26 13:07:19 +02:00
Marek Marczykowski-Górecki
a6c16d00be
vm/qubesvm: fix handling empty kernel value
Do not fail on empty kernel value - which means "use booloader on
root.img".

Fixes 3ddc052 "vm: move kernel presence validation to event handler"
2017-06-26 01:59:39 +02:00
Marek Marczykowski-Górecki
4208a98bd7
Merge branch 'core3-devel-20170619' 2017-06-23 02:53:17 +02:00
Marek Marczykowski-Górecki
c1f4c219f9
tests: adjust TC_00_QubesDaemonProtocol for reorganized api module 2017-06-21 06:59:58 +02:00
Marek Marczykowski-Górecki
ea0cbe3a56
tests: improve tests for qrexe exit code handling
Check if exit code retrieved from dom0 is really the one expected.

Fix typo in test_065_qrexec_exit_code_vm (testvm1/testvm2), adjust for
reporing remote exit code and remove expectedFailure.

QubesOS/qubes-issues#2861
2017-06-21 05:23:35 +02:00
Marek Marczykowski-Górecki
a73dcf6016
tests: wait for session in tests requiring running GUI
Since tests expose qubesd socket, qvm-start-gui should handle starting
GUI daemons (so, GUI session inside VM). Add synchronization with it
using qubes.WaitForSession service.
2017-06-21 04:45:46 +02:00
Marek Marczykowski-Górecki
376ac4b32d
tests: fix vm.run_for_stdio in some more places
When test expect to wait for remote process, use vm.run_for_stdio.
Additionally, when the call fail, (stdout, stderr) is not assigned - use
the one attached to exception object instead.
2017-06-21 04:33:10 +02:00
Marek Marczykowski-Górecki
a0f616f14e
tests: fix checking exit code
Since run_for_stdio raise an exception for non-zero exit code, it isn't
ignored anymore. So, check if qrexec-client-vm return expected value,
instead of keep ignoring it.

QubesOS/qubes-issues#2861
2017-06-21 00:17:43 +02:00
Marek Marczykowski-Górecki
72240c13b6
tests: fix vm_qrexec_gui/TC_10_Generic/test_000_anyvm_deny_dom0
When method (as expected) raise an exception, service output would not be
assigned. Extract it from exception object.
2017-06-21 00:17:43 +02:00
Marek Marczykowski-Górecki
a469c565f4
tests: fix QrexecPolicyContext
Flush new policy file to the disk, otherwise it will stay only in write
buffer.
2017-06-21 00:17:43 +02:00
Marek Marczykowski-Górecki
4241b39b94
tests: fix tests cleanup 2017-06-21 00:17:42 +02:00
Marek Marczykowski-Górecki
51a17ba749
tests: do not reload qubes.xml
In core3 this isn't needed anymore (and unit tests already check if
that's really true).
2017-06-21 00:17:42 +02:00
Marek Marczykowski-Górecki
ea5ca79133
tests: fix removing test VMs
Do it before shutting down qubesd socket - some things may require it
for VM removal/shutdown.
2017-06-21 00:17:41 +02:00
Marek Marczykowski-Górecki
eee6ab0c01
tests: use copy of qubes.xml, instead of empty one 2017-06-21 00:17:41 +02:00
Marek Marczykowski-Górecki
984a070f3e
tests: move create_*_file to SystemTestsMixin 2017-06-21 00:17:40 +02:00
Wojtek Porczyk
f56f7d13fb
tests/integ/vm_qrexec_gui: skip test_051_qrexec_simple_eof_reverse
QubesOS/qubes-issues#2851
2017-06-21 00:17:39 +02:00
Wojtek Porczyk
139f18fa1d
qubes/tests/integ/vm_qrexec_gui: some fixes 2017-06-21 00:17:39 +02:00
Wojtek Porczyk
0c0b0ea6ef
qubes/tests/integ/vm_qrexec_gui: change time.sleep to asyncio.sleep 2017-06-21 00:17:38 +02:00
Wojtek Porczyk
96a66ac6bd
qubes/api: refactor creating multiple qubesd sockets
Now there is a single function to do this, shared with tests.
2017-06-21 00:17:37 +02:00
Wojtek Porczyk
bec58fc861
qubes/tests: start qrexec policy responder for system tests 2017-06-20 13:00:20 +02:00
Wojtek Porczyk
4b8e5c3704
qubes/tests/run: refuse to run tests if qubesd is running
Test suite creates some VMs and needs to pass the knowledge about them
to qrexec policy checker. This is done using Admin API, so we need to
substitute qubesd with our own API server.
2017-06-20 13:00:20 +02:00
Wojtek Porczyk
858e547525
qubes: reorganise API protocols
Now instantiating API servers is handled by common function. This is,
among other reasons, for creating ad-hoc sockets for tests.
2017-06-20 13:00:20 +02:00
Marek Marczykowski-Górecki
9242202db2
admin: implement admin.vm.tag.*
QubesOS/qubes-issues#2622
2017-06-20 00:54:16 +02:00
Marek Marczykowski-Górecki
aadbe223c3
admin: add admin.vm.volume.Clone
QubesOS/qubes-issues#2622
2017-06-20 00:54:15 +02:00
Marek Marczykowski-Górecki
86a935e779
qubes.NotifyTools: ignore '/qubes-tools/version' completely
It isn't used for anything, so simply ignore it for good.

https://github.com/QubesOS/qubes-core-admin/pull/109#discussion_r121421409
2017-06-14 10:44:24 +02:00
Marek Marczykowski-Górecki
55669c350c
api/misc: fix retrieving requested features from QubesDB
qdb.list return list of bytes().
2017-06-14 10:44:24 +02:00
Marek Marczykowski-Górecki
160ab964bc
vm: improve sending events for tags and features
1. Send the event after setting tag/feature
2. Provide old value for feature
3. Rename 'key' kwarg to 'feature'
2017-06-14 10:44:23 +02:00
Marek Marczykowski-Górecki
93ccb8bbda
Merge remote-tracking branch 'qubesos/pr/111'
* qubesos/pr/111:
  vm: drop 'internal' property
  qmemman: make sure to release lock
  qmemman: fix meminfo parsing for python 3
  devices: drop 'data' and 'frontend_domain' fields, rename 'devclass' to 'bus'
2017-06-12 23:22:55 +02:00
Marek Marczykowski-Górecki
ba4da5cbe0
Merge remote-tracking branch 'qubesos/pr/110'
* qubesos/pr/110:
  storage: use direct object references, not only identifiers
  vm: fix volume_config
  storage/lvm: prefix VM LVM volumes with 'vm-'
  storage: fix VM rename
2017-06-12 23:22:41 +02:00
Marek Marczykowski-Górecki
37245acdcf
vm: change vm.updates_available to a 'updates-available' feature 2017-06-12 12:34:23 +02:00
Marek Marczykowski-Górecki
ee442c754f
api/misc: integrate qubes.NotifyTools logic with qubes.FeaturesRequest
Make qubes.NotifyTools reuse logic of qubes.FeaturesRequest, then move
actual request processing to 'features-request' event handler. At the
same time implement handling 'qrexec' and 'gui' features request -
allowing to set template features when wasn't already there.
Behavior change: template is no longer allowed to change feature value
(regardless of being True or False). This means the user will always be
able to override what template have set.
2017-06-12 12:22:39 +02:00
Marek Marczykowski-Górecki
9d99232515
vm: drop 'internal' property
It isn't used anywhere in the code right now. And when it will be
needed, it should be a "feature" not "property".
2017-06-12 10:06:43 +02:00
Marek Marczykowski-Górecki
3564250298
devices: drop 'data' and 'frontend_domain' fields, rename 'devclass' to 'bus'
Drop DeviceInfo.data - device extension should provide a subclass with
proper individual fields.
Drop DeviceAssignment.frontend_domain - this information is redundant -
frontend domain is defined by where DeviceAssignment is attached.
Rename DeviceCollection.devclass to bus - devclass if confusing here,
because this term is also used for DeviceInfo subclass.
2017-06-12 10:06:42 +02:00
Marek Marczykowski-Górecki
0f12870803
storage: use direct object references, not only identifiers
Reference objects, not their IDs - this way when object is modified, it
is visible everywhere where it is used. Main changes:
- volume.pool - Pool object
- volume.source - Volume object

Since volume have Pool object reference now, move volume related
functions into Volume class (from Pool class). This avoids horrible
`storage.get_pool(volume).something(volume)` construct.

One issue here is since volume.source reference a Volume object from a
different VM - VM's template, now VM load order is important. Since we
don't have control over it, initialize vm.storage when needed - possibly
while initializing storage of different VM. Since we don't have cycles
in AppVM-TemplateVM dependencies, it is safe.

Also, since this commit, volume.source (if defined) always points at
volume of the same name from VM's template. Using volumes with something
else as a source is no longer supported.

QubesOS/qubes-issues#2256
2017-06-09 04:55:06 +02:00
Marek Marczykowski-Górecki
fa1da422d1
vm/adminvm: don't base AdminVM on QubesVM
dom0 isn't real VM and most properties doesn't apply to it. Lets make it
more explicit.
2017-06-06 21:57:27 +02:00
Marek Marczykowski-Górecki
67fcda2cde
tests: basic tests for AppVM
Specifically, check if root volume is updated after template switch.
2017-06-06 03:33:31 +02:00
Marek Marczykowski-Górecki
313d7422e7
tests: remove obsolete tests, formatting 2017-06-06 03:33:31 +02:00
Marek Marczykowski-Górecki
28737d16ce
Migrate qubes.NotifyTools, qubes.FeaturesRequest, qubes.NotifyUpdates
Make them call into qubesd. Create separate socket for "misc" calls - VM
accessible, but not part of Admin API.
2017-06-06 03:33:30 +02:00
Marek Marczykowski-Górecki
1ed0b14d93
vm: make vm.hvm=True by default
QubesOS/qubes-issues#2185
2017-06-05 23:38:30 +02:00
Marek Marczykowski-Górecki
9bb5054e50
ext: BlockDevices extension
Handle block devices exposed by VMs
2017-06-05 23:33:58 +02:00
Wojtek Porczyk
d9f5192cc2 last fixes (thanks, @marmarek!) 2017-06-05 14:34:06 +02:00
Wojtek Porczyk
dc793be81f Fixes from marmarek's review 2017-06-01 12:31:14 +02:00
Wojtek Porczyk
cb3c9a82b3 qubes/tests: fix qrexec policy context
Now it removes policy file if it was created while entering the context.
2017-06-01 12:31:14 +02:00
Wojtek Porczyk
42cbd9ff68 qubes.tests asyncio, part 2
QubesOS/qubes-issues#2622
2017-05-29 19:08:38 +02:00
Wojtek Porczyk
df03800278 Fix miscellaneous warnings
Among them:
- explicitly close files (possibly using with: syntax)
- use non-deprecated methods
2017-05-29 17:34:15 +02:00
Wojtek Porczyk
b256af3bfb qubes.tests asyncio
QubesOS/qubes-issues#2622
2017-05-29 17:34:15 +02:00
Marek Marczykowski-Górecki
607dcbaf37
admin-api: fix handling admin.vm.property.Set with None VM value
Setting VMProperty to None VM should be encoded as '' value (according
to VMProperty._none_value). But value validation rejected this value.

QubesOS/qubes-issues#2622
2017-05-26 15:08:18 +02:00
Marek Marczykowski-Górecki
3177fc504f
tests: basic tests for admin.vm.volume.Import 2017-05-26 15:08:16 +02:00
Marek Marczykowski-Górecki
6c29c653a0
tests: improve qubes_basdir_patching
File storage pool use qubes.config.qubes_base_dir, not
qubes.config.system_path, so patch that too.
2017-05-26 15:08:15 +02:00
Marek Marczykowski-Górecki
64b83fa95a
admin: raise QubesNoSuchPropertyError for non-existing properties
Accessing non-existing property is a common action (for example
hasattr() do try to access the property). So, introduce specific
exception, inheriting from AttributeError. It will behave very similar
to standard (non-Admin-API) property access.

This exception is reported to the Admin API user, so it will be possible
to distinguish between non-existing property and access denied. But it
isn't any significant information leak, as list of valid properties is
publicly available in the source code.

QubesOS/qubes-issues#853
2017-05-23 04:32:59 +02:00
Marek Marczykowski-Górecki
f93583e2be
devices: adjust XML serialization of device options
Use '<option name="option_name">option_value</option>' instead of
'<options option_name="option_value"/>'. It's more consistent with the
rest of qubes.xml - have one thing per element.

Also, add options deserialization test.
2017-05-22 17:11:20 +02:00
Marek Marczykowski-Górecki
6bc44b43de
pci: adjust PCIDeviceExtension device identifier syntax
':' is not allowed in device identifier, replace it with '_'.

Warning: this breaks existing qubes.xml
2017-05-22 17:11:19 +02:00
Marek Marczykowski-Górecki
fe37e0933e
Devices-related formating fixes
QubesOS/qubes-issues#
2017-05-22 03:21:13 +02:00
Marek Marczykowski-Górecki
f7cb8f27c5
admin: add admin.vm.Remove method
QubesOS/qubes-issues#2622
2017-05-19 18:49:21 +02:00
Marek Marczykowski-Górecki
dc2563271c
tests: very basic tests for LinuxKernel pool 2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki
774f4c8d49
tests: admin.vm.devices.* Admin API
QubesOS/qubes-issues#2622
2017-05-16 09:05:22 +02:00
Marek Marczykowski-Górecki
661752bbc1
devices: list persistent devices also when not currently attached
When device extension do not return some "persistent" device as
currently attached, still return it, as it will be attached at next
domain startup. User can distinguish such devices by having
frontend_domain=None (or other VM).
Also, return a set from DeviceCollection.assignments().
2017-05-16 09:05:22 +02:00
Marek Marczykowski-Górecki
f3c7fb8129
admin: add missing parts to admin.vm.Clone
Clone all the things.
2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki
8de82f390e
tests: Tags() and Features() 2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki
ba86d6da79
vm: implement tag manager to fire events on change
While at it, adjust implementation to specification: tags don't have
value, only one bit of information (present/not present).

Fixes QubesOS/qubes-issues#2686
2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki
68a426f0ba
admin: add methods for global properties
QubesOS/qubes-issues#2622
2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki
0160d6e2f0
admin: fix handling default template in admin.vm.Create.* methods 2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki
fcfb2e004a
tests: more for admin.Events method 2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki
e598735c3b
tests: minor formatting 2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki
625c03c504
Rename MgmtAPI to AdminAPI - part 3: Admin API
QubesOS/qubes-issues#853
2017-05-12 19:28:08 +02:00
Marek Marczykowski-Górecki
cd489f46e1
Rename MgmtAPI to AdminAPI - part 1: classes
QubesOS/qubes-issues#853
2017-05-12 19:28:07 +02:00
Marek Marczykowski-Górecki
83eef56f9d
Remove tools that are moved to -client repository
QubesOS/qubes-issues#853
2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki
da3eed582b
tests: rearrage LVM thin tests to run at least some in devel env
This requires creating LVM volume group, so create on based on loop dev
in /tmp.
This is rather rough, but if any of this fails, run the tests anyway -
it will simply skip LVM tests.
2017-05-12 18:10:17 +02:00
Marek Marczykowski-Górecki
53be79b3b7
mgmt: vm.Create* and vm.Clone
Create methods are dynamically created based on available VM classes.

QubesOS/qubes-issues#2622
2017-05-12 17:04:18 +02:00
Marek Marczykowski-Górecki
52c3753d61
storage: support asynchronous storage pool implementations
Allow specific pool implementation to provide asynchronous
implementation. vm.storage.* methods will detect if given implementation
is synchronous or asynchronous and will act accordingly.
Then it's up to pool implementation how asynchronous should be achieved.
Do not force it using threads (`run_in_executor()`). But pool
implementation is free to use threads, if consider it safe in a
particular case.

This commit does not touch any pool implementation - all of them are
still synchronous.

QubesOS/qubes-issues#2256
2017-05-12 17:04:18 +02:00
Marek Marczykowski-Górecki
da7496794a
events: add support for per-instance handlers 2017-05-12 14:25:32 +02:00
Marek Marczykowski-Górecki
027df9567c
mgmt: add mgmt.label.Index method
QubesOS/qubes-issues#2622
2017-05-12 14:25:29 +02:00
Marek Marczykowski-Górecki
55037e0446
tests: tools.qubesd
For now only QubesDaemonProtocol test. main function still untested

QubesOS/qubes-issues#2622
2017-05-12 14:25:29 +02:00
Marek Marczykowski-Górecki
0b737b3a55
mgmt: vm.features handling (mgmt.vm.feature.*)
Actual implementation and tests.

QubesOS/qubes-issues#2622
2017-05-12 14:25:28 +02:00
Marek Marczykowski-Górecki
643cef1e65
mgmt: send artificial 'connection-established' event just after connection
This allows avoid race condition between  registering event handlers and
performing some action. The important thing is the event sent after
registering event handlers in qubesd. This means state changes (like
VM start/stop) after 'connection-established' event will be included in
event stream.

QubesOS/qubes-issues#2622
2017-05-12 14:25:28 +02:00
Marek Marczykowski-Górecki
8cc0af1eda
tests: mgmt.Events
QubesOS/qubes-issues#2622
2017-05-12 14:25:27 +02:00
Marek Marczykowski-Górecki
73df1b8ea7
tests: VM lifecycle methods tests
QubesOS/qubes-issues#2622
2017-05-12 14:25:27 +02:00
Marek Marczykowski-Górecki
2aa0de3d5b
tools: remove qvm-ls tool and related integration in qubes.property
qvm-ls tool (as all other tools) will be accessing properties through
API, so no need (nor sense) for this tool-specific attributes in
qubes.property. The only somehow used was ls_width, and in fact it made
the output unnecessary wide.

The tool itself is already moved to core-mgmt-client repository.

QubesOS/qubes-issues#853
2017-05-12 14:25:27 +02:00
Marek Marczykowski-Górecki
50b812190b
Merge remote-tracking branch 'kalkin/device-assignments' into core3-devel
* kalkin/device-assignments: (21 commits)
  PCI extension cache PCIDevice objects
  Make pylint ♥
  Fix pylint warning no-else-return
  Fix pylint warning len-as-conditional
  device-list-attached event returns a dev/options tupples list
  DeviceAssignment options are now a dict
  Remove WrongAssignment exception
  Rename qubes.devices.BlockDevice to qubes.storage.BlockDevice
  Update relaxng devices option element
  Fix tests broken by the new assignment api
  Fix qubes.tests.devices
  Fix pci device integration tests
  qvm-device add support for assignments
  Update ext/pci to new api
  BaseVM add DeviceAssignment xml serialization
  Migrate DeviceCollection to new API
  Add PersistentCollection helper to qubes.devices
  Add DeviceAssignment
  qvm-device validates device parameters
  qvm-device fix handling of non block devices
  ...
2017-04-21 17:09:33 +02:00
Bahtiar `kalkin-` Gadimov
79407a8717
Make pylint ♥ 2017-04-15 23:50:11 +02:00
Bahtiar `kalkin-` Gadimov
9da28c9c15
device-list-attached event returns a dev/options tupples list 2017-04-15 23:49:40 +02:00
Bahtiar `kalkin-` Gadimov
5a8cc9bdd3
Remove WrongAssignment exception
Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>
2017-04-15 23:49:39 +02:00
Bahtiar `kalkin-` Gadimov
b1b005964f
Fix tests broken by the new assignment api 2017-04-15 23:48:59 +02:00
Bahtiar `kalkin-` Gadimov
827ca283f3
Fix qubes.tests.devices
Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>
2017-04-15 23:48:59 +02:00
Bahtiar `kalkin-` Gadimov
e84114d3a9
Fix pci device integration tests
Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>
2017-04-15 23:48:59 +02:00
Marek Marczykowski-Górecki
0ada6d0b64
tests: disable GTK tests on travis
The environment there is way too old.

QubesOS/qubes-issues#910
2017-04-07 17:07:31 +02:00
Marek Marczykowski-Górecki
e76ede3ad0
tests: plug rpc-window tests into main test runner
QubesOS/qubes-issues#910
2017-04-07 17:07:30 +02:00
Marek Marczykowski-Górecki
564408eb3f
tests: qubespolicy tests
Fixes QubesOS/qubes-issues#2460
2017-04-06 15:43:17 +02:00
Marek Marczykowski-Górecki
3388054eae
mgmt: move property value sanitization to property definition
This also means we don't check if a VM with given name (in case of
VMProperty) exists in the system, at this stage. But this is ok, lets
not duplicate work of property setter.

QubesOS/qubes-issues#2622
2017-03-31 11:55:24 +02:00
Marek Marczykowski-Górecki
8371ffcd0f
tests: storage and labels
QubesOS/qubes-issues#2622
2017-03-16 20:04:07 +01:00
Marek Marczykowski-Górecki
2a223811a5
tests: add missing import 2017-03-16 20:04:06 +01:00
Marek Marczykowski-Górecki
f1c8f3220d
tests: deal with events containing dict in kwargs
If kwargs contains dict as one of values, it isn't hashable and can't be
used as value in frozenset/tuple. Convert such values into
frozenset(dict.items()). Only one (more) level is supported, but it
should be enough.

Solution from http://stackoverflow.com/a/13264725
2017-03-16 20:04:06 +01:00
Marek Marczykowski-Górecki
beaa0e9348
tests/mgmt: check if argument/payload is rejected when should be
Instead of creating such tests for each method separately, use unittest
subTest functionality to handle all of them at once.
2017-03-16 20:04:06 +01:00
Marek Marczykowski-Górecki
944bb26369
tests/mgmt: VM property related functions 2017-03-16 20:04:05 +01:00
Marek Marczykowski-Górecki
2d2672ec58
vm/qubesvm: convert firewall_conf into dumb, read-only property
Don't allow anything else than firewall.xml.
2017-03-16 20:04:03 +01:00
Marek Marczykowski-Górecki
a6c7da6061
tests: be even more defensive on cleaning up VMs
Don't fail even if qubes-test.xml do not load at all because of syntax
error - for example empty file.
2017-02-27 02:37:52 +01:00
Marek Marczykowski-Górecki
45709b510a
backup: minor fixes after bringing back scrypt support 2017-02-27 02:37:51 +01:00
Marek Marczykowski-Górecki
13fc810363
tests: some more fixes for core3 API 2017-02-27 02:37:50 +01:00
Marek Marczykowski-Górecki
3ecc0a9bcb
tests: improve devices API unit test
Check fired events - inspired by qvm-device test.
2017-02-27 02:37:50 +01:00
Marek Marczykowski-Górecki
7f2ca33774
tests: fix importing template in non-default pool 2017-02-27 02:37:50 +01:00
Marek Marczykowski-Górecki
9ace4e66f1
tests: more py3k related fixes 2017-02-27 02:37:49 +01:00
Marek Marczykowski-Górecki
cae68f64ca
tests: just one more test in vm_qrexec_gui to core3 API 2017-02-27 02:37:48 +01:00
Marek Marczykowski-Górecki
3f29345d32
tests/storage: read-only volume should not have save_on_stop=True 2017-02-27 02:37:48 +01:00
Marek Marczykowski-Górecki
9cad353939
tests: py3k related fixes - bytes/str
Adjust usage of bytes vs str type.
2017-02-27 02:35:49 +01:00
Wojtek Porczyk
be53db4db9 qubes/events: they accept only keyword arguments
Positional arguments are hereby deprecated, with immediate effect.

QubesOS/qubes-issues#2622
2017-02-21 14:46:42 +01:00
Marek Marczykowski-Górecki
bd9300b38e
tests: copy pool configuration into qubes-test.xml
If template choosen for the tests is installed in non-default storage
pool, this pool also needs to be imported into qubes-test.xml.
2017-02-15 00:01:33 +01:00
Marek Marczykowski-Górecki
c3fc4062d8
tests: add basic test for qvm-features 2017-02-15 00:01:33 +01:00
Marek Marczykowski-Górecki
01aedb7f18
storage: fix handling snap_on_start=True file volumes
Use the right cow image and apply the second layer to provide read-write
access. The correct setup is:
 - base image + base cow -> read-only snapshot (base changes "cached"
   until committed)
 - read-only snapshot + VM cow -> read-write snapshot (changes discarded
   after VM shutdown)

This way, even VM without Qubes-specific startup scripts will can
benefit from Template VMs, while VMs with Qubes-specific startup scripts
may still see original root.img content (for possible signature
verification, when storage domain got implemented).

QubesOS/qubes-issues#2256
2017-02-14 23:59:07 +01:00
Marek Marczykowski-Górecki
48f78dfbc8
tests: check if snap_on_start=True volumes are not persistent
Content should be reset back to base volume at each VM startup.
Disposable VMs depend on this behaviour.

QubesOS/qubes-issues#2256
2017-02-14 23:59:07 +01:00
Wojtek Porczyk
1be75d9c83 misc python3 fixes 2017-02-07 17:07:52 +01:00
Wojtek Porczyk
d74567d65f qubes: port core to python3
fixes QubesOS/qubes-issues#2074
2017-01-20 16:42:51 +01:00
Wojtek Porczyk
68ad60c1b3 Merge remote-tracking branch 'origin/master' into core3-devel
Conflicts:
	core/qubes.py
	doc/Makefile
	doc/manpages/qvm-prefs.rst
	doc/qvm-tools/qvm-add-appvm.rst
	doc/qvm-tools/qvm-backup-restore.rst
	doc/qvm-tools/qvm-backup.rst
	doc/qvm-tools/qvm-block.rst
	doc/qvm-tools/qvm-clone.rst
	doc/qvm-tools/qvm-firewall.rst
	doc/qvm-tools/qvm-ls.rst
	doc/qvm-tools/qvm-pci.rst
	doc/qvm-tools/qvm-run.rst
	doc/qvm-tools/qvm-shutdown.rst
	doc/qvm-tools/qvm-start.rst
	doc/qvm-tools/qvm-template-commit.rst
	qvm-tools/qvm-ls
	qvm-tools/qvm-prefs
	qvm-tools/qvm-remove
	tests/__init__.py
	vm-config/xen-vm-template.xml
2016-11-30 03:07:39 +01:00
Wojtek Porczyk
37bfd0d2a3 Merge remote-tracking branch 'marmarek/core3-fake-ip' into core3-devel 2016-11-15 17:40:30 +01:00
Wojtek Porczyk
a4d50409df Merge remote-tracking branch 'marmarek/core3-storage-fixes' into core3-devel 2016-11-15 17:36:53 +01:00
Wojtek Porczyk
fd953f4f27 Merge remote-tracking branch 'marmarek/core3-backup2' into core3-devel 2016-11-15 17:34:12 +01:00
Marek Marczykowski-Górecki
b011cef8af
tests/storage: add tests for basic volumes properties
Things like if read-only volume is really read-only, volatile is
volatile etc.

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
ec81b3046f
tests: add missing app.save() before starting a domain
Otherwise domain will be unknown to other processes (like qrexec
services).
2016-11-01 00:28:37 +01:00
Marek Marczykowski-Górecki
d999d91049
tests: few more tests for fake/custom IP
QubesOS/qubes-issues#1143
QubesOS/qubes-issues#1477
2016-10-31 03:39:46 +01:00
Marek Marczykowski-Górecki
ea33fef9cc
tests: drop dispvm_netvm tests
This property no longer exists in core3.
2016-10-31 03:10:12 +01:00
Marek Marczykowski-Górecki
5072acc8f2
tests: custom VM IP
QubesOS/qubes-issues#1477
2016-10-31 03:09:45 +01:00
Marek Marczykowski-Górecki
4585f2b503
tests: add tests for fake IP feature
QubesOS/qubes-issues#1143
2016-10-31 02:17:21 +01:00
Marek Marczykowski-Górecki
8cf19e3c92
tests/backupcompatibility: verify restored VM properties 2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
64ac7f6e8d
tests/backup: check non-ASCII passphrase
QubesOS/qubes-issues#2398
2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
fc00dd211e
tests/backup: test backup with non-ASCII passphrase 2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki
6ee200236c
tests/backup: verify migration into LVM thin pool 2016-10-28 11:53:31 +02:00
Marek Marczykowski-Górecki
673fe4423a
tests: handle LVM thin pool 2016-10-28 11:53:31 +02:00
Marek Marczykowski-Górecki
fbecd08a58
tests/backup: exclude some VMs during restore 2016-10-28 11:53:31 +02:00
Wojtek Porczyk
5e62d3f7cb qubes/tests: substitute_entry_points
New context manager for temporary overriding entry point groups.

fixes QubesOS/qubes-issues#2111
2016-10-24 15:16:39 +02:00
Wojtek Porczyk
4c73c1b93a More green paint 2016-10-19 16:09:58 +02:00
Marek Marczykowski-Górecki
5babb68031
tests/backupcompatibility: verify if all files got restored
There is still no verification of disk images content, nor VM
properties...
2016-10-19 01:54:44 +02:00
Wojtek Porczyk
8097da7cab Paint the project green for testbench launch 2016-10-18 19:07:20 +02:00
Wojtek Porczyk
c81346ba51 qubes/test/run: Print tracebacks of expected fails
fixes QubesOS/qubes-issues#2376
2016-10-14 17:20:14 +02:00
Wojtek Porczyk
526f2c3751 Merge remote-tracking branch 'marmarek/core3-backup' into core3-devel 2016-10-14 15:29:37 +02:00
Wojtek Porczyk
c6c0a545e6 Merge remote-tracking branch 'origin/pull/58/head' into core3-devel 2016-10-11 11:37:15 +02:00
Marek Marczykowski-Górecki
36eb7f923f
qubes/tarwriter: add simple sparse-tar writer module
tar can't write archive with _contents_ of block device. We need this to
backup LVM-based disk images. To avoid dumping image to a file first,
create a simple tar archiver just for this purpose.

Python is not the fastest possible technology, it's 3 times slower than
equivalent written in C. But it's much easier to read, much less
error-prone, and still process 1GB image under 1s (CPU time, leaving
along actual disk reads). So, it's acceptable.
2016-10-05 01:54:41 +02:00
Marek Marczykowski-Górecki
c4632d6be8
tests/backup: test idea 2016-09-29 02:08:30 +02:00
Marek Marczykowski-Górecki
f2d79b9379
tests/backup: use round volume size
When handling LVM volumes, size must be multiply of 4MB.
2016-09-29 02:08:30 +02:00
Marek Marczykowski-Górecki
e938aa61ab
tests: cleanup test LVM volumes
Handle the case when vm.remove_from_disk does not cleanup all the
things.
2016-09-29 02:08:30 +02:00
Marek Marczykowski-Górecki
0a35bd06aa
backup: support relocating files to different storage pool
To ease all this, rework restore workflow: first create QubesVM objects,
and all their files (as for fresh VM), then override them with data
from backup - possibly redirecting some files to new location. This
allows generic code to create LVM volumes and then only restore its
content.
2016-09-29 02:08:29 +02:00
Marek Marczykowski-Górecki
4d45dd5549
tests/backup: check backup+restore of LVM based VM
The test fails for now...
2016-09-29 01:59:55 +02:00
Marek Marczykowski-Górecki
226695534b
tests/backup: handle non-default pool in BackupTestsMixin 2016-09-29 01:59:54 +02:00
Marek Marczykowski-Górecki
e1d9de1cc2
tests/backup: minor fix for python3 2016-09-29 01:59:54 +02:00
Marek Marczykowski-Górecki
6d5959b31d
tests/backup: use proper logging instead of print 2016-09-29 01:59:53 +02:00
Marek Marczykowski-Górecki
016c3d8e88
tests/backup: check restored disk images 2016-09-29 01:59:53 +02:00
Bahtiar `kalkin-` Gadimov
8d9b6f19fd
Add catch-all '*' to event handlers 2016-09-28 21:28:43 +02:00
Marek Marczykowski-Górecki
e499b529ad
tests: move BackupTestMixin to qubes.tests.int.backup
This is much more logical place, don't pollute main qubes.tests module.
2016-09-25 16:31:31 +02:00
Marek Marczykowski-Górecki
d4f29bb8b7
qubes: add qvm-firewall tool
Fixes QubesOS/qubes-issues#1815
2016-09-19 20:36:32 +02:00
Marek Marczykowski-Górecki
202042bd8d
tests: update for new firewall API
QubesOS/qubes-issues#1815
2016-09-19 20:36:31 +02:00
Marek Marczykowski-Górecki
e01f7b97d9
qubes/vm: plug in new firewall code, create QubesDB entries
QubesOS/qubes-issues#1815
2016-09-19 20:36:31 +02:00
Marek Marczykowski-Górecki
1da75a676f
qubes/firewall: new firewall interface
First part - handling firewall.xml and rules formatting.
Specification on https://qubes-os.org/doc/vm-interface/

TODO (for dom0):
 - plug into QubesVM object
 - expose rules in QubesDB (including reloading)
 - drop old functions (vm.get_firewall_conf etc)

QubesOS/qubes-issues#1815
2016-09-19 20:36:30 +02:00
Marek Marczykowski-Górecki
0ea7afd245
tests: adjust LVM tests for non-default VG name 2016-09-19 20:35:25 +02:00
Marek Marczykowski-Górecki
9ae3dc2ebf
tests/qvm-device: tolerate different 'column' tool versions
Newer one do output column even if empty, but older one (in Travis-CI
env) do not. Ignore trailing spaces to work with both version.
2016-09-13 02:01:22 +02:00
Marek Marczykowski-Górecki
16db68b053
Merge branch 'core3-devices' into core3-devel
* core3-devices:
  Fix core2migration and tests for new devices API
  tests: more qubes.devices tests
  qubes/ext/pci: implement pci-no-strict-reset/BDF feature
  qubes/tools: allow calling qvm-device as qvm-devclass (like qvm-pci)
  qubes: make pylint happy
  qubes/tools: add qvm-device tool (and tests)
  tests: load qubes.tests.tools.qvm_ls
  tests: PCI devices tests
  tests: add context manager to catch stdout
  qubes/ext/pci: move PCI devices handling to an extension
  qubes/devices: use more detailed exceptions than just KeyError
  qubes/devices: allow non-persistent attach
  qubes/storage: misc fixes for VM-exposed block devices handling
  qubes: new devices API

Fixes QubesOS/qubes-issues#2257
2016-09-04 21:32:01 +02:00
Marek Marczykowski-Górecki
5cf15ef3da
tests: improve DispVM error message 2016-09-04 21:00:02 +02:00
Marek Marczykowski-Górecki
11191ea694
Fix core2migration and tests for new devices API
QubesOS/qubes-issues#2257
2016-09-04 20:55:07 +02:00
Marek Marczykowski-Górecki
e536bcca30
tests: more qubes.devices tests
QubesOS/qubes-issues#2257
2016-09-03 20:41:06 +02:00
Marek Marczykowski-Górecki
93e88e0c22
qubes/ext/pci: implement pci-no-strict-reset/BDF feature
Instead of old per-VM flag 'pci_strictreset', now implement this as
per-device flag using features. To not fail on particular device
assignment set 'pci-no-strict-reset/DEVICE-BDF' to True. For
example 'pci-no-strict-reset/00:1b.0'.

QubesOS/qubes-issues#2257
2016-09-03 20:41:06 +02:00
Marek Marczykowski-Górecki
5c7f589330
qubes: make pylint happy 2016-09-03 20:41:05 +02:00
Marek Marczykowski-Górecki
e8d011b83f
qubes/tools: add qvm-device tool (and tests)
Add a tool to manipulate various devices.

QubesOS/qubes-issues#2257
2016-09-03 20:41:05 +02:00
Marek Marczykowski-Górecki
6aae6863b0
tests: load qubes.tests.tools.qvm_ls
The test module was implemented, but not loaded
2016-09-03 20:41:04 +02:00
Marek Marczykowski-Górecki
fbb086aed5
tests: PCI devices tests
Just basic things for now.

QubesOS/qubes-issues#2257
2016-09-03 20:41:04 +02:00
Marek Marczykowski-Górecki
4bfb523974
tests: add context manager to catch stdout
This will avoid code duplication in tools tests.
2016-09-03 20:41:03 +02:00
Marek Marczykowski-Górecki
aa67a4512e
qubes/ext/pci: move PCI devices handling to an extension
Implement required event handlers according to documentation in
qubes.devices.

A modification of qubes.devices.DeviceInfo is needed to allow dynamic,
read-only properties.

QubesOS/qubes-issues#2257
2016-09-03 20:41:03 +02:00
Marek Marczykowski-Górecki
d7a3c0d319
qubes: new devices API
Allow device plugin to list attached and available devices. Enforce
at API level every device being exposed by some domain.

This commit only changes devices API, but not update existing users
(pci) yet.

QubesOS/qubes-issues#2257
2016-09-03 20:40:39 +02:00
Marek Marczykowski-Górecki
0293c1c7ef
qubes/vm: move misc XML tags loading to separate method 2016-08-18 11:57:11 +02:00
Marek Marczykowski-Górecki
d8286b6dab
tests: port vm_qrexec_gui to core3 API 2016-08-17 22:41:57 +02:00
Marek Marczykowski-Górecki
e19df4c5b6
tests: port dispvm tests to core3 API
Some tests do not apply, as there is no savefile and attributes
propagation is much simpler. Dropped tests:
 - test_000_firewall_propagation
 - test_001_firewall_propagation
 - test_000_prepare_dvm

QubesOS/qubes-issues#2253
2016-08-17 04:15:35 +02:00
Marek Marczykowski-Górecki
6b546ca1e6
tests/backupcompatibility: no errors are expected on restoring R1 backup
Fallback to default kernel is enabled by default.
2016-08-17 01:31:33 +02:00
Marek Marczykowski-Górecki
690727fef8
tests/backup: remove VMs in reversed order
One place had missing `reversed()`.
2016-08-17 01:30:34 +02:00
Marek Marczykowski-Górecki
dd93650ea7
tests/backup: fix handling different VM types
Not all VM types have 'template' property.
2016-08-17 00:49:53 +02:00
Marek Marczykowski-Górecki
660c726315
tests: add qvm-check 2016-08-09 05:14:13 +02:00
Marek Marczykowski-Górecki
96d032919a
tests: minor fix to revert changes tests, mark as expected failure
While the test should be ok now, the functionality itself doesn't work (at
least on file volumes).
2016-08-09 05:14:12 +02:00
Marek Marczykowski-Górecki
ac68ffe39f
tests: correctly compare DeviceManager objects
Usefull for assertEqual(vm1.devices, vm2.devices) - clone tests, backup
tests etc.
2016-08-09 05:14:11 +02:00
Marek Marczykowski-Górecki
ec7a84ea9d
minor: formatting
QubesOS/qubes-issues#
2016-08-09 05:14:10 +02:00
Marek Marczykowski-Górecki
af7b4087f0
tests: port few more tests to core3 API 2016-08-09 05:14:10 +02:00
Marek Marczykowski-Górecki
faa341ffa2
tests/backup: adjust for core3 API properties to verify after restore 2016-08-09 05:14:10 +02:00
Marek Marczykowski-Górecki
7d1cbd6934
tests: handle per-template tests in BackupTestsMixin 2016-08-09 05:14:09 +02:00
Marek Marczykowski-Górecki
fba6eac07f
qubes/tests: do not undefine libvirt domain twice
Do not access vm.libvirt_domain after it being already removed - this
will redefine it again in libvirt, just to undefine it in a moment.
On the other hand, few lines below there is fallback libvirt cleanup, in
case of proper one not working.
2016-08-09 05:14:09 +02:00
Marek Marczykowski-Górecki
6d45b97357
tests: workaround 'int' name being shadowed by module name
qubes.tests.int shadows 'int' type in qubes.tests module.

QubesOS/qubes-issues#2241
2016-08-09 05:14:09 +02:00
Marek Marczykowski-Górecki
4e022382a5
Merge remote-tracking branch 'origin/master' into core3-devel 2016-08-08 00:11:46 +02:00
Bahtiar `kalkin-` Gadimov
7d11a7afbd
Add lvm tests 2016-08-03 02:57:41 +02:00
Wojtek Porczyk
f915115cfb Merge remote-tracking branch 'woju/pull/25/head' into core3-devel 2016-07-21 19:38:58 +02:00
Wojtek Porczyk
36e5bcd766 Merge remote-tracking branch 'origin/pull/39/head' into core3-devel 2016-07-21 16:43:25 +02:00
Bahtiar `kalkin-` Gadimov
71161bfb93
Add mock libvirt objects 2016-07-13 23:28:02 +02:00
Bahtiar `kalkin-` Gadimov
d380fb4aba
qubes.tests.app make pylint happy ♥ 2016-07-13 23:08:59 +02:00
Bahtiar `kalkin-` Gadimov
23e52775b6
qubes.tests.init make pylint happy ♥ 2016-07-13 23:08:58 +02:00
Bahtiar `kalkin-` Gadimov
61feb0ced7
Migrate backup to new storage api 2016-07-13 22:34:37 +02:00
Bahtiar `kalkin-` Gadimov
d1c606b952
qubes.storage.file use new storage API 2016-07-13 22:34:33 +02:00
Bahtiar `kalkin-` Gadimov
ca9797bb6b
qubes.tests.int.basic use new storage API 2016-07-13 22:34:32 +02:00
Bahtiar `kalkin-` Gadimov
4cc7b8d2a8
Fix qubes.tests.storage 2016-07-13 22:21:31 +02:00
Bahtiar `kalkin-` Gadimov
774cbd499e
Add TestVM.is_halted() to fix failing tests 2016-07-13 21:57:12 +02:00
Wojtek Porczyk
c899d1f960 qubes/tests: Show errors while loading external tests
Now failure to load external tests shows in which entry point the error
happened and a useful traceback. The traceback extends from the "try"
statement down to the actual error line, but it does not include the
frames above, ie. from the invocation to the load_tests routine. This is
a limitation of Python itself and usually not a problem.
2016-07-04 16:51:35 +02:00
Wojtek Porczyk
d28ab908cc Network and storage fixes 2016-06-29 16:42:35 +02:00
Bahtiar `kalkin-` Gadimov
e08ca1ff57
Remove QubesVM.is_template() 2016-06-23 14:23:31 +02:00
Wojtek Porczyk
66f46d8a25 Fix test error after bdfb85ac (refactor storage) 2016-06-21 10:10:45 +02:00
Wojtek Porczyk
d6ad8d34a6 Merge remote-tracking branch 'marmarek/patch-1' into core3-devel 2016-06-16 21:46:53 +02:00
Marek Marczykowski-Górecki
dcdb62721b
tests: mark TC_01_FileVolumes.test_003_read_volume with expected failure 2016-06-16 21:23:18 +02:00
Marek Marczykowski-Górecki
4e797663e9
tests: make storage tests working outside of dom0 2016-06-16 21:14:35 +02:00
Marek Marczykowski-Górecki
c9a55cc198
tests: use offline mode
QubesOS/qubes-issues#2008
2016-06-16 17:04:16 +02:00
Bahtiar `kalkin-` Gadimov
930fe417a8 Remove storage size and usage methods from QubesVM 2016-06-16 16:59:26 +02:00
Wojtek Porczyk
6a10daf7be Merge branch 'master' into core3-devel
Remains to be fixed:
  88cb62fc
  d2640b51
  958c2926
2016-06-13 19:03:46 +02:00
Wojtek Porczyk
6ade5736d7 pylint fixes 2016-06-10 21:27:29 +02:00
Wojtek Porczyk
3b08e85a2d qubes/tests/init: Fix VMCollection reference 2016-06-10 21:27:29 +02:00
Wojtek Porczyk
27d0e11872 qubes/tests/vm: fix tests 2016-06-02 15:46:01 +02:00
Marek Marczykowski-Górecki
485e75091b tests: fix checking rename 2016-06-02 13:22:43 +02:00
Marek Marczykowski-Górecki
077dec8d2e tests: fix rename test 2016-06-02 13:22:40 +02:00
Marek Marczykowski-Górecki
28591fa6be tests: remove/skip some tests
1. It is unclear yet whether dispvm_netvm will be implemented in core3, but
probably not.
2. Remove tests for setting memory/cpu above host resouces - rejecting
those values at property set time would break backup restore on some
machines (when migrating from bigger to smaller system).
2016-06-02 13:09:39 +02:00
Marek Marczykowski-Górecki
bf0966bfcf tests: fix get_label mockup to return meaningful values
Reject invalid argument.
2016-06-02 13:08:48 +02:00
Marek Marczykowski-Górecki
98115eb541 tests/backup: fix to core3 API - there is no verbose= argument 2016-06-02 13:08:03 +02:00
Marek Marczykowski-Górecki
07b72ef3fd tests: fix after moving VMCollection to qubes.app 2016-06-02 12:37:30 +02:00
Marek Marczykowski-Górecki
c965024287 qubes/vm: Implement Disposable VM
Implement DispVM as a VM based on AppVM.

QubesOS/qubes-issues#866
2016-06-02 12:37:19 +02:00
Wojtek Porczyk
786884ad7a qubes: fix netvm properties and tests
fixes QubesOS/qubes-issues#1816
2016-05-19 03:02:23 +02:00
Wojtek Porczyk
e3aae7bc17 qubes: Minor fixes
Fix mock TestApp object and argument parsing in qvm-run.
2016-05-05 16:58:43 +02:00
Wojtek Porczyk
c76790fbde qubes/tests/vm/qubesvm: fix test for vm.internal
On flipping .internal vm (re)creates appmenus. They need label.
2016-05-05 14:38:04 +02:00
Wojtek Porczyk
63c09a090c qubes: Combat import cycles
This commit eliminates import statements happening in the middle of the
file (between two classes definition). The cycles are still there. The
only magic module is qubes itself.
2016-05-05 14:33:09 +02:00
Bahtiar `kalkin-` Gadimov
7200e6153b
Rename default storage driver from xen to file
- Rename XenPool   ⇒ FilePool
- Rename XenVolume ⇒ FileVolume
2016-04-30 20:42:46 +02:00
Bahtiar `kalkin-` Gadimov
e3ae6cdc1b BackupTestsMixin.create_backup_vms uses volumes
Instead of using root_img to access the path it uses now the proper volumes
2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
d7fd66070a Fix revert template changes test 2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov
ef485ca32a Add linux-kernel to defaults['pool_config'] 2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov
3c66d4b54c Fix storage_xen test 2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov
79ac3d3770 Fix storage test and simplify TestVM 2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov
c3d8c899cc Add TemplateVM test for storage_xen 2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov
bd4674b658 Remove obsolete tests 2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov
d1d72466d6 Remove unneeded imports from tests/int.basic 2016-04-20 13:59:55 +02:00
Marek Marczykowski-Górecki
7117847816
tests: fix int.dom0_update
Environment must be preserved for QUBES_XML_PATH variable used to point
to a secondary (test) qubes.xml.
2016-04-20 02:26:56 +02:00
Marek Marczykowski-Górecki
a707840596
tests: fix and enable external tests loader
QubesOS/qubes-issues#1800
2016-04-20 02:26:16 +02:00
Marek Marczykowski-Górecki
a1575b98d4
tests: do not drop 'qubes.tests.' prefix
This cause troubles with "external" tests (loaded from outside of
qubes.tests module).
2016-04-20 02:25:56 +02:00
Marek Marczykowski-Górecki
eaf5efd814
tests: convert 'extra' test loader to core3 API
QubesOS/qubes-issues#1800
2016-04-11 13:50:19 +02:00
Marek Marczykowski-Górecki
8c6fe7ed90
Merge remote-tracking branch 'origin/master' into core3-devel-mm 2016-04-11 13:03:12 +02:00
Wojtek Porczyk
c5962910d0 qubes/events: Rework firing events for effect
From now, the handlers should yield their values, not return.
2016-04-08 12:35:11 +02:00
Marek Marczykowski-Górecki
20d53fbf69 tests: more VMs combinations in backup
VM with custom template, non-default NetVM

This requires to VMs being removed in reversed order (to first remove
VM, then its template).
2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
424d3054f3 backup: use vm.features to store backup-specific metadata
Do not keep them in system qubes.xml.
2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
c55ca8004a tests: port backupcompatibility to core3 API 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
44eed25511 tests/backup: report failure when error was expected but none detected 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
3342f637f4 backup: convert logging to proper python logging API
QubesOS/qubes-issues#1213
QubesOS/qubes-issues#1214
2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
019678bf4d tests: fix backup test to not break default template...
By default AppVM is template based. This means vm.root_img points at
default template's root image. Change this to StandaloneVM to have
independent root.img.
2016-04-07 13:06:25 +02:00
Marek Marczykowski-Górecki
e557fe9989 backup: restructude the code into classes
Introduce two main classes Backup and BackupRestore for storing the
state of the desired operation. Then a simple interface to adjust
parameters.

(Almost) no functional change.

QubesOS/qubes-issues#1213
QubesOS/qubes-issues#1214
2016-04-07 13:06:14 +02:00
Marek Marczykowski-Górecki
fd1f8def6a tests: do not store dom0 home backup inside of dom0 home itself 2016-04-07 13:05:30 +02:00
Marek Marczykowski-Górecki
e0686e1e02 backup: initial conversion to core3 API 2016-04-07 13:05:25 +02:00
Marek Marczykowski-Górecki
12df07edd0 tests: invalid property should not be saved to qubes.xml 2016-04-07 13:03:42 +02:00
Marek Marczykowski-Górecki
c07981da88 tests: allow real prefix notation for selecting tests 2016-04-07 13:03:35 +02:00
Marek Marczykowski-Górecki
7f5feac98b qubes/vm: fixes for features handling
Properly handle 'None' value, there is no difference between empty
string (<x></x>) and no content at all (<x/>).
2016-04-07 13:03:30 +02:00
Marek Marczykowski-Górecki
06c315c94b qubes: fix/update event names
Fix few typos, then rename:
 - recreate-appmenus -> domain-restore: there may be more to do than
 only appmenus
2016-04-07 12:59:39 +02:00
Marek Marczykowski-Górecki
6052143735 tests: qvm-prefs
QubesOS/qubes-issues#1248
2016-04-07 12:57:57 +02:00
Marek Marczykowski-Górecki
43435405e7 tests: update checking 'None' VM representation 2016-04-07 12:57:47 +02:00
Bahtiar `kalkin-` Gadimov
b1978abce5 Use entry_points for pool driver discovery
- Add qubes.storage entry point to setup.py
- Removed the old pool driver class loading logic
- Reworked pool tests
2016-04-06 10:11:19 +02:00
Marek Marczykowski-Górecki
54784405bd tests: adjust networking tests 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
34fe4a2d71 tests: add function to provide real network to test case
Very few tests should use it, as it makes the test result somehow
dependent on outside world state. But still, some of them need it.
2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
422f30b969 tests: port network tests to core3 API 2016-03-21 11:43:32 +01:00
Marek Marczykowski-Górecki
df5562d2c7 tests: fix rename test to report FAIL instead of ERROR when really failed 2016-03-21 11:43:32 +01:00
Wojtek Porczyk
394fa1e5d3 qubes/tests: tests for qubes.devices 2016-03-21 11:43:32 +01:00
Wojtek Porczyk
d09bd5ab6a qubes: Convert QubesVM and Extension discovery to pkg_resources
QubesOS/qubes-issues#1238
2016-03-21 11:43:32 +01:00
Wojtek Porczyk
93686eae06 qubes/vm: change services to features 2016-03-21 11:43:32 +01:00
Marek Marczykowski-Górecki
5dca051b32 tests: enable storage tests 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
c0c0e0022e tests: convert storage tests to core3 API
QubesOS/qubes-issues#
2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
14f31134c0 tests: move make_vm_name to QubesTestCase
This function is often useful also in unit tests, not only integration
tests.
2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
68ed50c97d tests: enable dom0 update tests 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
fb74126e56 tests: convert dom0 update test to core3 API 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
0307ec22f2 tests: resolve template name when copying from host collection 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
81dbe3628f tests: skip system tests if not in dom0 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
50819c726c tests: set QUBES_XML_PATH env variable to force usage of qubes-test.xml
This way even separate processes (even those started not directly - like
qrexec service calls) will use correct qubes.xml file.

Fixes QubesOS/qubes-issues#1730
2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
79a6da1163 tests: split off invalid values tests for better reporting
Most of them fails for now...
2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
c45c91c5f0 tests: improve verbosity of assertNotRaises even further
Include traceback in error message.
2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
5a409ad417 tests: few more (disabled for now) tests for hvm related properties 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
1b69f5c213 tests: make test cleanup more defensive - qubes.xml load failure
If qubes.xml doesn't even load, remove it anyway and clean up the VMs
the hard way.
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
9633573408 core: adjust units comment for host.memory_total, and fix related places
QubesOS/qubes-issues#1737
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
12941d67bf tests: do not unnecessary mangle test name passed as runner parameter
Otherwise `qubes.tests.run -l` output would not be useful as input for
test list.
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
05d006b873 tests: minor fixes in int.tools tests 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
163ab25d4b tests: enable qubes.tests.int.basic
It is fully converted to core3 API (or tests marked as 'skip' if not),
so can be reenabled.
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
9b1b82c592 tests: move properties tests to QubesVM unit tests
Since qvm-prefs is now a generic tool without any property-specific
stuff, we can tests properties getters/setters in QubesVM class unit
tests.
Now most of properties are tested. Some of those tests still fails
because missing/incomplete underlying implementation.
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
81bc615bce tests: convert tests/int/basic to core3 API
Not all tests converted - some of them will be removed in the next
commit. Also it doesn't mean that all tested functionality already
works...
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
2daba9e0eb core: migrate handy GUI functions and backup mixin from core2
That's all from tests/__init__.py, so remove that file.
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
bc82643580 tests: redo "84af738 tests: support VMs cleaned up in tearDownClass..."
Usage: VMs with name created by self.make_vm_name(name,
class_teardown=True) will be cleaned up in tearDownClass. It
should be used only in setUpClass.

Fixes QubesOS/qubes-issues#1691
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
0a45164be7 tests: few more FD leak fixes
QubesOS/qubes-issues#1697
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
47e346efbf tests: initialize Qubes() object in SystemTestsMixin, add handy helpers
Most of the tests needs existing template, so add helper to copy it from
"host" collection. Add also helpers to reload qubes.xml - in case of
modification of it from some other process.
2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
2966456387 tests: improve assertNotRaises reporting
Include whole exception value, not only its type.
2016-03-03 01:18:16 +01:00
Wojtek Porczyk
9eafa57539 Merge remote-tracking branch 'marmarek/master' into core3-devel
The following list is bollocks. There were many, many more.

Conflicts:
    core-modules/003QubesTemplateVm.py
    core-modules/005QubesNetVm.py
    core/qubes.py
    core/storage/__init__.py
    core/storage/xen.py
    doc/qvm-tools/qvm-pci.rst
    doc/qvm-tools/qvm-prefs.rst
    qubes/tools/qmemmand.py
    qvm-tools/qvm-create
    qvm-tools/qvm-prefs
    qvm-tools/qvm-start
    tests/__init__.py
    vm-config/xen-vm-template-hvm.xml

This commit took 2 days (26-27.01.2016) and put our friendship to test.
    --Wojtek and Marek
2016-03-03 01:13:51 +01:00
Wojtek Porczyk
638c31ce68 qubes/tests/vm/qubesvm: some more tests 2016-01-21 13:31:43 +01:00
Wojtek Porczyk
06cc064c8c qubes/tests: split init to init1 and init2
This is to keep the correct order of the tests. The exact dependency
graph is somewhat complicated and contains several cycles.
2016-01-21 13:31:43 +01:00
Wojtek Porczyk
689dd64b0d qubes: misc pylint fixes 2015-12-29 22:04:00 +01:00
Wojtek Porczyk
bf4dbe07d5 qubes/tests/vm/qubesvm: add basic tests for QubesVM
And already one fix to instantiation.
2015-12-29 20:35:04 +01:00
Wojtek Porczyk
66475e5ca7 qubes/tests/run: Fix running foregin TestCases
This is needed, because when tests fail to load, unittest module
generates artificial „LoadTestsFailure” class -- not based on our
QubesTestCase, so it does not have .log attribute.
2015-12-29 20:08:47 +01:00
Wojtek Porczyk
13b1a7201a qubes/tests: fix tearing down QubesTestCase 2015-12-29 16:49:17 +01:00
Wojtek Porczyk
ce0b927a98 qubes/tests/int/tools/qvm_run: add tests for qvm-run
Missing is test for --gui/--no-gui.

part of QubesOS/qubes-issues#1226
2015-12-29 03:50:35 +01:00
Wojtek Porczyk
f1a0b1af39 qubes/tools: add qvm-run, qvm-{,un}pause
Also change convention of calling main(): now command returns its
numeric value instead of bool.

Also fixed QSB#13

fixes QubesOS/qubes-issues#1226
2015-12-29 03:43:08 +01:00
Wojtek Porczyk
ec50a411fe qubes/tests/int/tools/qubes_create: fix wrong assert 2015-12-23 14:34:54 +01:00
Wojtek Porczyk
5c57b4cda7 qubes/tests: fix regression in test setup
Regression was caused by 648fe213 "core3: test fixing"
2015-12-23 14:34:54 +01:00
Wojtek Porczyk
0dc0fd306f core3: test fixing
part of QubesOS/qubes-issues#1248
2015-12-23 14:34:53 +01:00
Wojtek Porczyk
dc3fcc3141 qubes/tests: port again the backported test runner
part of QubesOS/qubes-issues#1248
2015-10-17 00:20:44 +02:00
Wojtek Porczyk
96efb4568a core3: add different exceptions
From now on there are different exceptions which can be raise on
different occasions.

fixes QubesOS/qubes-issues#1279
2015-10-17 00:17:12 +02:00
Wojtek Porczyk
ea44c0acf3 qubes: pylint fixes
Fix bunch of errors and warnings.
2015-10-05 23:49:39 +02:00
Wojtek Porczyk
17ac6cb225 core3: boolean properties can be set explicitely 2015-10-05 12:46:14 +02:00
Wojtek Porczyk
b4d51b016b core3: some properties can be set only once
Some properties should not be changed by user at will (like UUID). The
solution is to make them write-once, so they will be set when loading
from XML and frozen for the lifespan of the object holding the property.
When desperately needed, users may edit XML by hand.

fixes QubesOS/qubes-issues#1235
2015-10-05 12:46:14 +02:00
Wojtek Porczyk
12329e7b35 qubes/vm/qubesvm: fix log initialisation 2015-10-05 12:46:14 +02:00
Wojtek Porczyk
6241859dc8 qubes: Fix PropertyHolder.property_get_default
And test cases, some of them stubs.
2015-09-28 18:20:28 +02:00
Wojtek Porczyk
669a976d4e qubes/tools: add common action for setting properties 2015-06-29 17:40:43 +02:00
Wojtek Porczyk
7f16e0a73b qubes: disable events by default not to interfere with loading from XML 2015-06-29 17:39:28 +02:00
Wojtek Porczyk
d5f2e6ab74 qubes: rework qubes.xml handling
Now it more closely aligns with old core2 semantics. New semantics will
have to wait for qubesd.
2015-06-29 17:39:28 +02:00
Wojtek Porczyk
6a4820c381 qubes/tools: qvm-ls 2015-06-29 17:39:28 +02:00
Wojtek Porczyk
5d9b92a039 qubes: pylint fixes (minor programming errors) 2015-06-29 17:39:28 +02:00
Wojtek Porczyk
45977fc873 qubes: fix VM instantiation and loading 2015-06-29 17:39:27 +02:00
Wojtek Porczyk
6798790e1f qubes: pylint fixes (fix signatures) 2015-06-29 17:39:27 +02:00
Wojtek Porczyk
a686e810b9 qubes/tests: enhance test runner report 2015-06-29 17:39:27 +02:00
Wojtek Porczyk
adb144acfe qubes: pylint fixes (small mistakes and wrong names) 2015-06-29 17:39:27 +02:00
Wojtek Porczyk
4a85c823c3 qubes: pylint fixes (qualify imported variables) 2015-06-29 17:39:27 +02:00
Wojtek Porczyk
bf29d5e5b5 qubes: pylint fixes (disable unfounded messages) 2015-06-29 17:39:27 +02:00
Wojtek Porczyk
ac1739df6b qubes: pylint fixes (fix imports) 2015-06-29 17:39:27 +02:00
Wojtek Porczyk
ee06e7d7a2 qubes: documentation and licence fixes 2015-06-29 17:39:27 +02:00
Wojtek Porczyk
cdc3df66c8 qubes: pylint fixes (mostly whitespace) 2015-06-29 17:39:27 +02:00
Wojtek Porczyk
8e16dd2b28 qubes/storage: fix storage instantiation 2015-06-29 17:39:26 +02:00
Wojtek Porczyk
8805db5e5f core3 move: AdminVM class 2015-06-29 17:39:26 +02:00
Wojtek Porczyk
a13a41fbaf qubes/tests: fix testrunner dependency on being run in specific directory 2015-06-29 17:39:26 +02:00
Wojtek Porczyk
e5d2b49fd6 qubes/tests: fix colourful testrunner 2015-06-29 17:39:26 +02:00
Wojtek Porczyk
eda2f7cf73 qubes/tests: mark one test as expected failure
This is temporary, to be fixed in the future.
2015-06-29 17:39:26 +02:00
Wojtek Porczyk
af154b53fe qubes: change names of XML generating methods
Methods returning lxml.etree.Elements are called xml_ or lvxml_, meant for
qubes.xml or libvirt respectively.

Acknowledgement:
This commit is a result of core3 review by Marek.
2015-06-29 17:39:26 +02:00
Wojciech Zygmunt Porczyk
42d8e67556 doc: Add autogenerated qubes.xml documentation 2015-06-29 17:39:25 +02:00
Wojciech Zygmunt Porczyk
118edb6ac4 First RelaxNG schema of qubes.xml
Schema is most likely incomplete.
2015-06-29 17:39:25 +02:00
Wojtek Porczyk
6ec86ec9f7 qubes: property may be unset by assinging DEFAULT
Introducing qubes.property.DEFAULT special value, which may be assigned to any
property. Result is the same as del'ing a property.
2015-06-29 17:39:25 +02:00
Wojtek Porczyk
713ced8cd2 qubes/tests: add event asserts for qubes module 2015-06-29 17:39:25 +02:00
Wojtek Porczyk
74c3126b80 qubes/tests: add some tests for qubes.property 2015-06-29 17:39:25 +02:00
Wojtek Porczyk
eabc571102 qubes/tests: colourful test runner 2015-06-29 17:39:24 +02:00
Wojtek Porczyk
de9eb60f61 Developer's documentation for qubes.tests 2015-06-29 17:39:24 +02:00
Wojtek Porczyk
8ace0fa634 qubes/tests: skipping tests outside dom0
New variable and decorator enable skipping tests outside dom0, that is, without
connection to libvirtd.
2015-06-29 17:39:24 +02:00
Wojtek Porczyk
613b03d277 qubes/tests: Common TestEmitter class
qubes.tests.TestEmitter is intended to check whether specific event fired on
given emitter.
2015-06-29 17:39:24 +02:00
Wojtek Porczyk
a82bf7cc54 qubes/tests: Move unit tests inside qubes/, add runner 2015-06-29 17:39:24 +02:00