Commit Graph

5110 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
0b0cd41dc6
qubes-hcl-report: detect AMD interrupt remapping
There is slightly different message in xl dmesg.

Fixes QubesOS/qubes-issues#3208
2017-12-05 17:39:33 +01:00
Marek Marczykowski-Górecki
d54cef5554
app: fix creating dom0 object when not already present in qubes.xml
It's constant properties are now really constant, no need to provide
them explicitly.
2017-12-05 17:39:33 +01:00
Marek Marczykowski-Górecki
0afee4b05e
Improve checking for netvm loop
There were many cases were the check was missing:
 - changing default_netvm
 - resetting netvm to default value
 - loading already broken qubes.xml

Since it was possible to create broken qubes.xml using legal calls, do
not reject loading such file, instead break the loop(s) by setting netvm
to None when loop is detected. This will be also useful if still not all
places are covered...

Place the check in default_netvm setter. Skip it during qubes.xml loading
(when events_enabled=False), but still keep it in setter, to _validate_ the
value before any property-* event got fired.
2017-12-05 17:39:33 +01:00
Marek Marczykowski-Górecki
2d830caab9
app: check DEFAULT_LVM_POOL variable for default pool
Allow to hint default_pool what is default storage pool - especially
useful for tests.
2017-12-05 17:39:33 +01:00
Marek Marczykowski-Górecki
749e8497e3
api/admin: exclude regex attribute from DeviceInfo structure
DeviceInfo may contain 'regex' attribute - it isn't intended to be
reported through Admin API. Also, mark 'libvirt_regex' attribute as
private.
2017-12-05 17:39:32 +01:00
Marek Marczykowski-Górecki
da97f4d84c
qubesvm: make initial qmemman request consistent with libvirt config
If HVM have PCI device, it can't use PoD, so need 'maxmem' memory to be
started. Request that much from qmemman.
Note that is is somehow independent of enabling or not dynamic memory
management for the VM (`service.meminfo-writer` feature). Even if VM
initially had assigned maxmem memory, it can be later ballooned down.

QubesOS/qubes-issues#3207
2017-12-05 17:39:32 +01:00
Marek Marczykowski-Górecki
19a1579a99
tests: fix deadlock in filecopy test
Error window (where test need to send Enter key) is opened while
qvm-move-to-vm is still running.
2017-12-05 17:39:32 +01:00
Marek Marczykowski-Górecki
99874a0a25
tests: make waiting for window asyncio aware
For now just replace sleep with asyncio.sleep. Later it may make sense
to change subprocess.call too.
2017-12-05 17:39:32 +01:00
Wojtek Porczyk
bdaf92f9dc qubes-rpc/policy.RegisterArgument: readability fixes
Thanks, @rootkovska and @marmarek.
2017-11-30 16:05:06 +01:00
Wojtek Porczyk
61c164e1c3 qubes-rpc: policy.RegisterArgument
This qrexec is meant for services, which require some kind of
"registering" before use. After registering, the backend should invoke
this call with frontend as the intended destination, with the actual
service in argument of this call and the argument as the payload.

By default this qrexec is disabled by policy.

Signed-off-by: Wojtek Porczyk <woju@invisiblethingslab.com>
2017-11-30 00:26:21 +01:00
Marek Marczykowski-Górecki
d4af3b5d9f
version 4.0.14 2017-11-21 13:02:22 +01:00
Marek Marczykowski-Górecki
2cc5e4ce9a
qmemman: remove cyclic service ordering
Fixes QubesOS/qubes-issues#2224
2017-11-21 13:01:51 +01:00
Marek Marczykowski-Górecki
6f1cedff80
version 4.0.13 2017-11-21 11:28:57 +01:00
Jean-Philippe Ouellet
b41c118166
Fix typo in volume import end handler 2017-11-21 01:05:06 -05:00
Marek Marczykowski-Górecki
ac8980a903
version 4.0.12 2017-11-21 05:12:06 +01:00
Marek Marczykowski-Górecki
23ee8aeb47
vm/qubesvm: fix handling vm.storage.verify() errors
except section try to access qmemman_client variable - make it defined
at that stage.
2017-11-21 04:16:30 +01:00
Marek Marczykowski-Górecki
a92dd99fbb
Merge branch '20171107-storage'
* 20171107-storage:
  api/admin: add API for changing revisions_to_keep dynamically
  storage/file: move revisions_to_keep restrictions to property setter
  api/admin: hide dd statistics in admin.vm.volume.Import call
  storage/lvm: fix importing different-sized volume from another pool
  storage/file: fix preserving spareness on volume clone
  api/admin: add pool size and usage to admin.pool.Info response
  storage: add size and usage properties to pool object
2017-11-20 22:52:50 +01:00
Marek Marczykowski-Górecki
2c6c766968
Merge branch '20171107-tests-backup-api-misc'
* 20171107-tests-backup-api-misc:
  test: make race condition on xterm close less likely
  tests/backupcompatibility: fix handling 'internal' property
  backup: fix handling target write error (like no disk space)
  tests/backupcompatibility: drop R1 format tests
  backup: use offline_mode for backup collection
  qubespolicy: fix handling '$adminvm' target with ask action
  app: drop reference to libvirt object after undefining it
  vm: always log startup fail
  api: do not log handled errors sent to a client
  tests/backups: convert to new restore handling - using qubesadmin module
  app: clarify error message on failed domain remove (used somewhere)
  Fix qubes-core.service ordering
2017-11-20 22:52:26 +01:00
Marek Marczykowski-Górecki
9519f158ab
vm/qubesvm: vm.storage.stop() is a coroutine 2017-11-20 22:49:52 +01:00
Marek Marczykowski-Górecki
cf92a576ad
Merge remote-tracking branch 'qubesos/pr/159'
* qubesos/pr/159:
  qubes/vm: Improve stopped event handling
2017-11-20 22:49:05 +01:00
Marek Marczykowski-Górecki
3217c3ac4e
Merge remote-tracking branch 'qubesos/pr/156'
* qubesos/pr/156:
  tests: Add unit tests for pvh virt_mode
  Don't allow attached PCI devices and virt_mode = 'pvh'
2017-11-20 22:40:53 +01:00
Marek Marczykowski-Górecki
45af0913f2
test: make race condition on xterm close less likely
xterm is very fast on closing when application inside terminates. It is
so fast with closing on keydown event that xdotool do not manage to send
keyup event, resulting in xdotool crash. Add a little more time for
that.
2017-11-07 03:10:42 +01:00
Marek Marczykowski-Górecki
07d4f4b340
tests/backupcompatibility: fix handling 'internal' property 2017-11-07 03:10:42 +01:00
Marek Marczykowski-Górecki
5e6ba4ff5c
backup: fix handling target write error (like no disk space)
When writing process returns an error, prefer reporting that one,
instead of other process (which in most cases will be canceled, so
the error will be meaningless CancelledError).
Then, include sanitized stderr from VM process in the error message.
2017-11-07 03:10:42 +01:00
Marek Marczykowski-Górecki
11fd2cf115
tests/backupcompatibility: drop R1 format tests
It is no longer supported.
2017-11-07 03:10:41 +01:00
Marek Marczykowski-Górecki
abfed95bf2
backup: use offline_mode for backup collection
This object is used solely for manipulating qubes.xml for the backup. Do
not open any connection to hypervisor/QubesDB/libvirt etc. This will
help avoiding various leaks (memory, FD).
2017-11-07 03:10:41 +01:00
Marek Marczykowski-Górecki
d3cc2d50e3
qubespolicy: fix handling '$adminvm' target with ask action
All policy keywords needs to be expanded before sending it to
confirmation dialog. $dispvm was already handled, but $adminvm was
missing

Fixes QubesOS/qubes-issues#3283
2017-11-07 03:10:41 +01:00
Marek Marczykowski-Górecki
227378f2b3
app: drop reference to libvirt object after undefining it
Do not try to access that particular object (wrapper) when it got
undefined. If anyone want to access it, appropriate code should do a new
lookup, and probably re-define the object.
2017-11-07 03:10:41 +01:00
Marek Marczykowski-Górecki
b5e07238a0
vm: always log startup fail 2017-11-07 03:10:40 +01:00
Marek Marczykowski-Górecki
73c33525a0
api: do not log handled errors sent to a client
Those "errors" are already properly handled, and if necessary logged
independently by appropriate function. In some cases, such logs are
misleading (for example QubesNoSuchPropertyError is a normal thing
happening during qvm-ls).

Fixes QubesOS/qubes-issues#3238
2017-11-07 03:10:40 +01:00
Marek Marczykowski-Górecki
31a55dcd18
tests/backups: convert to new restore handling - using qubesadmin module
Besides converting itself, change how the test verify restore
correctness: first collect VM metadata (and hashes of data) into plain
dict, then compare against it. This allow to destroy old VMs objects
before restoring the backup, so avoid having duplicate objects of the
same VM - which results in weird effects like trying to undefine libvirt
object twice.
2017-11-07 03:10:40 +01:00
Marek Marczykowski-Górecki
59cebd5439
app: clarify error message on failed domain remove (used somewhere)
Point to system logs for more details. Do not include them directly in
the message for privacy reasons (Admin API client may not be given
permission to it).

QubesOS/qubes-issues#3273
QubesOS/qubes-issues#3193
2017-11-07 03:10:32 +01:00
Marek Marczykowski-Górecki
08584ce370
Fix qubes-core.service ordering
The service needs to be started after qubesd and qubes-qmemman to be
able to start domains. The same applies on shutdown - should be ordered
before shutting down qubesd.
2017-11-07 02:59:20 +01:00
Marek Marczykowski-Górecki
c3afdde3ef
api/admin: add API for changing revisions_to_keep dynamically
This one pool/volume property makes sense to change dynamically. There
may be more such properties, but lets be on the safe side and take
whitelist approach - allow only selected (just one for now), instead of
blacklisting any harmful ones.

QubesOS/qubes-issues#3256
2017-11-07 02:57:59 +01:00
Marek Marczykowski-Górecki
81f455e15d
storage/file: move revisions_to_keep restrictions to property setter
Do not check for accepted value only in constructor, do that in property
setter. This will allow enforcing the limit regardless of how the value
was set.

This is preparation for dynamic revisions_to_keep change.

QubesOS/qubes-issues#3256
2017-11-07 02:57:59 +01:00
Marek Marczykowski-Górecki
0327b6cd98
api/admin: hide dd statistics in admin.vm.volume.Import call
But still show errors, if occurs.
2017-11-07 02:57:59 +01:00
Marek Marczykowski-Górecki
439d9b87ff
storage/lvm: fix importing different-sized volume from another pool
Fixes QubesOS/qubes-issues#3257
2017-11-07 02:57:59 +01:00
Marek Marczykowski-Górecki
f3455b5d99
storage/file: fix preserving spareness on volume clone
Force creating sparse file, even if source volume is not such file (for
example block device).

Reported by @na--
QubesOS/qubes-issues#3255
2017-11-07 02:57:58 +01:00
Marek Marczykowski-Górecki
f18f4c9bff
api/admin: add pool size and usage to admin.pool.Info response
QubesOS/qubes-issues#3240
2017-11-07 02:57:58 +01:00
Marek Marczykowski-Górecki
e76372b934
storage: add size and usage properties to pool object
Add Pool.size and Pool.usage to the API. Implement them for LVM and File
pools. Add appropriate tests.

QubesOS/qubes-issues#3240
2017-11-07 02:57:58 +01:00
HW42
c695f3268c linux-stubdom: Set cmdline in HVM config 2017-10-30 16:19:01 +01:00
HW42
b907303a39 qubes/vm: Improve stopped event handling
The previous version did not ensure that the stopped/shutdown event was
handled before a new VM start. This can easily lead to problems like in
QubesOS/qubes-issues#3164.

This improved version now ensures that the stopped/shutdown events are
handled before a new VM start.

Additionally this version should be more robust against unreliable
events from libvirt. It handles missing, duplicated and delayed stopped
events.

Instead of one 'domain-shutdown' event there are now 'domain-stopped'
and 'domain-shutdown'. The later is generated after the former. This way
it's easy to run code after the VM is shutdown including the stop of
it's storage.
2017-10-21 05:57:57 +02:00
Marek Marczykowski-Górecki
682d9503ee
version 4.0.11 2017-10-21 04:37:18 +02:00
Marek Marczykowski-Górecki
e2eaa57f65
Merge branch 'tests-20171020'
* tests-20171020:
  tests: fix asyncio usage some more places
  tests: fix reporting in network tests
  tests: domain-shutdown event race condition
  tests: improve events tests
  tests: fix logic error in wait_for_window()
  tests: cleanup QubesDB connection on domain remove
2017-10-21 04:33:35 +02:00
Marek Marczykowski-Górecki
42f8619559
tests: fix asyncio usage some more places
gui_memory_pinning tests finally works. Also, fix DispVM and private
volume resize tests.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
511502ae7e
tests: fix reporting in network tests
Do not trash stdout with VM's command output. Log it as part of
(potential) failure message.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
c3430fe5f6
tests: domain-shutdown event race condition
Second tests, for actual events, not storage handling done by its
handler.

cc @HW42

QubesOS/qubes-issues#3164
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
96f2e503db
tests: improve events tests
Check property-related events, including their arguments.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
71fc5217f7
tests: fix logic error in wait_for_window()
Revert change introduced in cbc5dbb9b1
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
43c09467b1
tests: cleanup QubesDB connection on domain remove
If domain got removed during the tests (for example DispVM), vm.close()
wouldn't be called in cleanup and some file descriptors will be
leaked. Add event handler for cleaning this up. Do not use close()
method here, because it is destructive, but the object may still be used
by the test.
2017-10-21 04:06:14 +02:00