Commit Graph

5884 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
d3cc2d50e3
qubespolicy: fix handling '$adminvm' target with ask action
All policy keywords needs to be expanded before sending it to
confirmation dialog. $dispvm was already handled, but $adminvm was
missing

Fixes QubesOS/qubes-issues#3283
2017-11-07 03:10:41 +01:00
Marek Marczykowski-Górecki
227378f2b3
app: drop reference to libvirt object after undefining it
Do not try to access that particular object (wrapper) when it got
undefined. If anyone want to access it, appropriate code should do a new
lookup, and probably re-define the object.
2017-11-07 03:10:41 +01:00
Marek Marczykowski-Górecki
b5e07238a0
vm: always log startup fail 2017-11-07 03:10:40 +01:00
Marek Marczykowski-Górecki
73c33525a0
api: do not log handled errors sent to a client
Those "errors" are already properly handled, and if necessary logged
independently by appropriate function. In some cases, such logs are
misleading (for example QubesNoSuchPropertyError is a normal thing
happening during qvm-ls).

Fixes QubesOS/qubes-issues#3238
2017-11-07 03:10:40 +01:00
Marek Marczykowski-Górecki
31a55dcd18
tests/backups: convert to new restore handling - using qubesadmin module
Besides converting itself, change how the test verify restore
correctness: first collect VM metadata (and hashes of data) into plain
dict, then compare against it. This allow to destroy old VMs objects
before restoring the backup, so avoid having duplicate objects of the
same VM - which results in weird effects like trying to undefine libvirt
object twice.
2017-11-07 03:10:40 +01:00
Marek Marczykowski-Górecki
59cebd5439
app: clarify error message on failed domain remove (used somewhere)
Point to system logs for more details. Do not include them directly in
the message for privacy reasons (Admin API client may not be given
permission to it).

QubesOS/qubes-issues#3273
QubesOS/qubes-issues#3193
2017-11-07 03:10:32 +01:00
Marek Marczykowski-Górecki
08584ce370
Fix qubes-core.service ordering
The service needs to be started after qubesd and qubes-qmemman to be
able to start domains. The same applies on shutdown - should be ordered
before shutting down qubesd.
2017-11-07 02:59:20 +01:00
Marek Marczykowski-Górecki
c3afdde3ef
api/admin: add API for changing revisions_to_keep dynamically
This one pool/volume property makes sense to change dynamically. There
may be more such properties, but lets be on the safe side and take
whitelist approach - allow only selected (just one for now), instead of
blacklisting any harmful ones.

QubesOS/qubes-issues#3256
2017-11-07 02:57:59 +01:00
Marek Marczykowski-Górecki
81f455e15d
storage/file: move revisions_to_keep restrictions to property setter
Do not check for accepted value only in constructor, do that in property
setter. This will allow enforcing the limit regardless of how the value
was set.

This is preparation for dynamic revisions_to_keep change.

QubesOS/qubes-issues#3256
2017-11-07 02:57:59 +01:00
Marek Marczykowski-Górecki
0327b6cd98
api/admin: hide dd statistics in admin.vm.volume.Import call
But still show errors, if occurs.
2017-11-07 02:57:59 +01:00
Marek Marczykowski-Górecki
439d9b87ff
storage/lvm: fix importing different-sized volume from another pool
Fixes QubesOS/qubes-issues#3257
2017-11-07 02:57:59 +01:00
Marek Marczykowski-Górecki
f3455b5d99
storage/file: fix preserving spareness on volume clone
Force creating sparse file, even if source volume is not such file (for
example block device).

Reported by @na--
QubesOS/qubes-issues#3255
2017-11-07 02:57:58 +01:00
Marek Marczykowski-Górecki
f18f4c9bff
api/admin: add pool size and usage to admin.pool.Info response
QubesOS/qubes-issues#3240
2017-11-07 02:57:58 +01:00
Marek Marczykowski-Górecki
e76372b934
storage: add size and usage properties to pool object
Add Pool.size and Pool.usage to the API. Implement them for LVM and File
pools. Add appropriate tests.

QubesOS/qubes-issues#3240
2017-11-07 02:57:58 +01:00
HW42
c695f3268c linux-stubdom: Set cmdline in HVM config 2017-10-30 16:19:01 +01:00
HW42
b907303a39 qubes/vm: Improve stopped event handling
The previous version did not ensure that the stopped/shutdown event was
handled before a new VM start. This can easily lead to problems like in
QubesOS/qubes-issues#3164.

This improved version now ensures that the stopped/shutdown events are
handled before a new VM start.

Additionally this version should be more robust against unreliable
events from libvirt. It handles missing, duplicated and delayed stopped
events.

Instead of one 'domain-shutdown' event there are now 'domain-stopped'
and 'domain-shutdown'. The later is generated after the former. This way
it's easy to run code after the VM is shutdown including the stop of
it's storage.
2017-10-21 05:57:57 +02:00
Marek Marczykowski-Górecki
682d9503ee
version 4.0.11 2017-10-21 04:37:18 +02:00
Marek Marczykowski-Górecki
e2eaa57f65
Merge branch 'tests-20171020'
* tests-20171020:
  tests: fix asyncio usage some more places
  tests: fix reporting in network tests
  tests: domain-shutdown event race condition
  tests: improve events tests
  tests: fix logic error in wait_for_window()
  tests: cleanup QubesDB connection on domain remove
2017-10-21 04:33:35 +02:00
Marek Marczykowski-Górecki
42f8619559
tests: fix asyncio usage some more places
gui_memory_pinning tests finally works. Also, fix DispVM and private
volume resize tests.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
511502ae7e
tests: fix reporting in network tests
Do not trash stdout with VM's command output. Log it as part of
(potential) failure message.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
c3430fe5f6
tests: domain-shutdown event race condition
Second tests, for actual events, not storage handling done by its
handler.

cc @HW42

QubesOS/qubes-issues#3164
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
96f2e503db
tests: improve events tests
Check property-related events, including their arguments.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
71fc5217f7
tests: fix logic error in wait_for_window()
Revert change introduced in cbc5dbb9b1
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
43c09467b1
tests: cleanup QubesDB connection on domain remove
If domain got removed during the tests (for example DispVM), vm.close()
wouldn't be called in cleanup and some file descriptors will be
leaked. Add event handler for cleaning this up. Do not use close()
method here, because it is destructive, but the object may still be used
by the test.
2017-10-21 04:06:14 +02:00
Marek Marczykowski-Górecki
0b1a0b028b
app: fix setting default_*netvm
1. Fire both property-pre-del:netvm and property-del:netvm - those
events should be fired in pairs - especially one may assume the other
will be called too. This is the case here - one disconnect old netvm,
the other connect the new one.

2. Remove spurious 'newvalue' argument for property-del:netvm event.

3. Fix logic for default_fw_netvm/default_netvm usage. The former is
used if vm.provides_network=True.
2017-10-21 04:05:17 +02:00
Marek Marczykowski-Górecki
4500c4dcab
vm/adminvm: add is_halted() method
It is used in some places, for example attaching devices.

Fixes QubesOS/qubes-issues#3054
2017-10-21 04:05:17 +02:00
Marek Marczykowski-Górecki
433430884d
devices: clarify error on attaching non-persistent device to stopped VM
Fixes QubesOS/qubes-issues#3195
2017-10-21 04:05:16 +02:00
Marek Marczykowski-Górecki
9a9452ae3a
vm/net: fix handling netvm reset to default
If there was some netvm set, unset it first (same as with ordinary set).
Otherwise it will try to attach new netvm without detaching the old one
first.
2017-10-21 04:05:16 +02:00
Marek Marczykowski-Górecki
98818b0ad9
base: fix property-(pre-)del events arguments
oldvalue should contain the old value, even if it was default one. Same
as in property-(pre-)set events. If event want to check if that is
default value, it is always possible (in pre- event), but in practice
actual value is most useful.

This bug prevented netvm changing events from working when reseting
netvm.
2017-10-21 04:05:16 +02:00
Marek Marczykowski-Górecki
8f1bfa65f8
tests: fix typo 2017-10-21 04:05:15 +02:00
Marek Marczykowski-Górecki
4809c16e08
admin-api: add admin.*.property.GetDefault
Fixes QubesOS/qubes-issues#3197
2017-10-21 04:05:15 +02:00
Marek Marczykowski-Górecki
038ca6d5d7
base: add method for getting default property value
Allow to get default value even it isn't set currently. This will allow
(G)UI to present better view, without duplicating logic for default
value.

Fixes QubesOS/qubes-issues#3197
2017-10-21 04:05:15 +02:00
Marek Marczykowski-Górecki
85c3368a0d
comments fixes 2017-10-21 04:05:15 +02:00
Marek Marczykowski-Górecki
c2c3225622
tests: root fs resize tests - for all templates
Check resize on each template separately, because it involves VM's
scripts (either qubes.ResizeDisk service, or some startup script).

QubesOS/qubes-issues#3173
2017-10-21 04:05:14 +02:00
Marek Marczykowski-Górecki
77ce956c2d
tests: improve root volume resize tests
- clone all features, not just qrexec (especially include 'gui')
- do not leak VM reference on failed test
- add test for online root volume resize

QubesOS/qubes-issues#3173
2017-10-21 03:35:01 +02:00
Marek Marczykowski-Górecki
49e35e66aa
storage: improve error reporting
- make it clear that failed qubes.ResizeDisk service call means the need
to resize filesystem manually (but not necessarily the volume itself)
- propagate exceptions raised by async storage pool implementations

Related QubesOS/qubes-issues#3173
2017-10-21 03:35:01 +02:00
Marek Marczykowski-Górecki
27b96cce4c
firewall: skip expired rules
Expired rules are skipped while loading the firewall. Do that also when
such rules expired after loading the firewall. This applies to both
Admin API and actually applying the rules (sending them to appropriate
VM).

Related QubesOS/qubes-issues#3020
2017-10-21 03:35:00 +02:00
Marek Marczykowski-Górecki
ae473a8c26
version 4.0.10 2017-10-16 04:34:07 +02:00
Marek Marczykowski-Górecki
836d9f902a
Merge branch 'bug3164'
* bug3164:
  tests: add regression test for #3164
  storage/lvm: make sure volume cache is refreshed after changes
  storage/lvm: fix Volume.verify()
  storage/lvm: remove old volume only after successfully cloning new one
2017-10-16 04:28:26 +02:00
Marek Marczykowski-Górecki
96c077e520
Update readme
Master branch is no longer subject to rebases.
Also, update links.
2017-10-16 04:16:23 +02:00
Marek Marczykowski-Górecki
c8519a700f
tests: add regression test for #3164
This is a race condition, so to make it more likely to fail (if it's
broken), make some things manually. In normal circumstances this order
of actions is also possible, just less likely to happen. But as seen in
the bug report, happens from time to time.

QubesOS/qubes-issues#3164
2017-10-16 00:48:25 +02:00
Marek Marczykowski-Górecki
08583af1ff
storage/lvm: make sure volume cache is refreshed after changes
Even if start/stop fails, still refresh the cache. The failed operation
might still modified something.
2017-10-16 00:44:52 +02:00
Marek Marczykowski-Górecki
145ccfb34e
storage/lvm: fix Volume.verify()
This function is supposed to raise an exception if something is wrong,
not just return False. Document this.
2017-10-16 00:43:10 +02:00
Marek Marczykowski-Górecki
021047f950
storage/lvm: remove old volume only after successfully cloning new one
In some cases, it may happen that new volume (`self._vid_snap`) does not
exists. This is normally an error, but even in such a case, do not
remove the only remaining instance of volume (`self.vid`). Instead,
rename it temporarily and remove only after new volume is successfully
cloned.

Fixes QubesOS/qubes-issues#3164
2017-10-13 01:11:55 +02:00
Marek Marczykowski-Górecki
2164a8d7b8
Change license to LGPL v2.1+
See this thread for reasoning and acceptance from contributors:
https://groups.google.com/d/topic/qubes-devel/G7KzrfU0lWY/discussion
"Changing qubes-core-admin license to LGPL v2.1+"
2017-10-12 00:11:50 +02:00
Marek Marczykowski-Górecki
53903a3685
rpm: remove commented out rpm macro
Just commenting out macro isn't enough - % sequences are still
processed. Remove it completely, as it is already provided in build
environment.
2017-10-11 22:41:31 +02:00
HW42
4a84d5b222 tests: Add unit tests for pvh virt_mode 2017-10-11 21:11:36 +02:00
HW42
67c06eb428 Don't allow attached PCI devices and virt_mode = 'pvh' 2017-10-11 21:10:43 +02:00
Marek Marczykowski-Górecki
ec4c7d7263
version 4.0.9 2017-10-09 00:10:12 +02:00
Marek Marczykowski-Górecki
08640f4779
Merge remote-tracking branch 'qubesos/pr/154'
* qubesos/pr/154:
  Add basic PVHv2 support
2017-10-09 00:08:35 +02:00