Commit Graph

247 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
9ba84ecdad
Add code and test for migrating vm.hvm into vm.virt_mode
This will allow to load old qubes.xml - otherwise previous change render
the system unusable (until manual qubes.xml edit).

QubesOS/qubes-issues#2912
2017-07-17 12:27:17 +02:00
Marek Marczykowski-Górecki
21940bef90
Change vm.hvm into vm.virt_mode
Fixes QubesOS/qubes-issues#2912
2017-07-17 12:26:10 +02:00
Marek Marczykowski-Górecki
a42e500108
Merge branch 'tests-fixes-1'
* tests-fixes-1:
  api: extract function to make pylint happy
  tests/vm: simplify AppVM storage test
  storage: do not use deepcopy on volume configs
  api: cleanup already started servers when some later failed
  tests: fix block devices tests when running on real system
  tests: fix some FD leaks
2017-07-14 01:05:21 +02:00
Marek Marczykowski-Górecki
f2f89c7b0c
storage: do not use deepcopy on volume configs
There may be Pool or Volume object references, which is intentional to
keep them as is - not copy whole Pool/Volume objects.
2017-07-12 21:33:14 +02:00
Wojtek Porczyk
b04f612374 qubes: have "service" features' keys separated by period 2017-07-05 04:16:16 +02:00
Marek Marczykowski-Górecki
dddd94b339
Merge branch 'core3-storage3' 2017-07-04 16:44:53 +02:00
Marek Marczykowski-Górecki
51022cada5
Merge remote-tracking branch 'origin/policy-adminvm' 2017-07-04 15:51:30 +02:00
Marek Marczykowski-Górecki
cfbccc0a74
vm: drop old is_outdated/commit_changes
This code is unused now. Theoretically this is_outdated implementation
should be moved to FileVolume, but since we don't have VM reference
there, it isn't possible to read appropriate xenstore entry. As we're
phasing out file pool, simply ignore it for now.

QubesOS/qubes-issues#2256
2017-07-04 14:25:10 +02:00
Marek Marczykowski-Górecki
0e554296e3
storage: drop 'internal' and 'removable' volume properties
Since dynamic volumes (qvm-block) are moved to devices API, those two
are not needed anymore.

QubesOS/qubes-issues#2256
2017-07-04 14:25:08 +02:00
Marek Marczykowski-Górecki
075d991114
config: eliminate duplicated qubes_base_dir
Remove it from system_path dict, have it in one place.
2017-07-04 14:25:07 +02:00
Marek Marczykowski-Górecki
b5ad245614
vm/qubesvm: have VM lifecycle related events async
Those functions are coroutines anyway, so allow event handlers to be
too.
Some of this (`domain-create-on-disk`, `domain-remove-from-disk`) will
be useful for appmenus handling.
2017-07-04 13:04:24 +02:00
Marek Marczykowski-Górecki
6238254f49
events: merge fire_event and fire_event_pre functions
Those functions really do very similar things - lets merge them and add
simple parameter.
2017-07-04 13:04:21 +02:00
Marek Marczykowski-Górecki
f393ce62dc
Fix docstrings formatting
Fix warnings/errors reported by Sphinx
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
820539e909
storage: make volume snap_on_start/save_on_stop explicit
Always define those properties, always include them in volume config.
Also simplify overriding pool based on volume type defined by those:
override pool unless snap_on_start=True.

QubesOS/qubes-issues#2256
2017-07-04 04:11:14 +02:00
Marek Marczykowski-Górecki
dd1e05dc83
vm: drop rename related methods
Since VM name in immutable now, drop rename-related methods.

QubesOS/qubes-issues#2868
2017-07-04 04:11:13 +02:00
Marek Marczykowski-Górecki
3074a4064c
vm/qubesvm: forbid changing VM name
We've decided to make VM name immutable. This is especially important
for Admin API, where some parts (especially policy) are sticked to the
VM name.

Now, to rename the VM, one need to clone it under new name (thanks to
LVM, this is very quick action), then remove the old one.

Fixes QubesOS/qubes-issues#2868
2017-06-26 02:00:45 +02:00
Marek Marczykowski-Górecki
a6c16d00be
vm/qubesvm: fix handling empty kernel value
Do not fail on empty kernel value - which means "use booloader on
root.img".

Fixes 3ddc052 "vm: move kernel presence validation to event handler"
2017-06-26 01:59:39 +02:00
Marek Marczykowski-Górecki
3ddc052af3
vm: move kernel presence validation to event handler
Setter is called also on qubes.xml load, so missing kernel breaks
qubes.xml parsing - for example qubesd startup to fix that property.
2017-06-21 00:17:43 +02:00
Marek Marczykowski-Górecki
60443259d0
vm: raise CalledProcessError on failed service/command call 2017-06-21 00:17:42 +02:00
Marek Marczykowski-Górecki
5209bc370d
vm: init vm.storage and vm.volumes in BaseVM
This way also AdminVM will have (empty) properties there. It is much
cleaner than adding `if hasattr` or catching AttributeError everywhere.
2017-06-14 10:44:24 +02:00
Marek Marczykowski-Górecki
160ab964bc
vm: improve sending events for tags and features
1. Send the event after setting tag/feature
2. Provide old value for feature
3. Rename 'key' kwarg to 'feature'
2017-06-14 10:44:23 +02:00
Marek Marczykowski-Górecki
93ccb8bbda
Merge remote-tracking branch 'qubesos/pr/111'
* qubesos/pr/111:
  vm: drop 'internal' property
  qmemman: make sure to release lock
  qmemman: fix meminfo parsing for python 3
  devices: drop 'data' and 'frontend_domain' fields, rename 'devclass' to 'bus'
2017-06-12 23:22:55 +02:00
Marek Marczykowski-Górecki
ba4da5cbe0
Merge remote-tracking branch 'qubesos/pr/110'
* qubesos/pr/110:
  storage: use direct object references, not only identifiers
  vm: fix volume_config
  storage/lvm: prefix VM LVM volumes with 'vm-'
  storage: fix VM rename
2017-06-12 23:22:41 +02:00
Marek Marczykowski-Górecki
37245acdcf
vm: change vm.updates_available to a 'updates-available' feature 2017-06-12 12:34:23 +02:00
Marek Marczykowski-Górecki
caa03a9279
vm/qubesvm: simplify 'None' kernel handling 2017-06-12 12:26:57 +02:00
Marek Marczykowski-Górecki
9d99232515
vm: drop 'internal' property
It isn't used anywhere in the code right now. And when it will be
needed, it should be a "feature" not "property".
2017-06-12 10:06:43 +02:00
Marek Marczykowski-Górecki
0f12870803
storage: use direct object references, not only identifiers
Reference objects, not their IDs - this way when object is modified, it
is visible everywhere where it is used. Main changes:
- volume.pool - Pool object
- volume.source - Volume object

Since volume have Pool object reference now, move volume related
functions into Volume class (from Pool class). This avoids horrible
`storage.get_pool(volume).something(volume)` construct.

One issue here is since volume.source reference a Volume object from a
different VM - VM's template, now VM load order is important. Since we
don't have control over it, initialize vm.storage when needed - possibly
while initializing storage of different VM. Since we don't have cycles
in AppVM-TemplateVM dependencies, it is safe.

Also, since this commit, volume.source (if defined) always points at
volume of the same name from VM's template. Using volumes with something
else as a source is no longer supported.

QubesOS/qubes-issues#2256
2017-06-09 04:55:06 +02:00
Marek Marczykowski-Górecki
fa1da422d1
vm/adminvm: don't base AdminVM on QubesVM
dom0 isn't real VM and most properties doesn't apply to it. Lets make it
more explicit.
2017-06-06 21:57:27 +02:00
Marek Marczykowski-Górecki
89e025ca3b
vm: make vm.get_power_state() return 'Halted' in offline_mode
Again, if libvirt or even Xen isn't running, we can safely assume VM
isn't too.
2017-06-06 03:33:31 +02:00
Marek Marczykowski-Górecki
63c23c835b
vm: reduce code duplication
Move inheriting volume from template to a helper function.

No functional change.

QubesOS/qubes-issues#2256
2017-06-06 03:33:30 +02:00
Marek Marczykowski-Górecki
28737d16ce
Migrate qubes.NotifyTools, qubes.FeaturesRequest, qubes.NotifyUpdates
Make them call into qubesd. Create separate socket for "misc" calls - VM
accessible, but not part of Admin API.
2017-06-06 03:33:30 +02:00
Marek Marczykowski-Górecki
f80fddf7ae
vm: restore code for suspending domains with PCI devices 2017-06-06 03:33:29 +02:00
Marek Marczykowski-Górecki
4c4f903fe3
vm: change kernel=None to kernel=''
vm.kernel property have type 'str'. Putting None there makes a lot of
troubles: it gets encoded as 'None' in qubes.xml and then loaded back as
'None' string, not None value. Also it isn't possible to assign None
value to str property throgh Admin API.

kernel='' is equally good to specify "no kernel from dom0".

QubesOS/qubes-issues#2622
2017-06-05 23:38:30 +02:00
Marek Marczykowski-Górecki
1ed0b14d93
vm: make vm.hvm=True by default
QubesOS/qubes-issues#2185
2017-06-05 23:38:30 +02:00
Marek Marczykowski-Górecki
5d3204c730
vm: remove prepare_dvm parameter from methods and events
It was related to DispVM savefile preparation, but it is no longer
applicable in Qubes 4.0
2017-06-05 23:38:30 +02:00
Marek Marczykowski-Górecki
018877a19c
vm: avoid starting the same VM multiple times simultaneously
While libvirt handle locking itself, there is also Qubes-specific
startup part. Especially starting qrexec-daemon and waiting until
qrexec-agent connect to it. When someone will attempt to start VM the
second time (or simply assume it's already running) - qrexec will not be
connected yet and the operation will fail. Solve the problem by wrapping
the whole vm.start() function with a lock, including a check if VM is
running and waiting for qrexec.

Also, do not throw exception if VM is already running.

This way, after a call to vm.start(), VM will be started with qrexec
connected - regardless of who really started it.
Note that, it will not solve the situation when someone check if VM is
running manually, like:

    if not vm.is_running():
        yield from vm.start()

Such code should be changed to simply:

    yield from vm.start()

Fixes QubesOS/qubes-issues#2001
Fixes QubesOS/qubes-issues#2666
2017-06-05 23:38:30 +02:00
Marek Marczykowski-Górecki
b5a4573290
storage: remove DomainPool
Replaced by BlockDevice extension
2017-06-05 23:33:58 +02:00
Wojtek Porczyk
bbe757d0a7 Make pylint very ♥ 2017-05-30 15:06:05 +02:00
Wojtek Porczyk
b212a75ba3 qubes/vm/qubesvm: don't use .run_service() for .run()
QubesOS/qubes-issues#2622
2017-05-29 17:34:15 +02:00
Wojtek Porczyk
a9755ed76a qubes/ext/gui: remove is-fully-usable and is_guid_running 2017-05-29 17:34:15 +02:00
Marek Marczykowski-Górecki
e54cc11a2c
vm: expose to VM only features with 'service/' prefix
And place them in /qubes-service/ QubesDB directory. This allows
extensions to easily store some data not exposed to VM, but also have
control what VM will see. And at the same time, it make it compatible
with existing services framework

QubesOS/qubes-issues#1637
2017-05-26 15:08:19 +02:00
Marek Marczykowski-Górecki
1692601fcd
typo fix 2017-05-23 15:35:21 +02:00
Marek Marczykowski-Górecki
29f3c9b58f
vm: don't try to define libvirt domain when just checking its state
When libvirt domain is not defined, it isn't running for sure.
This commit fixes the case when vm.is_running() appears anywhere in the
code used during libvirt xml building. In this case, it's mostly about
PCI device description for libvirt.
2017-05-22 17:11:19 +02:00
Marek Marczykowski-Górecki
dde9085920
vm: fix autostart-related events handling
event signature contains 'name', not 'prop' (it's no longer positional
argument).
2017-05-22 03:21:13 +02:00
Marek Marczykowski-Górecki
ae28d32af7
vm: make sure qmemman connection is closed even on failed VM startup
Qmemman hold global lock while serving requests, so not closing
connection will prevent any further action.
2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki
0b64e3fbe0
storage: make LinuxKernel pool track vm kernel version
Do not initialize it only at qubes.xml load time, but re-read vm.kernel
property each time the path is constructed. While at it, add support for
vm.kernel set to 'None' - simply don't include modules.img (xvdd) then.
2017-05-17 11:17:21 +02:00
Marek Marczykowski-Górecki
595d983659
storage: make verify() asyncio aware 2017-05-17 11:17:21 +02:00
Marek Marczykowski-Górecki
680dbf432c
vm: Features fixes
clear: dict should not be mutated during interation.
2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki
ba86d6da79
vm: implement tag manager to fire events on change
While at it, adjust implementation to specification: tags don't have
value, only one bit of information (present/not present).

Fixes QubesOS/qubes-issues#2686
2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki
3c7d2289ce
vm: stubdom_xid: use canonical method for checking VM state
Don't depend on xid==-1 for not running VM.
This breaks some tests, as xid can be a mock object (not comparable to
int).
2017-05-16 09:05:00 +02:00