Commit Graph

367 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
77323cd30d
Start using 'async def' syntax
The Python version (and pylint) are new enough finally.
Adjust QubesVM.run* functions for now.
Python 3.8 brings also AsyncMock() which makes tests slightly less ugly.
2020-11-26 05:13:48 +01:00
Marek Marczykowski-Górecki
1500ed8fcb
Make pylint happy
- ignore raise-missing-from
- fix super-with-arguments
2020-08-23 02:55:40 +02:00
Marek Marczykowski-Górecki
01b33e58d7
Merge remote-tracking branch 'origin/pr/350'
* origin/pr/350:
  Changed feature keyboard_layout to a property
2020-07-15 16:21:38 +02:00
Marek Marczykowski-Górecki
6adf56f4ba
Merge remote-tracking branch 'origin/pr/355'
* origin/pr/355:
  Removed unused vm.icon_path property
2020-07-15 16:19:53 +02:00
Marta Marczykowska-Górecka
bed8e578d7
Removed unused vm.icon_path property
The property was not used for anything, but caused numerous
problems due to symlinks.

fixes QubesOS/qubes-issues#5934
2020-07-14 17:52:09 +02:00
Marta Marczykowska-Górecka
06e2d14a97
Changed feature keyboard_layout to a property
Purpose: make it easier to implement more robust keyboard layout tools and
propagation.

references QubesOS/qubes-issues#1396
references QubesOS/qubes-issues#4294
2020-07-14 01:32:55 +02:00
Marek Marczykowski-Górecki
bd6d71a555
vm: emit property-reset:stubdom_xid event on domain start/stop too
Similart to property-reset:xid, emit property-reset:stubdom_xid when
domain is started/stopped. This allows client side of the Admin API
(qubes-core-admin-client) to invalidate the cache when necessary.

Found by audio tests: #352
2020-07-08 12:54:35 +02:00
Marek Marczykowski-Górecki
719eb5c2a2
Fix revisions_to_keep volume property when creating VM in a specific pool
The revisions_to_keep should be inherited from the pool by default (or
whatever else logic is in the storage pool driver). When creating VM in
a specific pool, volumes config is re-initialized to include right
defaults. But the config cleaning logic in `_clean_volume_config()`
failed to remove revisions_to_keep property initialized by the default
pool driver. This prevented new pool driver to apply its own default
logic.

An extreme result was inability to create a VM in 'file' pool at all,
because it refuses any revisions_to_keep > 1, and the default LVM
pool has revisions_to_keep = 2.
2020-05-28 01:45:18 +02:00
Marek Marczykowski-Górecki
d61d24b055
Fire property-reset event when default value might change
Those are only some cases, the most obvious ones:
 - defaults inherited from a template
 - xid and start_time on domain start/stop
 - IP related properties
 - icon

QubesOS/qubes-issues#5834
2020-05-23 04:50:02 +02:00
Marek Marczykowski-Górecki
dc2cf1db5d
Convert handler to use property-reset instead of property-del
There was also one case of triggering property-{del => reset}
synthetically on default value change. Adjust it too and drop -pre-
event call in that case.

QubesOS/qubes-issues#5834
2020-05-23 03:57:42 +02:00
Marek Marczykowski-Górecki
c7d3635972
Merge remote-tracking branch 'origin/pr/342'
* origin/pr/342:
  Added a servicevm feature extension
  Removed unused Qubes Manager extension
  Add vm.icon property
2020-05-11 01:51:52 +02:00
Marta Marczykowska-Górecka
c34b53d83a
Add vm.icon property
This is a property for handling vm icons that change depending on
vm type.
Depends on QubesOS/qubes-artwork#17

references QubesOS/qubes-issues#5767
2020-05-05 16:14:10 +02:00
Rusty Bird
6605bf406d
collections.Callable -> collections.abc.Callable
"Deprecated since version 3.3, will be removed in version 3.10"
- https://docs.python.org/3/library/collections.html
2020-04-07 21:30:21 +00:00
Marek Marczykowski-Górecki
d05592ba52
Add pre- events to vm features
Allow extensions to validate feature values, before it is set.
2020-03-17 18:46:12 +01:00
Marek Marczykowski-Górecki
9ec86f3c41
Workaround different behaviour of asyncio's Process.communicate()
In asyncio's Process.communicate() input=None does not close stdin.
Workaround it by using b'' instead of None.

https://bugs.python.org/issue39744
2020-03-10 11:56:48 +01:00
M. Vefa Bicakci
2d55841a5e
qubesvm: Let HVMs enter 'Running' state
Prior to this commit, a properly configured Linux HVM would not
transition from the 'Transient' state to the 'Running' state according
to qvm-ls output, even if the HVM in question had the 'qrexec' feature
disabled.

This issue is caused by an unconditional qrexec check in the
'on_domain_is_fully_usable' method, and is resolved by adding
a check that short-circuits the qrexec check if the aforementioned
feature is not enabled for the VM in question.
2020-03-08 22:39:16 -04:00
Frédéric Pierret (fepitre)
43786e1478
Make pylint happy
- Drop unused 'asyncio'
- Drop useless 'else' after return
2020-03-08 17:05:34 +01:00
Frédéric Pierret (fepitre)
3f4ee9f8d9
xid: ensure vm is not running 2020-03-08 17:05:34 +01:00
Frédéric Pierret (fepitre)
795ff1233a
Support for AudioVM 2020-03-08 17:05:33 +01:00
Rusty Bird
d527e01136
Remove more core2 code 2020-02-15 17:48:32 +00:00
Rusty Bird
1f71a06aed
Remove python2 compat __future__ imports 2020-02-15 17:33:34 +00:00
Pawel Marczewski
e8baf2a157
Remove leftovers from QubesVM.run_for_stdio() refactoring
We no longer use qubes.VMShell there.
2020-01-24 16:59:42 +01:00
Marek Marczykowski-Górecki
361550c621
vm: improve error message about missing IOMMU
Handle this case specifically, as way too many users ignore the message
during installation and complain it doesn't work later.

Name the problem explicitly, instead of pointing at libvirt error log.

Fixes QubesOS/qubes-issues#4689
2019-10-30 15:45:52 +01:00
Marek Marczykowski-Górecki
4dac995089
Merge branch 'start-logging'
* start-logging:
  vm: log startup errors in every case
2019-10-23 13:50:25 +02:00
Marek Marczykowski-Górecki
5cc453f097
vm: remove dead (commented out) code
default_user property was re-implemented, remove dead version.
2019-10-22 16:44:02 +02:00
Frédéric Pierret (fepitre)
d2d1ffb806
Make pylint happier 2019-10-20 16:40:40 +02:00
Frédéric Pierret (fepitre)
27aad9bd38
Handle GuiVM properties 2019-10-20 13:22:31 +02:00
Frédéric Pierret (fepitre)
a52cb6bb91
Make PEP8 happier 2019-10-20 13:22:29 +02:00
Marek Marczykowski-Górecki
d5b0a6e5b6
vm: log startup errors in every case 2019-10-10 19:41:02 +02:00
Marek Marczykowski-Górecki
51af2ed27c
vm: add domain-shutdown-failed event
Similar to domain-start-failed, add an event fired when
domain-pre-shutdown was fired but the actual operation failed.
Note it might not catch all the cases, as shutdown() may be called with
wait=False, which means it won't wait fot the actual shutdown. In that
case, timeout won't result in domain-shutdown-failed event.

QubesOS/qubes-issues#5380
2019-10-08 23:35:24 +02:00
Marek Marczykowski-Górecki
0300e895f2
Fix domain-start-failed event documentation. 2019-10-08 23:35:18 +02:00
Marek Marczykowski-Górecki
732231efb0
vm: refuse to start a VM not present in a collection
Check early (but after grabbing a startup_lock) if VM isn't just
removed. This could happen if someone grabs its reference from other
places (netvm of something else?) or just before removing it.
This commit makes the simple removal from the collection (done as the
first step in admin.vm.Remove implementation) efficient way to block
further VM startups, without introducing extra properties.

For this to be effective, removing from the collection, needs to happen
with the startup_lock held. Modify admin.vm.Remove accordingly.
2019-09-29 06:14:21 +02:00
Marek Marczykowski-Górecki
5fa75d73be
Make pylint happy 2019-09-27 16:29:20 +02:00
Rusty Bird
aa931fdbf7
vm/qubesvm: call storage.remove() *before* removing dir_path
Give the varlibqubes storage pool, which places volume images inside the
VM directory, a chance to remove them (and e.g. to log that).
2019-06-28 10:29:34 +00:00
Rusty Bird
cfa1c7171e
storage: fix typo 2019-06-28 10:29:32 +00:00
Frédéric Pierret (fepitre)
06ec862be4
qubesdb: add qubes-mac path entry 2019-05-23 11:30:08 +02:00
Marek Marczykowski-Górecki
60bbbdd702
Merge branch 'kernelopts-files'
* kernelopts-files:
  vm: allow files in kernels_dir override built-in default kernelopts
2019-03-08 18:08:12 +01:00
Marek Marczykowski-Górecki
092fb9659d
Make pylint happy
Fix multiple instances of 'no-else-raise' warning.
2019-02-27 18:40:18 +01:00
Marek Marczykowski-Górecki
2de5a8e894
vm,templates: allow to obtain common kernelopts from a kernel package
If kernel package ships default-kernelopts-common.txt file, use that
instead of hardcoded Linux-specific options.
For Linux kernel it may include xen_scrub_pages=0 option, but only if
initrd shipped with this kernel re-enable this option later.

QubesOS/qubes-issues#4839
QubesOS/qubes-issues#4736
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
723b33a2f6
vm: move comment about memory overhead constants near their definition 2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
9257a6d14f
Do not abort suspend hooks if any qubes.Suspend* service fails to run
First of all, do not try to call those services in VMs not having qrexec
installed - for example Windows VMs without qubes tools.
Then, even if service call fails for any other reason, only log it but
do not prevent other services from being called. A single uncooperative
VM should generally be able only to hurt itself, not break other VMs
during suspend.

Fixes QubesOS/qubes-issues#3489
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
f9593ce3e6
vm: allow files in kernels_dir override built-in default kernelopts
If default-kernelopts-pci.txt is present, it will override default
built-in kernelopts for the VMs with PCI device assigned.
Similarly if default-kernelopts-nopci.txt is present, it will override
default kernelopts for VMs without PCI devices.
For template-based VMs, kernelopts of the template takes precedence over
default-kernelopts-nopci.txt but not default-kernelopts-pci.txt.

Fixes QubesOS/qubes-issues#4839
2019-02-23 12:53:49 +01:00
Marek Marczykowski-Górecki
a9ec2bb2c3
vm/qubesvm: fix race condition in failed startup handling
Instead of checking if domain is still running/paused, try to kill it
anyway and ignore appropriate exception. Otherwise domain could die
before the check and killing.
2019-01-19 03:25:20 +01:00
Marek Marczykowski-Górecki
3728230e3c
Merge branch 'maxmem' 2018-12-09 18:38:21 +01:00
AJ Jordan
d4e567cb10
Fix typo 2018-12-06 20:43:39 -05:00
Marek Marczykowski-Górecki
7d1bcaf64c Introduce management_dispvm property
The new property is meant for management stack (Salt) to set which DVM
template should be used to maintain given VM. Since the DispVM based on
it will be given ultimate control over target VM (qubes.VMShell
service), it should be trusted. The one pointed to by default_dispvm
not necessary is one.

The property defaults to the value from the template (if any), and then
to a global management_dispvm property. By default it is set to None.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-12-03 19:18:26 +01:00
Marek Marczykowski-Górecki
4dc8631010
Use maxmem=0 to disable qmemman, add more automation to it
Use maxmem=0 for disabling dynamic memory balance, instead of cryptic
service.meminfo-writer feature. Under the hood, meminfo-writer service
is also set based on maxmem property (directly in qubesdb, not
vm.features dict).
Having this as a property (not "feature"), allow to have sensible
handling of default value. Specifically, disable it automatically if
otherwise it would crash a VM. This is the case for:
 - domain with PCI devices (PoD is not supported by Xen then)
 - domain without balloon driver and/or meminfo-writer service

The check for the latter is heuristic (assume presence of 'qrexec' also
can indicate balloon driver support), but it is true for currently
supported systems.

This also allows more reliable control of libvirt config: do not set
memory != maxmem, unless qmemman is enabled.

memory != maxmem only makes sense if qmemman for given domain is
enabled.  Besides wasting some domain resources for extra page tables
etc, for HVM domains this is harmful, because maxmem-memory difference
is made of Popupate-on-Demand pool, which - when depleted - will kill
the domain. This means domain without balloon driver will die as soon
as will try to use more than initial memory - but without balloon driver
it sees maxmem memory and doesn't know about the lower limit.

Fixes QubesOS/qubes-issues#4135
2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
35a53840f1
vm: send domain-start-failed event also if some device is missing
Checking device presence wasn't covered with try/except that send the
event.
2018-11-15 18:25:29 +01:00
Marek Marczykowski-Górecki
0eab082d85
ext/core-features: make 'template-postinstall' event async
It makes a lot of sense to call long-running operations in that event
handler, including calling back into the VM. Allow that by using
fire_event_async, not just fire_event.

Also, document the event.
2018-11-15 18:25:29 +01:00
Marek Marczykowski-Górecki
328697730b
vm: fix deadlock on qrexec timeout handling
vm.kill() will try to get vm.startup_lock, so it can't be called while
holding it already.
Fix this by extracting vm._kill_locked(), which expect the lock to be
already taken by the caller.
2018-11-04 17:05:55 +01:00