Commit Graph

119 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
c093e14077
Merge branch 'libvirt-template-r41'
* libvirt-template-r41:
  libvirt: adjust domain xml template for upstream PVH format
2019-03-08 18:07:07 +01:00
Marek Marczykowski-Górecki
933882ee9b
tests: fix race condition in firewall formatting test
Use fixed timestamp.
2019-02-27 15:38:22 +01:00
Marek Marczykowski-Górecki
2de5a8e894
vm,templates: allow to obtain common kernelopts from a kernel package
If kernel package ships default-kernelopts-common.txt file, use that
instead of hardcoded Linux-specific options.
For Linux kernel it may include xen_scrub_pages=0 option, but only if
initrd shipped with this kernel re-enable this option later.

QubesOS/qubes-issues#4839
QubesOS/qubes-issues#4736
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
adcc9d7e77
libvirt: adjust domain xml template for upstream PVH format
The final PVH support upstream use different machine type name.
2019-02-19 00:59:05 +01:00
Wojtek Porczyk
ff612a870b qubes/features: check_with_(template_and_)adminvm
- Two new methods: .features.check_with_adminvm() and
  .check_with_template_and_adminvm(). Common code refactored.

- Two new AdminAPI calls to take advantage of the methods:
  - admin.vm.feature.CheckWithAdminVM
  - admin.vm.feature.CheckWithTemplateAndAdminVM

- Features manager moved to separate module in anticipation of features
  on app object in R5.0. The attribute Features.vm renamed to
  Features.subject.

- Documentation, tests.
2018-12-20 18:28:33 +01:00
Marek Marczykowski-Górecki
3728230e3c
Merge branch 'maxmem' 2018-12-09 18:38:21 +01:00
Marek Marczykowski-Górecki
7a6e4b284f
vm/mix/net: prevent setting provides_network=false if qube is still used
Add symmetric check to the one in 'netvm' property.

Fixes QubesOS/qubes-issues#4552
2018-12-06 20:37:50 +01:00
Marek Marczykowski-Górecki
162196a1f8
tests: fix lookup domain by name 2018-12-04 16:12:09 +01:00
Marek Marczykowski-Górecki
7d1bcaf64c Introduce management_dispvm property
The new property is meant for management stack (Salt) to set which DVM
template should be used to maintain given VM. Since the DispVM based on
it will be given ultimate control over target VM (qubes.VMShell
service), it should be trusted. The one pointed to by default_dispvm
not necessary is one.

The property defaults to the value from the template (if any), and then
to a global management_dispvm property. By default it is set to None.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-12-03 19:18:26 +01:00
Marek Marczykowski-Górecki
62bc462a23
tests: default maxmem 2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
b8052f864a
tests: more cases for libvirt xml generation
Related to automatic mem balance enabling/disabling. Check how it behave
in presence of PCI devices, or explicit disabling it.
2018-11-21 02:13:25 +01:00
Marek Marczykowski-Górecki
4dc8631010
Use maxmem=0 to disable qmemman, add more automation to it
Use maxmem=0 for disabling dynamic memory balance, instead of cryptic
service.meminfo-writer feature. Under the hood, meminfo-writer service
is also set based on maxmem property (directly in qubesdb, not
vm.features dict).
Having this as a property (not "feature"), allow to have sensible
handling of default value. Specifically, disable it automatically if
otherwise it would crash a VM. This is the case for:
 - domain with PCI devices (PoD is not supported by Xen then)
 - domain without balloon driver and/or meminfo-writer service

The check for the latter is heuristic (assume presence of 'qrexec' also
can indicate balloon driver support), but it is true for currently
supported systems.

This also allows more reliable control of libvirt config: do not set
memory != maxmem, unless qmemman is enabled.

memory != maxmem only makes sense if qmemman for given domain is
enabled.  Besides wasting some domain resources for extra page tables
etc, for HVM domains this is harmful, because maxmem-memory difference
is made of Popupate-on-Demand pool, which - when depleted - will kill
the domain. This means domain without balloon driver will die as soon
as will try to use more than initial memory - but without balloon driver
it sees maxmem memory and doesn't know about the lower limit.

Fixes QubesOS/qubes-issues#4135
2018-11-21 02:13:25 +01:00
Rusty Bird
bee69a98b9
Add default_qrexec_timeout to qubes-prefs
When a VM (or its template) does not explicitly set a qrexec_timeout,
fall back to a global default_qrexec_timeout (with default value 60),
instead of hardcoding the fallback value to 60.

This makes it easy to set a higher timeout for the whole system, which
helps users who habitually launch applications from several (not yet
started) VMs at the same time. 60 seconds can be too short for that.
2018-09-16 18:42:48 +00:00
Marek Marczykowski-Górecki
af2435c0d4
Make some properties default to template's value (if any)
Multiple properties are related to system installed inside the VM, so it
makes sense to have them the same for all the VMs based on the same
template. Modify default value getter to first try get the value from a
template (if any) and only if it fails, fallback to original default
value.
This change is made to those properties:
 - default_user (it was already this way)
 - kernel
 - kernelopts
 - maxmem
 - memory
 - qrexec_timeout
 - vcpus
 - virt_mode

This is especially useful for manually installed templates (like
Windows).

Related to QubesOS/qubes-issues#3585
2018-07-16 22:02:58 +02:00
Marek Marczykowski-Górecki
2dee554ab7
vm/mix/net: make vm.gateway6 consistent with vm.gateway
Use VM's actual IP address as a gateway for other VMs, instead of
hardcoded link-local address. This is important for sys-net generated
ICMP diagnostics packets - those must _not_ have link-local source
address, otherwise wouldn't be properly forwarded back to the right VM.
2018-04-03 00:20:06 +02:00
Yassine Ilmi
65d5c41fdb
Missed one test, adding default-user in assert for test test_621_qdb_vm_with_network in TC_90 2018-02-01 01:03:05 +00:00
Yassine Ilmi
a0d45aac9c
replaced underscore by dash and update test accordingly 2018-02-01 00:50:42 +00:00
Yassine Ilmi
0eb978c59b
Updated assert content for test_620_qdb_standalone in TC_90_QubesVM 2018-02-01 00:33:12 +00:00
Marek Marczykowski-Górecki
2c2b89e2e0
tests: adjust for type=pvh change 2018-01-27 01:34:41 +01:00
Marek Marczykowski-Górecki
eb846f6647
Merge remote-tracking branch 'qubesos/pr/187'
* qubesos/pr/187:
  Don't fail create/clone if /var/lib/qubes/TYPE/NAME/ exists
  Make 'qvm-volume revert' really use the latest revision
  Fix wrong mocks of Volume.revisions
2018-01-22 15:39:13 +01:00
Marek Marczykowski-Górecki
74eb3f3208
Merge remote-tracking branch 'qubesos/pr/185'
* qubesos/pr/185:
  vm: remove doc for non-existing event `monitor-layout-change`
  vm: include tag/feature name in event name
  events: add support for wildcard event handlers
2018-01-22 15:32:57 +01:00
Rusty Bird
4ae854fdaf
Don't fail create/clone if /var/lib/qubes/TYPE/NAME/ exists 2018-01-21 22:28:47 +00:00
Marek Marczykowski-Górecki
d2a7cbb83e
tests: mock vmm.xs
Now it is needed by some unit tests (those calling create_qdb_entries).
2018-01-15 15:58:34 +01:00
Marek Marczykowski-Górecki
85e80f2329
vm/qubesvm: revert backup_timestamp to '%s' format
Human readable format `str(datetime.datetime)` is a nightmare for Admin
API level communication. Especially setting the property in a format
that it was read was not supported, and handling such format in
untrusted input handling code is a bad idea. Revert to a simple intiger
format.
2018-01-12 05:34:45 +01:00
Marek Marczykowski-Górecki
50d34755fa
vm: include tag/feature name in event name
Rename events:
 - domain-feature-set -> domain-feature-set:feature
 - domain-feature-delete -> domain-feature-delete:feature
 - domain-tag-add -> domain-tag-add:tag
 - domain-tag-delete -> domain-tag-delete:tag

Make it consistent with property-* events. It makes more sense to
include tag/feature name in event name, so handler can watch a single
tag/feature - which is the most common case. Otherwise, most handlers
would begin with `if feature == '...'` anyway, wasting time on most
events.

In cases where multiple features/tags should be handled by a single
handler, it is now possible to register a handler with wildcard, for
example `domain-feature-set:*`.
2018-01-06 15:05:34 +01:00
Marek Marczykowski-Górecki
466bf89aae
Fix starting VM with kernel=None
When dom0 do not provide the kernel, it should also not set kernel
command line in libvirt config. Otherwise qemu in stubdom fails to start
because it get -append option without -kernel, which is illegal
configuration.

Fixes QubesOS/qubes-issues#3339
2017-12-14 23:26:52 +01:00
Marek Marczykowski-Górecki
1187e43697
tests: more vm.create_qdb_entries() tests 2017-12-14 02:09:31 +01:00
Marek Marczykowski-Górecki
e12a66f103
vm/mix/net: use ipaddress module for ip and ip6 properties
It has built-in validation, which is much more elegant than custom regex
or socket call.

Suggested by @woju
QubesOS/qubes-issues#718
2017-12-07 01:40:31 +01:00
Marek Marczykowski-Górecki
0786edf8a3
tests: add IPv6-related unit tests
Check produced libvirt XML, and QubesDB entries

QubesOS/qubes-issues#718
2017-12-07 01:40:30 +01:00
Marek Marczykowski-Górecki
18f159f8ec
Add IPv6 related VM properties
Add property for IPv6 address ('ip6'). Build default value similarly to
IPv4 - common prefix + QID or Disp ID (for DispVMs).
This all is disabled unless 'ipv6' feature is enabled. It is inherited
from netvm (not template).
Even when enabled, VM may decide to not use it - or simply not support
it.

QubesOS/qubes-issues#718
2017-12-07 01:40:30 +01:00
Marek Marczykowski-Górecki
088c255355
tests: add create_qdb_entries() unit test 2017-12-05 17:39:34 +01:00
Marek Marczykowski-Górecki
0afee4b05e
Improve checking for netvm loop
There were many cases were the check was missing:
 - changing default_netvm
 - resetting netvm to default value
 - loading already broken qubes.xml

Since it was possible to create broken qubes.xml using legal calls, do
not reject loading such file, instead break the loop(s) by setting netvm
to None when loop is detected. This will be also useful if still not all
places are covered...

Place the check in default_netvm setter. Skip it during qubes.xml loading
(when events_enabled=False), but still keep it in setter, to _validate_ the
value before any property-* event got fired.
2017-12-05 17:39:33 +01:00
Marek Marczykowski-Górecki
3217c3ac4e
Merge remote-tracking branch 'qubesos/pr/156'
* qubesos/pr/156:
  tests: Add unit tests for pvh virt_mode
  Don't allow attached PCI devices and virt_mode = 'pvh'
2017-11-20 22:40:53 +01:00
HW42
c695f3268c linux-stubdom: Set cmdline in HVM config 2017-10-30 16:19:01 +01:00
Marek Marczykowski-Górecki
2164a8d7b8
Change license to LGPL v2.1+
See this thread for reasoning and acceptance from contributors:
https://groups.google.com/d/topic/qubes-devel/G7KzrfU0lWY/discussion
"Changing qubes-core-admin license to LGPL v2.1+"
2017-10-12 00:11:50 +02:00
HW42
4a84d5b222 tests: Add unit tests for pvh virt_mode 2017-10-11 21:11:36 +02:00
Marek Marczykowski-Górecki
3a5e8482c0
Merge branch 'fixes-20170929'
* fixes-20170929:
  vm: do not start QubesDB watch instance multiple times
  vm: report storage.stop() errors to log
  vm: move comment
  storage: fix method name in LinuxModules volume
  Prevent removing domain that is referenced from anywhere
  vm: add vm.klass property
  Move QubesVM.{name,qid,uuid,label} to BaseVM
  vm: do not allow deleting template property from AppVM and DispVM
  vm/qubesvm: emit event on failed startup
  vm/qubesvm: remove duplicated qmemman_client.close()
  vm/dispvm: cleanup DispVM also on failed startup
  vm/dispvm: fix error message
  ext/block: properly list devtype=cdrom option
  block: fix handling non-existing devices
  block: improve handling device name and description
2017-10-03 12:04:00 +02:00
Marek Marczykowski-Górecki
12b7e22d27
vm: do not start QubesDB watch instance multiple times
vm.create_qdb_entries can be called multiple times - for example when
changing VM IP. Move starting qdb watcher to start(). And just in case,
cleanup old watcher (if still exists) before starting new one.
This fixes one FD leak.
2017-09-29 12:05:23 +02:00
Marek Marczykowski-Górecki
3075767bb8
tests: add more complex test for netvm cyclic connection
Something more complex than VM connecting to itself.
2017-09-29 11:53:57 +02:00
Marek Marczykowski-Górecki
9f88fa7f0c
Move QubesVM.{name,qid,uuid,label} to BaseVM
Reduce strange code in BaseVM (accessing non-existing self.name) and
code duplication.
2017-09-29 11:53:00 +02:00
Marek Marczykowski-Górecki
dc0e1a5481
vm: do not allow deleting template property from AppVM and DispVM
There is intentionally no default template in terms of qubes.property
definition, to not cause problems when switching global default_template
property - like breaking some VMs, or forcing the user to shutdown all
of them for this. But this also means it shouldn't be allowed to reset
template to "default" value, because it will result in a VM without
template at all.

Fixes QubesOS/qubes-issues#3115
2017-09-29 11:53:00 +02:00
Marek Marczykowski-Górecki
e0cadfdd67
Merge remote-tracking branch 'qubesos/pr/150'
* qubesos/pr/150:
  qubes/tests: moar fixes
  test-packages: add missing libvirt classes
  qubes/tests: do not deadlock on .drain()
  qubes/vm: put name= first in __repr__
  tests: fix some memory leaks
  tests: complain about memory leaks
  tests: use one event loop and one libvirtaio impl
2017-09-22 20:54:11 +02:00
Wojtek Porczyk
bd123a3812 tests: fix some memory leaks
This is work in progress. There are still more leaks.
2017-09-19 17:01:29 +02:00
HW42
c6c6f681ae tests: Fix expected HVM xml
The last two commits (intentionally) changed the libxl xml. So adapt the
expected value in the test.
2017-09-15 16:19:10 +02:00
Marek Marczykowski-Górecki
99bd193688
Rename 'dispvm_allowed' to 'template_for_dispvms'
'dispvm_allowed' name was confusing, because it suggested being able to
spawn new DispVMs, not being a template for DispVM.

Fixes QubesOS/qubes-issues#3047
2017-09-05 02:07:26 +02:00
Marek Marczykowski-Górecki
c247ddff72
dispvm: fix setting up new DispVM
Clone properties from DispVM template after setting base properties
(qid, name, uuid). This means we can use standard clone_properties()
function. Otherwise various setters may fail - for example
netvm setter require uuid property initialized (for VM lookup in VM
collection).
Also, make dispvm_allowed check more robust - include direct creation of
DispVM, and also check just before VM startup (if property was changed
in the meantime).

Fixes QubesOS/qubes-issues#3057
2017-09-05 02:07:26 +02:00
Marek Marczykowski-Górecki
691a6f4d8c
vm/dispvm: add auto_cleanup property, unify creating new DispVM
Add auto_cleanup property, which remove DispVM after its shutdown
- this is to unify DispVM handling - less places needing special
handling after DispVM shutdown.

New DispVM inherit all settings from respective AppVM. Move this from
classmethod `DispVM.from_appvm()`, to DispVM constructor. This unify
creating new DispVM with any other VM class.
Notable exception are attached devices - because only one running VM can
have a device attached, this would prevent second DispVM started from
the same AppVM. If one need DispVM with some device attached, one can
create DispVM with auto_cleanup=False. Such DispVM will still not have
persistent storage (as any other DispVM).

Tests included.

QubesOS/qubes-issues#2974
2017-08-06 20:54:09 +02:00
Marek Marczykowski-Górecki
34ff40c721
tests: fix AdminVM test 2017-07-29 05:07:45 +02:00
Marek Marczykowski-Górecki
5d9bc00885
tests: fix AdminVM test
Since it is no longer child of QubesVM, constructor do not take 'qid'
and 'name' arguments.
Also:
- remove other dropped properties tests (netvm, storage related)
- make the test working in non-dom0
2017-07-26 03:26:40 +02:00
Marek Marczykowski-Górecki
c5667791e8
storage: move volume_config['source'] filling to one place
Don't set 'source' volume in various places (each VM class constructor
etc), do it as part of volume initialization. And when it needs to be
re-calculated, call storage.init_volume again.

This code was duplicated, and as usual in such a case, those copies
were different - one have set 'size', the other one not.

QubesOS/qubes-issues#2256
2017-07-26 03:26:11 +02:00