Commit Graph

5725 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
933882ee9b
tests: fix race condition in firewall formatting test
Use fixed timestamp.
2019-02-27 15:38:22 +01:00
Marek Marczykowski-Górecki
2de5a8e894
vm,templates: allow to obtain common kernelopts from a kernel package
If kernel package ships default-kernelopts-common.txt file, use that
instead of hardcoded Linux-specific options.
For Linux kernel it may include xen_scrub_pages=0 option, but only if
initrd shipped with this kernel re-enable this option later.

QubesOS/qubes-issues#4839
QubesOS/qubes-issues#4736
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
e110cbecb3
storage: fallback kernels_dir path if there is no 'kernel' volume
Return meaningful value for kernels_dir if VM has no 'kernel' volume.
Right now it's mostly useful for tests, but could be also used for new
VM classes which doesn't have modules.img, but still use dom0-provided
kernel.
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
723b33a2f6
vm: move comment about memory overhead constants near their definition 2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
9257a6d14f
Do not abort suspend hooks if any qubes.Suspend* service fails to run
First of all, do not try to call those services in VMs not having qrexec
installed - for example Windows VMs without qubes tools.
Then, even if service call fails for any other reason, only log it but
do not prevent other services from being called. A single uncooperative
VM should generally be able only to hurt itself, not break other VMs
during suspend.

Fixes QubesOS/qubes-issues#3489
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
eeec2e0ddd
qubespolicy: forbid qrexec loopback connections at policy level
libxenvchan currently can't handle loopback connections. Since error
reporting on vchan connection setup is far from perfect, try to avoid
making such connections at all.

QubesOS/qubes-issues#951
Fixes QubesOS/qubes-issues#4804
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
202e3df6b6
vm/mixin/net: disconnect network interface on backend shutdown/crash
Since we have more reliable domain-shutdown event delivery (it si
guaranteed to be delivered before subsequent domain start, even if
libvirt fails to report it), it's better to move detach_network call to
domain-shutdown handler. This way, frontend domain will see immediately
that the backend is gone. Technically it already know that, but at least
Linux do not propagate that anywhere, keeping the interface up,
seemingly operational, leading to various timeouts.
Additionally, by avoiding attach_network call _just_ after
detach_network call, it avoids various race conditions (like calling
cleanup scripts after new device got already connected).

While libvirt itself still doesn't cleanup devices when the backend
domain is gone, this will emulate it within qubesd.

Fixes QubesOS/qubes-issues#3642
Fixes QubesOS/qubes-issues#1426
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
357231ca8f
tests: network reconnect after network provider shutdown/crash
QubesOS/qubes-issues#3642
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
e69bd3c572
tests: fix race condition between mkfs and udev
Just after LVM volume is created, udev tries to analyze it. This
prevents mkfs from creating filesystem there.
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
50a89aac3f
tests: check if page scrubbing is still enabled when switching kernel
QubesOS/qubes-issues#4736
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
5222650868
vm/adminvm: add stubs for kill/shutdown functions
Report proper error message, instead of throwing AttributeError on such
calls.

Fixes QubesOS/qubes-issues#4645
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
318ed439ff
storage/file: gracefully handle not mounted pool
Fixes QubesOS/qubes-issues#4668
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
27a0fe25ab
app: allow removing VM referencing only itself
Fixes QubesOS/qubes-issues#4224
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
7433f83581
tests: add in-use storage pool removal test
QubesOS/qubes-issues#4454
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
5aa49168e5
tests: make storage.TC_00_Pool usable outside of dom0 2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
2bd709501b
tests: adjust cleanup to added in-use storage pool detection on removal 2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
80e57e16be
Allow setting default_template to none
It may make sense to force explicit template choice on VM creation,
especially with more restrictive qrexec policy.
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
4f5687440f
Prevent removal of in-use storage pool
Fixes QubesOS/qubes-issues#4454
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
23bfc18535
Add pool-add, pool-pre-delete, pool-delete events 2019-02-27 06:04:01 +01:00
Marek Marczykowski-Górecki
d8b6d3efde
Make add_pool/remove_pool coroutines, allow Pool.{setup,destroy} as coroutines
Pool setup/destroy may be a time consuming operation, allow them to be
asynchronous. Fortunately add_pool and remove_pool are used only through
Admin API, so the change does not require modification of other
components.
2019-02-27 06:03:57 +01:00
Marek Marczykowski-Górecki
adbc618545
Use correct bool properties parsing by default
Boolean properties require specific setter to properly handle literal
"True" and "False" values. Previously it required all bool properties to
include 'setter=qubes.property.bool' in addition to 'type=bool'.
This fixes loading some boolean properties from qubes.xml. Specifically
at least include_in_backups on DispVM class lacked setter, which
resulted in property being reset to True automatically on qubesd
restart.

Fixes QubesOS/qubes-issues#4831
2019-02-23 15:43:19 +01:00
Marek Marczykowski-Górecki
f9593ce3e6
vm: allow files in kernels_dir override built-in default kernelopts
If default-kernelopts-pci.txt is present, it will override default
built-in kernelopts for the VMs with PCI device assigned.
Similarly if default-kernelopts-nopci.txt is present, it will override
default kernelopts for VMs without PCI devices.
For template-based VMs, kernelopts of the template takes precedence over
default-kernelopts-nopci.txt but not default-kernelopts-pci.txt.

Fixes QubesOS/qubes-issues#4839
2019-02-23 12:53:49 +01:00
Nicolas Iooss
c5cfb81b94
doc: fix paths of jinja templates for libvirt Xen configuration
The jinja templates are located in either /etc/qubes/templates or
/usr/share/qubes/templates. /usr/share/templates is not used and
/usr/share/qubes/template/ does not exist.

Fix the paths that are documented in
https://dev.qubes-os.org/projects/core-admin/en/latest/libvirt.html
2019-02-20 23:49:57 +01:00
Marek Marczykowski-Górecki
5b28c17eed
version 4.0.39 2019-02-19 22:33:59 +01:00
Marek Marczykowski-Górecki
adcc9d7e77
libvirt: adjust domain xml template for upstream PVH format
The final PVH support upstream use different machine type name.
2019-02-19 00:59:05 +01:00
Marek Marczykowski-Górecki
8962452502
Make DispVMs started from a DispVM to use the same DVM template by default
If a specific DVM template is used for given DispVM, make new DispVMs
called from it use the same DVM template (unless explicitly overridden).
This prevent various isolation bypass cases, like using a chain of
DispVMs to access network.
2019-02-18 19:25:26 +01:00
Marek Marczykowski-Górecki
322306ec65
version 4.0.38 2019-01-22 00:31:59 +01:00
Marek Marczykowski-Górecki
86f856af57
Merge branch 'devel-20181130'
* devel-20181130:
  rpm: drop uneeded dependencies:
2019-01-22 00:30:44 +01:00
Marek Marczykowski-Górecki
bfb09f567f
Fix saving 'updates-available' flag reported by TemplateBasedVM
Look for the first updateable template up in the template chain, instead
of going just one level up. Especially this applies to
DispVM -> AppVM -> TemplateVM case.
If DispVM reports available updates, 'updates-available'
flag should be set on relevant TemplateVM, not AppVM (*-dvm).

Include test for the new case.

Fixes QubesOS/qubes-issues#3736
2019-01-19 03:25:21 +01:00
Marek Marczykowski-Górecki
18b0fa9db7
tests: improve mock setup in api_misc
Make self.src.template separate mock instance for better control over
results.
2019-01-19 03:25:20 +01:00
Marek Marczykowski-Górecki
a88733149d
Add logrotate for qubes logs
Fixes QubesOS/qubes-issues#3603
2019-01-19 03:25:20 +01:00
Marek Marczykowski-Górecki
a9ec2bb2c3
vm/qubesvm: fix race condition in failed startup handling
Instead of checking if domain is still running/paused, try to kill it
anyway and ignore appropriate exception. Otherwise domain could die
before the check and killing.
2019-01-19 03:25:20 +01:00
Marek Marczykowski-Górecki
4e5a14774a
storage/lvm: fix listing volume revisions when VM name include volume name
some-vm-root is a valid VM name, and in that case it's volume can be
named some-vm-root-private. Do not let it confuse revision listing,
check for unexpected '-' in volume revision number.

The proper solution would be to use different separator, that is not
allowed in VM names. But that would require migration code that is
undesired in the middle of stable release life cycle.

Fixes QubesOS/qubes-issues#4680
2019-01-19 03:25:19 +01:00
Marek Marczykowski-Górecki
dbd85c75e2
Merge branch 'tests-20181223'
* tests-20181223:
  tests: drop expectedFailure from qubes_desktop_run test
  tests: grub in HVM qubes
  tests: update dom0_update for new updates available flag
  tests: regression test LVM listing code
  tests/extra: wrap ProcessWrapper.wait() to be asyncio-aware
  tests: adjust backupcompat for new maxmem handling
2019-01-19 03:24:29 +01:00
Marek Marczykowski-Górecki
ed6acf4a33
Merge remote-tracking branch 'origin/pr/249'
* origin/pr/249:
  qubes/utils: parse_size: Fix string comparison
2019-01-17 22:28:36 +01:00
M. Vefa Bicakci
d934e8114a
qubes/utils: parse_size: Fix string comparison
This commit resolves a bug which causes strings such as "350MiB" to be
rejected by parse_size, due to the fact that parse_size changes the case
of letters in the input string ("350MiB") to uppercase ("350MIB"), but
fails to do the same for the elements of the units conversion table.

The correction is simple: Apply the same case change to the units
table elements before comparison.
2019-01-16 21:18:50 -05:00
Marek Marczykowski-Górecki
2181b64b97
Merge remote-tracking branch 'origin/pr/248'
* origin/pr/248:
  Drop lvm2-python-libs which is not used anymore
2019-01-15 21:17:05 +01:00
Marek Marczykowski-Górecki
186702646e
tests: drop expectedFailure from qubes_desktop_run test 2019-01-12 02:41:22 +01:00
Marek Marczykowski-Górecki
08621e7225
tests: grub in HVM qubes
Test grub in both PV and HVM. This change will also ease adding grub
test for PVH, when supported.
2019-01-12 02:41:22 +01:00
Marek Marczykowski-Górecki
adc3c4daa4
tests: update dom0_update for new updates available flag
Pending updates for dom0 are now signaled the same way as for templates
- with 'updates-available' feature, not a flag file.
2019-01-12 02:41:22 +01:00
Marek Marczykowski-Górecki
4c4e008368
tests: regression test LVM listing code
The revisions listing code mistakes VM name suffix with volume name.

QubesOS/qubes-issues#4680
2019-01-06 04:33:16 +01:00
Marek Marczykowski-Górecki
b08804e7c8
tests/extra: wrap ProcessWrapper.wait() to be asyncio-aware
The user of ExtraTestCase don't need to know anything about asyncio.
vm.run().wait() normally is a coroutine, but provide a wrapper that
handle asyncio.

This fixes FD leak in input proxy tests.
2019-01-06 04:33:16 +01:00
Marek Marczykowski-Górecki
0099aa1037
tests: adjust backupcompat for new maxmem handling
Since 4dc86310 "Use maxmem=0 to disable qmemman, add more automation to
it" meminfo-writer service is not accessible directly. maxmem property
is used to encode memory management instead.
2019-01-06 04:33:16 +01:00
Frédéric Pierret (fepitre)
536efb7519
Drop lvm2-python-libs which is not used anymore 2019-01-02 10:54:08 +01:00
Marek Marczykowski-Górecki
c5cd3628c6
doc: add qubes.devices to doc index 2018-12-27 01:44:38 +01:00
Wojtek Porczyk
17704c8d0a
qubes/ext/gui: set videoram qdb entries
This is for adjusting videoram size. See
QubesOS/qubes-gui-agent-linux@d222e16003
2018-12-20 18:28:40 +01:00
Wojtek Porczyk
ff612a870b qubes/features: check_with_(template_and_)adminvm
- Two new methods: .features.check_with_adminvm() and
  .check_with_template_and_adminvm(). Common code refactored.

- Two new AdminAPI calls to take advantage of the methods:
  - admin.vm.feature.CheckWithAdminVM
  - admin.vm.feature.CheckWithTemplateAndAdminVM

- Features manager moved to separate module in anticipation of features
  on app object in R5.0. The attribute Features.vm renamed to
  Features.subject.

- Documentation, tests.
2018-12-20 18:28:33 +01:00
Marek Marczykowski-Górecki
53ae36281e
version 4.0.37 2018-12-09 18:45:49 +01:00
Marek Marczykowski-Górecki
3728230e3c
Merge branch 'maxmem' 2018-12-09 18:38:21 +01:00
Marek Marczykowski-Górecki
3606010591
Merge branch 'devel-20181205'
* devel-20181205:
  vm/dispvm: fix /qubes-vm-presistence qubesdb entry
  vm/mix/net: prevent setting provides_network=false if qube is still used
  tests: updates-available notification
  tests/network: reduce code duplication
  tests: listen on 'misc' socket too
2018-12-09 18:35:35 +01:00