Commit Graph

5410 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
9b5256f002
version 4.0.21 2018-01-30 00:18:20 +01:00
Marek Marczykowski-Górecki
a6a7efc9a7
vm/mix/net: fix handling network detach/attach on VM startup
- catch both QubesException and libvirtError - do not kill starting VM
just because an error while connecting _other_ VMs to it
- try to detach network first (and do not abort on error) - if
libvirt/libxl will manage to cleanup stale interface this way, the
attach operation below may succeed.

Fixes QubesOS/qubes-issues#3163
2018-01-29 23:06:21 +01:00
Marek Marczykowski-Górecki
86026e364f
Fix starting PCI-having HVMs on early system boot and later
1. Make sure VMs are started after dom0 actual memory usage is reported
to qmemman, otherwise dom0 will hold 4GB, even if just a little over 1GB
is needed at that time.

2. Request only vm.memory MB from qmemman, instead of vm.maxmem. While
HVM with PCI devices indeed do not support populate-on-demand, this is
already handled in libvirt XML.

The later may often cause VM startup fail on systems with 8GB of memory,
because maxmem is 4GB there and with dom0 keeping the other 4GB (see
point 1) there is not enough memory to start any sych VM.

Fixes QubesOS/qubes-issues#3462
2018-01-29 22:57:32 +01:00
Marek Marczykowski-Górecki
2a8fd9399e
version 4.0.20 2018-01-27 01:47:54 +01:00
Marek Marczykowski-Górecki
2c2b89e2e0
tests: adjust for type=pvh change 2018-01-27 01:34:41 +01:00
Marek Marczykowski-Górecki
76836d4ff6
rpm: require libvirt new enough for type-pvh 2018-01-27 01:21:05 +01:00
Marek Marczykowski-Górecki
99134efc3c
tests: set virt_mode='hvm' for PCI tests 2018-01-26 23:49:11 +01:00
Simon Gaiser
c3d287a33c Switch to new PVH config variant 2018-01-24 02:48:00 +01:00
Marek Marczykowski-Górecki
30ba4b8e3b
version 4.0.19 2018-01-23 14:14:09 +01:00
Marek Marczykowski-Górecki
eb846f6647
Merge remote-tracking branch 'qubesos/pr/187'
* qubesos/pr/187:
  Don't fail create/clone if /var/lib/qubes/TYPE/NAME/ exists
  Make 'qvm-volume revert' really use the latest revision
  Fix wrong mocks of Volume.revisions
2018-01-22 15:39:13 +01:00
Marek Marczykowski-Górecki
e577de7d49
Merge remote-tracking branch 'qubesos/pr/186'
* qubesos/pr/186:
  tests: check if udev do not touch VM's volumes
2018-01-22 15:33:50 +01:00
Marek Marczykowski-Górecki
74eb3f3208
Merge remote-tracking branch 'qubesos/pr/185'
* qubesos/pr/185:
  vm: remove doc for non-existing event `monitor-layout-change`
  vm: include tag/feature name in event name
  events: add support for wildcard event handlers
2018-01-22 15:32:57 +01:00
Marek Marczykowski-Górecki
2041e6f01c
rpm: require new enough libvirt for pci permissive setting 2018-01-22 15:31:12 +01:00
Marek Marczykowski-Górecki
9bd75d7808
Merge remote-tracking branch 'qubesos/pr/184'
* qubesos/pr/184:
  pci: Add permissive options

Fixes QubesOS/qubes-issues#3476
2018-01-22 15:30:26 +01:00
Rusty Bird
4ae854fdaf
Don't fail create/clone if /var/lib/qubes/TYPE/NAME/ exists 2018-01-21 22:28:47 +00:00
Rusty Bird
fe77b0ec85
Make 'qvm-volume revert' really use the latest revision
admin.vm.volume.ListSnapshots returned volume revisions in undefined
order, but 'qvm-volume revert' assumes the list to be in chronological
order. Make that assumption true.
2018-01-20 23:20:23 +00:00
Rusty Bird
bf1f1ac5ff
Fix wrong mocks of Volume.revisions
It's a dict, not a list.
2018-01-20 23:20:22 +00:00
Marek Marczykowski-Górecki
ab15d43622
tests: check if udev do not touch VM's volumes
Based on shell version by @rustybird
2018-01-19 18:05:50 +01:00
Marek Marczykowski-Górecki
91639d7933
app: define values for default_dispvm=None, updatevm=None
If those VMs are not set, default to None, instead of throwing
AttributeError.

Fixes QubesOS/qubes-issues#3475
2018-01-19 04:50:04 +01:00
Marek Marczykowski-Górecki
206d3f136d
version 4.0.18 2018-01-19 02:45:34 +01:00
Marek Marczykowski-Górecki
2691efda8e
Merge branch 'devel-20180115'
* devel-20180115:
  tests: update PCI devices tests for core3 API
  tests: make tests.extra.VMWrapper hashable
  qmemman: do not close stdout/stderr in daemon mode
  qmemman: fix early crash
  qubesvm: do not try to define libvirt object in offline mode
  tests: one more missing virt_mode=hvm
  qubesvm: PVH minor improvements
  tests: minor fixes
  tests: add run_service and qrexec_policy wrappers to ExtraTestCase
  tests: clear PCIDevice cache after each test
2018-01-19 02:45:11 +01:00
Simon Gaiser
132f0608aa pci: Add permissive options 2018-01-19 02:03:45 +01:00
Marek Marczykowski-Górecki
faaebcc268
version 4.0.17 2018-01-18 22:51:56 +01:00
Marek Marczykowski-Górecki
5ea8eda3ea
replace console entry points with just importing the module
importing pkg_resources and looking up entry points wastes 100ms+
of time, which is totally unnecessary

This is based on
QubesOS/qubes-core-admin-client@b731ef3885
by @qubesuser
2018-01-18 22:13:37 +01:00
Marek Marczykowski-Górecki
e9cc6ee3db
version 4.0.16 2018-01-18 21:07:34 +01:00
Marek Marczykowski-Górecki
b245bbca6f
tests: update PCI devices tests for core3 API
- use asyncio where needed
- attach now takes DeviceAssignment, not DeviceInfo
- PCI ident have ':' replaced with '_'
2018-01-18 17:36:38 +01:00
Marek Marczykowski-Górecki
edbfd3843e
tests: make tests.extra.VMWrapper hashable
Allow using VMWrapper as dict key, same as QubesVM.
2018-01-18 17:36:37 +01:00
Marek Marczykowski-Górecki
46177c7c9f
qmemman: do not close stdout/stderr in daemon mode
Allow exceptions to be logged to syslog/journald
2018-01-18 17:36:37 +01:00
Marek Marczykowski-Górecki
ca41ca66cd
qmemman: fix early crash
clear_outdated_error_markers crashes if memory stats are not retrieved
yet. In practice it crashes at the very first call during daemon
startup, making the whole qmemman unusable.

This fixes bf4306b815
    qmemman: clear "not responding" flags when VM require more memory

QubesOS/qubes-issues#3265
2018-01-18 17:36:37 +01:00
Marek Marczykowski-Górecki
dce3b609b4
qubesvm: do not try to define libvirt object in offline mode
The idea is to not touch libvirt at all.
2018-01-18 17:36:37 +01:00
Marek Marczykowski-Górecki
f2b9be3607
tests: one more missing virt_mode=hvm 2018-01-17 15:23:22 +01:00
Marek Marczykowski-Górecki
7905783861
qubesvm: PVH minor improvements
- use capital letters in acronyms in documentation to match upstream
documentation.
- refuse to start a PVH with without kernel set - provide meaningful
error message
2018-01-16 21:42:20 +01:00
Marek Marczykowski-Górecki
4d59f883a0
tests: minor fixes
- FD leak
- switch to xterm to test also on minimal template
2018-01-16 21:41:38 +01:00
Marek Marczykowski-Górecki
06e82eccb0
tests: add run_service and qrexec_policy wrappers to ExtraTestCase
Provide same API as in core2, especially without exposing asyncio
usage. This allows qubes-usb-proxy and qubes-split-gpg tests to run.
2018-01-16 21:39:22 +01:00
Marek Marczykowski-Górecki
c17b634913
tests: clear PCIDevice cache after each test
This is yet another place where references to VM objects contribute to
object leaks.
2018-01-16 21:32:15 +01:00
Rusty Bird
8506547923
block-snapshot: fix ShellCheck warnings (3/3)
These were the remaining warnings that only affected the newer
block-snapshot in this git repo.
2018-01-16 06:20:02 +00:00
Rusty Bird
50f35b3ec1
block-snapshot: fix ShellCheck warnings (2/3)
Forward-ported from qubes-core-agent-linux:

    commit aad6fa6d190d24393e326a4c2ff7ebc3b5921641
    Author: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
    Date:   Sat Sep 30 04:56:02 2017 +0200

        Hint shellcheck where to look for sourced files, if in repository

        This will ease running shellcheck from the repository.
2018-01-16 06:20:01 +00:00
Rusty Bird
45a06e7fad
block-snapshot: fix ShellCheck warnings (1/3)
Forward-ported from qubes-core-agent-linux:

    commit e95b6f8d0357ac9a32fb7e35e5734781460d8f24
    Author: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
    Date:   Sat Sep 30 04:53:38 2017 +0200

        Fix shellcheck warnings in block-snapshot script
2018-01-16 06:20:00 +00:00
Rusty Bird
342c36ed36
block-snapshot: remove trailing spaces
Forward-ported from qubes-core-agent-linux:

    commit 2eb0ed2be14350d6df1fce2af855805133a4a416
    Author: Patrick Schleizer <adrelanos@riseup.net>
    Date:   Thu Oct 15 04:34:55 2015 +0200

        removed trailing spaces
2018-01-16 06:19:58 +00:00
Rusty Bird
9dade5c219
block-snapshot: remove R3.2 template commit code
qvm-template-commit does not exist in R4.0, where FileVolume.stop()
takes care of committing.
2018-01-16 06:19:57 +00:00
Marek Marczykowski-Górecki
d2a7cbb83e
tests: mock vmm.xs
Now it is needed by some unit tests (those calling create_qdb_entries).
2018-01-15 15:58:34 +01:00
Marek Marczykowski-Górecki
ae7031fe7e
tests: explicitly set virt_mode to HVM when needed
HVM is no longer default
2018-01-15 15:57:54 +01:00
Marek Marczykowski-Górecki
241f1d1d3b
tests: do not leak open file 2018-01-15 15:57:30 +01:00
Marek Marczykowski-Górecki
f1a5ca64fd
Merge remote-tracking branch 'qubesos/pr/180'
* qubesos/pr/180:
  vm/qubesvm: default to PVH unless PCI devices are assigned
  vm/qubesvm: expose 'start_time' property over Admin API
  vm/qubesvm: revert backup_timestamp to '%s' format
  doc: link qvm-device man page for qvm-block, qvm-pci, qvm-usb
2018-01-15 04:22:28 +01:00
Marek Marczykowski-Górecki
21760d8ff0
Merge remote-tracking branch 'qubesos/pr/179'
* qubesos/pr/179:
  qmemman: request VMs balloon down with 16MB safety margin
  qmemman: clear "not responding" flags when VM require more memory
  qmemman: slightly improve logging
  qmemman: reformat code, especially comments
2018-01-15 04:21:40 +01:00
Marek Marczykowski-Górecki
b20c3d3458
Merge remote-tracking branch 'qubesos/pr/174'
* qubesos/pr/174:
  tests: fix (system) network tests after switching to ipaddress module
  tests: resurrect extra tests loader
  tests: basic salt integration tests
2018-01-15 04:20:32 +01:00
Marek Marczykowski-Górecki
4ff53879a0
vm/qubesvm: default to PVH unless PCI devices are assigned
Fixes QubesOS/qubes-issues#2185
2018-01-15 03:34:46 +01:00
Marek Marczykowski-Górecki
d9da747ab0
vm/qubesvm: expose 'start_time' property over Admin API
It is useful at least for Qubes Manager.
2018-01-12 05:34:46 +01:00
Marek Marczykowski-Górecki
85e80f2329
vm/qubesvm: revert backup_timestamp to '%s' format
Human readable format `str(datetime.datetime)` is a nightmare for Admin
API level communication. Especially setting the property in a format
that it was read was not supported, and handling such format in
untrusted input handling code is a bad idea. Revert to a simple intiger
format.
2018-01-12 05:34:45 +01:00
Marek Marczykowski-Górecki
bcf42c13fa
storage/lvm: check for LVM LV existence and type when creating ThinPool
Check if requested thin pool exists and really is thin pool.

QubesOS/qubes-issues#3438
2018-01-12 05:12:08 +01:00