Commit Graph

5188 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
9bd75d7808
Merge remote-tracking branch 'qubesos/pr/184'
* qubesos/pr/184:
  pci: Add permissive options

Fixes QubesOS/qubes-issues#3476
2018-01-22 15:30:26 +01:00
Marek Marczykowski-Górecki
91639d7933
app: define values for default_dispvm=None, updatevm=None
If those VMs are not set, default to None, instead of throwing
AttributeError.

Fixes QubesOS/qubes-issues#3475
2018-01-19 04:50:04 +01:00
Marek Marczykowski-Górecki
206d3f136d
version 4.0.18 2018-01-19 02:45:34 +01:00
Marek Marczykowski-Górecki
2691efda8e
Merge branch 'devel-20180115'
* devel-20180115:
  tests: update PCI devices tests for core3 API
  tests: make tests.extra.VMWrapper hashable
  qmemman: do not close stdout/stderr in daemon mode
  qmemman: fix early crash
  qubesvm: do not try to define libvirt object in offline mode
  tests: one more missing virt_mode=hvm
  qubesvm: PVH minor improvements
  tests: minor fixes
  tests: add run_service and qrexec_policy wrappers to ExtraTestCase
  tests: clear PCIDevice cache after each test
2018-01-19 02:45:11 +01:00
Simon Gaiser
132f0608aa pci: Add permissive options 2018-01-19 02:03:45 +01:00
Marek Marczykowski-Górecki
faaebcc268
version 4.0.17 2018-01-18 22:51:56 +01:00
Marek Marczykowski-Górecki
5ea8eda3ea
replace console entry points with just importing the module
importing pkg_resources and looking up entry points wastes 100ms+
of time, which is totally unnecessary

This is based on
QubesOS/qubes-core-admin-client@b731ef3885
by @qubesuser
2018-01-18 22:13:37 +01:00
Marek Marczykowski-Górecki
e9cc6ee3db
version 4.0.16 2018-01-18 21:07:34 +01:00
Marek Marczykowski-Górecki
b245bbca6f
tests: update PCI devices tests for core3 API
- use asyncio where needed
- attach now takes DeviceAssignment, not DeviceInfo
- PCI ident have ':' replaced with '_'
2018-01-18 17:36:38 +01:00
Marek Marczykowski-Górecki
edbfd3843e
tests: make tests.extra.VMWrapper hashable
Allow using VMWrapper as dict key, same as QubesVM.
2018-01-18 17:36:37 +01:00
Marek Marczykowski-Górecki
46177c7c9f
qmemman: do not close stdout/stderr in daemon mode
Allow exceptions to be logged to syslog/journald
2018-01-18 17:36:37 +01:00
Marek Marczykowski-Górecki
ca41ca66cd
qmemman: fix early crash
clear_outdated_error_markers crashes if memory stats are not retrieved
yet. In practice it crashes at the very first call during daemon
startup, making the whole qmemman unusable.

This fixes bf4306b815
    qmemman: clear "not responding" flags when VM require more memory

QubesOS/qubes-issues#3265
2018-01-18 17:36:37 +01:00
Marek Marczykowski-Górecki
dce3b609b4
qubesvm: do not try to define libvirt object in offline mode
The idea is to not touch libvirt at all.
2018-01-18 17:36:37 +01:00
Marek Marczykowski-Górecki
f2b9be3607
tests: one more missing virt_mode=hvm 2018-01-17 15:23:22 +01:00
Marek Marczykowski-Górecki
7905783861
qubesvm: PVH minor improvements
- use capital letters in acronyms in documentation to match upstream
documentation.
- refuse to start a PVH with without kernel set - provide meaningful
error message
2018-01-16 21:42:20 +01:00
Marek Marczykowski-Górecki
4d59f883a0
tests: minor fixes
- FD leak
- switch to xterm to test also on minimal template
2018-01-16 21:41:38 +01:00
Marek Marczykowski-Górecki
06e82eccb0
tests: add run_service and qrexec_policy wrappers to ExtraTestCase
Provide same API as in core2, especially without exposing asyncio
usage. This allows qubes-usb-proxy and qubes-split-gpg tests to run.
2018-01-16 21:39:22 +01:00
Marek Marczykowski-Górecki
c17b634913
tests: clear PCIDevice cache after each test
This is yet another place where references to VM objects contribute to
object leaks.
2018-01-16 21:32:15 +01:00
Rusty Bird
8506547923
block-snapshot: fix ShellCheck warnings (3/3)
These were the remaining warnings that only affected the newer
block-snapshot in this git repo.
2018-01-16 06:20:02 +00:00
Rusty Bird
50f35b3ec1
block-snapshot: fix ShellCheck warnings (2/3)
Forward-ported from qubes-core-agent-linux:

    commit aad6fa6d190d24393e326a4c2ff7ebc3b5921641
    Author: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
    Date:   Sat Sep 30 04:56:02 2017 +0200

        Hint shellcheck where to look for sourced files, if in repository

        This will ease running shellcheck from the repository.
2018-01-16 06:20:01 +00:00
Rusty Bird
45a06e7fad
block-snapshot: fix ShellCheck warnings (1/3)
Forward-ported from qubes-core-agent-linux:

    commit e95b6f8d0357ac9a32fb7e35e5734781460d8f24
    Author: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
    Date:   Sat Sep 30 04:53:38 2017 +0200

        Fix shellcheck warnings in block-snapshot script
2018-01-16 06:20:00 +00:00
Rusty Bird
342c36ed36
block-snapshot: remove trailing spaces
Forward-ported from qubes-core-agent-linux:

    commit 2eb0ed2be14350d6df1fce2af855805133a4a416
    Author: Patrick Schleizer <adrelanos@riseup.net>
    Date:   Thu Oct 15 04:34:55 2015 +0200

        removed trailing spaces
2018-01-16 06:19:58 +00:00
Rusty Bird
9dade5c219
block-snapshot: remove R3.2 template commit code
qvm-template-commit does not exist in R4.0, where FileVolume.stop()
takes care of committing.
2018-01-16 06:19:57 +00:00
Marek Marczykowski-Górecki
d2a7cbb83e
tests: mock vmm.xs
Now it is needed by some unit tests (those calling create_qdb_entries).
2018-01-15 15:58:34 +01:00
Marek Marczykowski-Górecki
ae7031fe7e
tests: explicitly set virt_mode to HVM when needed
HVM is no longer default
2018-01-15 15:57:54 +01:00
Marek Marczykowski-Górecki
241f1d1d3b
tests: do not leak open file 2018-01-15 15:57:30 +01:00
Marek Marczykowski-Górecki
f1a5ca64fd
Merge remote-tracking branch 'qubesos/pr/180'
* qubesos/pr/180:
  vm/qubesvm: default to PVH unless PCI devices are assigned
  vm/qubesvm: expose 'start_time' property over Admin API
  vm/qubesvm: revert backup_timestamp to '%s' format
  doc: link qvm-device man page for qvm-block, qvm-pci, qvm-usb
2018-01-15 04:22:28 +01:00
Marek Marczykowski-Górecki
21760d8ff0
Merge remote-tracking branch 'qubesos/pr/179'
* qubesos/pr/179:
  qmemman: request VMs balloon down with 16MB safety margin
  qmemman: clear "not responding" flags when VM require more memory
  qmemman: slightly improve logging
  qmemman: reformat code, especially comments
2018-01-15 04:21:40 +01:00
Marek Marczykowski-Górecki
b20c3d3458
Merge remote-tracking branch 'qubesos/pr/174'
* qubesos/pr/174:
  tests: fix (system) network tests after switching to ipaddress module
  tests: resurrect extra tests loader
  tests: basic salt integration tests
2018-01-15 04:20:32 +01:00
Marek Marczykowski-Górecki
4ff53879a0
vm/qubesvm: default to PVH unless PCI devices are assigned
Fixes QubesOS/qubes-issues#2185
2018-01-15 03:34:46 +01:00
Marek Marczykowski-Górecki
d9da747ab0
vm/qubesvm: expose 'start_time' property over Admin API
It is useful at least for Qubes Manager.
2018-01-12 05:34:46 +01:00
Marek Marczykowski-Górecki
85e80f2329
vm/qubesvm: revert backup_timestamp to '%s' format
Human readable format `str(datetime.datetime)` is a nightmare for Admin
API level communication. Especially setting the property in a format
that it was read was not supported, and handling such format in
untrusted input handling code is a bad idea. Revert to a simple intiger
format.
2018-01-12 05:34:45 +01:00
Marek Marczykowski-Górecki
be7b278032
doc: link qvm-device man page for qvm-block, qvm-pci, qvm-usb
Those three are special cases of qvm-device tool, so lets use its
documentation too.
2018-01-11 03:44:53 +01:00
Marek Marczykowski-Górecki
4bca631350
qmemman: request VMs balloon down with 16MB safety margin
It looks like Linux balloon driver do not always precisely respect
requested target memory, but perform some rounding. Also, in some cases
(HVM domains), VM do not see all the memory that Xen have assigned to it
- there are some additional Xen pools for internal usage.
Include 16MB safety margin in memory requests to account for those two
things. This will avoid setting "no_response" flag for most of VMs.

QubesOS/qubes-issues#3265
2018-01-11 03:41:55 +01:00
Marek Marczykowski-Górecki
bf4306b815
qmemman: clear "not responding" flags when VM require more memory
Clear slow_memset_react/no_progress flags when VM request more memory
than it have assigned. If there is some available, it may be given to
such VM, solving the original problem (not reacting to balloon down
request). In any case, qmemman algorithm should not try to take away
memory from under-provisioned VM.

Fixes QubesOS/qubes-issues#3265
2018-01-11 03:41:54 +01:00
Marek Marczykowski-Górecki
4cf6a93b5b
qmemman: slightly improve logging
Add logging more info about each domain state:
 - last requested target
 - no_progress and slow_memset_react flags

This makes it unnecessary to log separately when those flags are cleared.
2018-01-11 03:41:54 +01:00
Marek Marczykowski-Górecki
8e288d9f81
qmemman: reformat code, especially comments
Indent comments to match code indentation to make it readable. Also,
wrap long lines. Fix few typos in comments.

No functional change.
2018-01-07 17:04:25 +01:00
Marek Marczykowski-Górecki
a66c9afb18
Merge remote-tracking branch 'qubesos/pr/177'
* qubesos/pr/177:
  Use default_dispvm (not default_template) for new DispVM
2018-01-05 16:29:14 +01:00
Marek Marczykowski-Górecki
d83a07177b
Merge remote-tracking branch 'qubesos/pr/176'
* qubesos/pr/176:
  Do not remove VMs installed via rpm
2018-01-05 16:27:12 +01:00
Rusty Bird
4a2fabc17f
Use default_dispvm (not default_template) for new DispVM
When creating a new VM of type DispVM without specifying any template
(e.g. "qvm-create --class DispVM --label red foo"), use default_dispvm.
Otherwise it would fail saying "Got empty response from qubesd."
2018-01-02 23:19:02 +00:00
Christopher Laprise
47b49c4755
Do not remove VMs installed via rpm 2017-12-29 23:24:41 -05:00
Marek Marczykowski-Górecki
2b364f0cd9
Allow to choose emulated video model via qvm-features
Add feature named 'video-model' to choose custom video model. It needs
to be supported by libvirt: https://libvirt.org/formatdomain.html#elementsVideo

Example usage:

    qvm-features vm-name video-model cirrus

QubesOS/qubes-issues#2488
QubesOS/qubes-issues#3432
2017-12-28 02:50:22 +01:00
Marek Marczykowski-Górecki
3fa0972317
tests: fix (system) network tests after switching to ipaddress module 2017-12-23 16:43:24 +01:00
Marek Marczykowski-Górecki
3668a73ca2
tests: resurrect extra tests loader
Load integration tests from outside of core-admin repository, through
entry points.
Create wrapper for VM object to keep very basic compatibility with tests
written for core2. This means if test use only basic functionality
(vm.start(), vm.run()), the same test will work for both core2 and
core3. This is especially important for app-* repositories, where the
same version serves multiple Qubes branches.
This also hides asyncio usage from tests writer.

See QubesOS/qubes-issues#1800 for details on original feature.
2017-12-23 02:26:16 +01:00
Marek Marczykowski-Górecki
962742880f
tests: basic salt integration tests
Test base functions of dom0 module (creating VM, setting property) and
configuring system inside of VM (through DispVM). The later is done for
each available template (the process use salt installed in that
template, not copied from dom0).

QubesOS/qubes-issues#3316
2017-12-23 02:26:15 +01:00
Marek Marczykowski-Górecki
3065e0de94
version 4.0.15 2017-12-22 14:47:23 +01:00
Marek Marczykowski-Górecki
32c6083e1c
Make pylint happy
Fix thing detected by updated pylint in Travis-CI
2017-12-21 18:19:10 +01:00
Marek Marczykowski-Górecki
f5fe10e2ad
vm/adminvm: fix type of AdminVM.qid
It is defined as int in QubesVM.qid, make it consistent.
2017-12-21 18:18:12 +01:00
Marek Marczykowski-Górecki
9a4d2abf1f
tests: booting VM from ISO image
Check two cases: if ISO image is in dom0 and if its in a VM.

QubesOS/qubes-issues#3339
2017-12-14 23:26:53 +01:00
Marek Marczykowski-Górecki
96bd734852
storage/lvm: clean -snap LV on volume removal 2017-12-14 23:26:52 +01:00