Commit Graph

5767 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
7cb46e6929
Merge remote-tracking branch 'origin/pr/298'
* origin/pr/298:
  tests/network: let xl devd bring the interfaces up
  tests/network: improve error reporting
  api/admin: implement *.property.GetAll methods
  Avoid resetting clocksync service of just enabled clockvm
  doc/tests: extend qubes-specific quirks in tests
  tests: add include and exclude lists for extra tests loader
2019-12-07 05:04:42 +01:00
Marek Marczykowski-Górecki
8965e9a8e4
tests/network: let xl devd bring the interfaces up
xl devd doesn't use startup notify, so when the service is started
(according to systemd) it may still be initializing interfaces. Add a
little sleep for that.
2019-12-05 23:31:15 +01:00
Marek Marczykowski-Górecki
ffa1a40e6e
tests/network: improve error reporting
Include stdout/stderr of failed command during netvm setup.
2019-12-05 23:31:15 +01:00
Marek Marczykowski-Górecki
10f99e5c4a
api/admin: implement *.property.GetAll methods
Allow getting all the VM properties with one call. This greatly improve
performance of an applications retrieving many/all of them (qvm-ls,
qubes manager etc)

QubesOS/qubes-issues#5415
Fixes QubesOS/qubes-issues#3293
2019-12-05 23:31:14 +01:00
Rusty Bird
87081d6ee3
storage/reflink: _cleanup() -> _remove_incomplete_files()
"cleanup" sounds related to the concept of a volume being "dirty" - but
it's unrelated. Rename it for clarity.
2019-12-03 18:21:55 +00:00
Rusty Bird
d7478d128b
storage/reflink: document hardcoded sizeof(int) for FICLONE
One alternative would look like

    import ctypes
    sizeof_int = ctypes.sizeof(ctypes.c_int)
    FICLONE = (1073741824 % 256**sizeof_int) | 37897 | (sizeof_int << 16)

but, even if the above really(?) is a 100% correct Python port of

    $ echo FICLONE | cpp -include linux/fs.h | tail -n 1

it still seems more likely that the ctypes package is somehow buggy
somewhere than for Qubes storage to run on an exotic architecture with
non 32 bit ints (in the foreseeable future).

So just document the baked in assumption.
2019-12-03 18:21:54 +00:00
Rusty Bird
3f0286220c
storage/reflink: simplify _replace_file() comment 2019-12-03 18:21:52 +00:00
Rusty Bird
9d5deffb13
storage/reflink: open in binary mode for loopdev resize ioctl
The default (= text) mode for a loop device which contains a VM image
looked weird, even though it didn't make a difference here because the
dev_io object was never actually read from.
2019-12-03 18:21:51 +00:00
Rusty Bird
4cd9e42416
storage/reflink: use a conditional expression 2019-12-03 18:21:50 +00:00
Marek Marczykowski-Górecki
5d77cf2298
Avoid resetting clocksync service of just enabled clockvm
When setting global clockvm property, 'clocksync' service is
automatically added to the new value and then removed from the old one.
But if the new value is the same as the old one, the service gets
removed from the just set new value.
Check for this case explicitly.

Fixes QubesOS/qubes-issues#4939
2019-11-30 05:20:08 +01:00
Marek Marczykowski-Górecki
13f35da24a
doc/tests: extend qubes-specific quirks in tests
- per-VM (template) tests
- extra tests loader
2019-11-30 04:38:10 +01:00
Marek Marczykowski-Górecki
7c18b187de
tests: add include and exclude lists for extra tests loader
'extra' tests run is getting ridiculously long. Allow splitting it into
several jobs. Since this appears as just one class from the test loader
perspective, implement it as environment variables:
 - QUBES_TEST_EXTRA_INCLUDE - load just selected tests
 - QUBES_TEST_EXTRA_EXCLUDE - skip selected tests (to select "the rest"
   tests)
2019-11-30 04:35:18 +01:00
Marek Marczykowski-Górecki
fd8e89c546
Merge remote-tracking branch 'origin/pr/297'
* origin/pr/297:
  doc: remove useless _static generated by sphinx
  cleanup-dispvms: fix python shebang
  spec: fix missing dependency
  Fix Sphinx 2 new API for Fedora 31+
  doc: Make PEP8 happier
  qmemmand: separate SystemState init xc and xs to a 'init' method
  doc: drop moved elsewhere components
2019-11-27 02:12:37 +01:00
Marek Marczykowski-Górecki
09ab00cafd
Merge remote-tracking branch 'origin/pr/296'
* origin/pr/296:
  ext/block: prefer connecting cdrom as xvdd
  tests: extend mock objects in QubesVM tests
2019-11-26 23:56:33 +01:00
Marek Marczykowski-Górecki
4f6b9694fe
travis: fix dom0 distribution 2019-11-22 22:13:05 +01:00
Frédéric Pierret (fepitre)
bc723b8755
doc: remove useless _static generated by sphinx 2019-11-22 21:39:38 +01:00
Frédéric Pierret (fepitre)
6624ed3383
cleanup-dispvms: fix python shebang 2019-11-22 21:39:35 +01:00
Frédéric Pierret (fepitre)
3761982d90
spec: fix missing dependency 2019-11-22 21:39:33 +01:00
Frédéric Pierret (fepitre)
be1c193e3c
Fix Sphinx 2 new API for Fedora 31+ 2019-11-22 21:39:31 +01:00
Frédéric Pierret (fepitre)
b4177cf7d2
doc: Make PEP8 happier 2019-11-22 21:39:30 +01:00
Frédéric Pierret (fepitre)
0d20639cc1
qmemmand: separate SystemState init xc and xs to a 'init' method
It prevents any connection to xen when initiallizing the class
notably in doc build generation.
2019-11-22 21:38:27 +01:00
Frédéric Pierret (fepitre)
3ba9c6f8d1 doc: drop moved elsewhere components 2019-11-21 18:16:33 +01:00
Marek Marczykowski-Górecki
6c7af109e5
ext/block: prefer connecting cdrom as xvdd
Only first 4 disks can be emulated as IDE disks by QEMU. Specifically,
CDROM must be one of those first 4 disks, otherwise it will be
ignored. This is especially important if one wants to boot the VM from
that CDROM.
Since xvdd normally is a kernel-related volume (boot image, modules) it
makes perfect sense to re-use it for CDROM. It is either set for kernel
volume (in which case, VM should boot from it and not the CDROM), or
(possibly bootable) CDROM.

This needs to be done in two places:
 - BlockExtension for dynamic attach
 - libvirt xen.xml - for before-boot attach

In theory the latter would be enough, but it would be quite confusing
that device will get different options depending on when it's attached
(in addition to whether the kernel is set - introduced here).

This all also means, xvdd not always is a "system disk". Adjust listing
connected disks accordingly.
2019-11-19 14:03:21 +01:00
Marek Marczykowski-Górecki
9bf0cce11e
tests: extend mock objects in QubesVM tests
- allow TestQubesDB to be populated with initial data
- support list() method
- allow to register pre-created VM instance (useful for AdminVM, which
        don't accept setting qid)
2019-11-18 23:45:02 +01:00
Frédéric Pierret (fepitre)
e092b40350
tests: make PEP8 happier 2019-11-17 01:54:27 +01:00
Frédéric Pierret (fepitre)
3a6ed2d0cd
tests: fix test with not None default_guivm 2019-11-17 01:48:20 +01:00
Frédéric Pierret (fepitre)
85edf511cb
default_guivm: set to dom0 due to migration R4.0->R4.1 problems 2019-11-16 13:04:42 +01:00
Marta Marczykowska-Górecka
783832adde
Correct inconsistent behavior on unavailable usage data
fixes QubesOS/qubes-issues#5463
2019-11-15 20:01:43 +01:00
Marek Marczykowski-Górecki
fcb74d70f8
version 4.1.5 2019-11-13 05:42:45 +01:00
Marek Marczykowski-Górecki
ba105e89c6
Merge branch 'devel20191029' 2019-11-12 23:11:30 +01:00
Marek Marczykowski-Górecki
7de9f3e078
tests: is_iommu_supported function
QubesOS/qubes-issues#4689
2019-11-12 22:45:30 +01:00
Marek Marczykowski-Górecki
c79e91d59d
Merge remote-tracking branch 'origin/pr/292'
* origin/pr/292:
  Throw BackupAlreadyRunningError when backup is already running
2019-11-12 22:26:53 +01:00
Marta Marczykowska-Górecka
7a4455bbfa
Throw BackupAlreadyRunningError when backup is already running
Instead of generic PermissionDenied.

requires https://github.com/QubesOS/qubes-core-admin-client/pull/115
references QubesOS/qubes-issues#5432
2019-11-11 19:16:03 +01:00
Frédéric Pierret (fepitre)
2ccdd4ee8e
gui: make pylint happy 2019-11-11 11:27:40 +01:00
Marek Marczykowski-Górecki
77cf310c47
storage/kernels: fix listing volumes
Pool.volumes property is implemented in a base class, individual drivers
should provide list_volumes() method as a backend for that property.
Fix this in a LinuxKernel pool.
2019-11-10 01:14:34 +01:00
Marek Marczykowski-Górecki
263f218d40
api/admin: implement admin.pool.volume.List method
Similar to admin.vm.volume.List. There are still other
admin.pool.volume.* methods missing, but lets start with just this one.
Those with both pool name and volume id arguments may need some more
thoughts.
2019-11-09 18:38:56 +01:00
Frédéric Pierret (fepitre)
5f934b43ab
tests: add app guivm tests 2019-11-09 17:06:46 +01:00
Frédéric Pierret (fepitre)
728766d191
default_guivm: fire property-set on default_guivm 2019-11-09 16:10:16 +01:00
Frédéric Pierret (fepitre)
78d0d2cabb
gui: set guivm windows prefix 2019-11-07 18:11:32 +01:00
Marek Marczykowski-Górecki
dd037f4663
storage/file: get volume size from the actual image file size
Don't realy on a volume configuration only, it's easy to miss updating
it. Specifically, import_volume() function didn't updated the size based
on the source volume.
The size that the actual VM sees is based on the
file size, and so is the filesystem inside. Outdated size property can
lead to a data loss if the user perform an action based on a incorrect
assumption - like extending size, which actually will shrink the volume.

Fixes QubesOS/qubes-issues#4821
2019-10-31 21:53:35 +01:00
Marek Marczykowski-Górecki
598d059c57
tests: check if storage driver adjust the size on import_volume/clone
Regression test for QubesOS/qubes-issues#4821
2019-10-31 01:17:26 +01:00
Marek Marczykowski-Górecki
5908ab1568
app: fix get_free_xen_memory function
It's app.vmm.xc, not app.xc. Since nobody have noticed it, this function
might be unused...
2019-10-30 15:46:11 +01:00
Marek Marczykowski-Górecki
361550c621
vm: improve error message about missing IOMMU
Handle this case specifically, as way too many users ignore the message
during installation and complain it doesn't work later.

Name the problem explicitly, instead of pointing at libvirt error log.

Fixes QubesOS/qubes-issues#4689
2019-10-30 15:45:52 +01:00
Marek Marczykowski-Górecki
89e3ae3945
version 4.1.4 2019-10-28 04:18:43 +01:00
Marek Marczykowski-Górecki
8603571cbc
tests: skip networking tests for minimal templates
Minimal templates don't have networking packages installed by default.
2019-10-28 03:30:49 +01:00
Marek Marczykowski-Górecki
4dac995089
Merge branch 'start-logging'
* start-logging:
  vm: log startup errors in every case
2019-10-23 13:50:25 +02:00
Marek Marczykowski-Górecki
de3c7edd4f
travis: switch to python 3.6
For system site-packages to work, it needs to be the same as in the base
distribution. Switch to the version in Ubuntu bionic

QubesOS/qubes-issues#4613
2019-10-23 04:43:33 +02:00
Marek Marczykowski-Górecki
5c3bb1cbd0
Merge remote-tracking branch 'origin/pr/288'
* origin/pr/288:
  travis: switch to bionic
2019-10-23 04:40:07 +02:00
Marek Marczykowski-Górecki
cc56c6f96a
Merge remote-tracking branch 'origin/pr/278'
* origin/pr/278:
  Added admin.pool.UsageDetails API method
  Add metadata info to LVM AdminAPI
2019-10-23 04:36:49 +02:00
Marta Marczykowska-Górecka
2f6497e48d
Added admin.pool.UsageDetails API method
admin.pool.UsageDetails reports the usage data, unlike
admin.pool.Info, which should report the config/unchangeable data.
At the moment admin.Pool.Info still reports usage, to maintain
compatibility, but once all relevant tools are updated,
it should just return configuration data.
2019-10-23 03:04:30 +02:00