Commit Graph

5726 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
c5aaf3abd9
tests: make libvirt mockup more robust
If not in offline_mode, return actual mock for libvirt connection object
instead of always raising exception.
2019-09-10 03:34:11 +02:00
Marek Marczykowski-Górecki
05e48748d2
tests: update for not needing custom kernel modules anymore
kernel-devel package isn't needed in VMs anymore.
2019-09-10 03:33:42 +02:00
Frédéric Pierret (fepitre)
7d1b6e24a9
Add qubes.ConnectTCP RPC for allowing dom0 sockets binding 2019-08-29 11:40:20 +02:00
Marek Marczykowski-Górecki
7a32b06171
version 4.1.2 2019-08-11 06:50:33 +02:00
Marek Marczykowski-Górecki
5d5f102378
Merge remote-tracking branch 'origin/pr/277'
* origin/pr/277:
  admin: add admin.deviceclass.List
  admin: replace single quote to double for docstring
2019-08-08 14:05:00 +02:00
Marek Marczykowski-Górecki
32b0d7df43
Merge remote-tracking branch 'origin/pr/275'
* origin/pr/275:
  api/stats: improve cpu_usage normalization, add cpu_usage_raw
  Use 128k blocks when importing volume data
2019-08-08 14:04:27 +02:00
Frédéric Pierret (fepitre)
e32ce14ab5
qubes-rpc: add qubes.ConnectTCP 2019-08-08 12:28:30 +02:00
ejose19
2602df5e19 Fix invalid timezone
This fixes an invalid response generated by get_timezone when the time zones are composed by 3 parts, for example:

America/Argentina/Buenos_Aires
America/Indiana/Indianapolis

Update utils.py
2019-08-06 18:23:00 -03:00
Frédéric Pierret (fepitre)
7ff01b631d
admin: add admin.deviceclass.List
QubesOS/qubes-issues#5213
2019-08-06 11:01:02 +02:00
Frédéric Pierret (fepitre)
7f670614c8
admin: replace single quote to double for docstring 2019-08-06 10:04:30 +02:00
Marek Marczykowski-Górecki
39d64eabc8
api/stats: improve cpu_usage normalization, add cpu_usage_raw
Give raw cpu_time value, instead of normalized one (to number of vcpus),
as documented.
Move the normalization to cpu_usage calculation. At the same time, add
cpu_usage_raw without it, if anyone needs it.

QubesOS/qubes-issues#4531
2019-08-01 04:51:05 +02:00
Marek Marczykowski-Górecki
a98f505c8f
Use 128k blocks when importing volume data
Cloning volume was handled earlier, now od the same on importing volumes
(any storage).

Fixes QubesOS/qubes-issues#5134
2019-08-01 04:50:25 +02:00
Brendan Hoar
a5378b5958
Update lvm.py 2019-07-31 19:40:33 -04:00
Marek Marczykowski-Górecki
205f6215b2
Merge remote-tracking branch 'origin/pr/271'
* origin/pr/271:
  Fix too long line
  Prevent 'qubesd' for crashing if any device backend is not available
2019-07-31 17:23:20 +02:00
Marek Marczykowski-Górecki
9e226ab5cf
Merge remote-tracking branch 'origin/pr/273'
* origin/pr/273:
  tests: check importing empty data into ReflinkVolume
  tests: check importing empty data into ThinVolume
  tests: check importing empty data into FileVolume
  tests: improve cleanup after LVM tests
2019-07-31 15:37:13 +02:00
Marek Marczykowski-Górecki
6c8bdfaa4b
Fix too long line 2019-07-31 15:36:03 +02:00
Marek Marczykowski-Górecki
11038bcdec
Merge remote-tracking branch 'origin/pr/270'
* origin/pr/270:
  spec: add socat dependency for qvm-console* tools
2019-07-31 15:33:25 +02:00
Marek Marczykowski-Górecki
afb0de43d4
tests: check importing empty data into ReflinkVolume
Verify if it really discards old content.

QubesOS/qubes-issues#5192
2019-07-28 22:08:52 +02:00
Marek Marczykowski-Górecki
19186f7840
tests: check importing empty data into ThinVolume
Verify if it really discards old content.

QubesOS/qubes-issues#5192
2019-07-28 22:08:37 +02:00
Marek Marczykowski-Górecki
790c2ad8cb
tests: check importing empty data into FileVolume
Verify if it really discards old content.

QubesOS/qubes-issues#5192
2019-07-28 22:06:30 +02:00
Marek Marczykowski-Górecki
8414d0153f
tests: improve cleanup after LVM tests
Remove test volumes - this way if a test fails, subsequent tests have a
chance to succeed.
2019-07-28 21:41:04 +02:00
Frédéric Pierret (fepitre)
74f8d4531e
Prevent 'qubesd' for crashing if any device backend is not available 2019-07-20 15:27:09 +02:00
Frédéric Pierret (fepitre)
16d0c8ce90
spec: add socat dependency for qvm-console* tools 2019-07-11 09:39:32 +02:00
Rusty Bird
aa931fdbf7
vm/qubesvm: call storage.remove() *before* removing dir_path
Give the varlibqubes storage pool, which places volume images inside the
VM directory, a chance to remove them (and e.g. to log that).
2019-06-28 10:29:34 +00:00
Rusty Bird
cfa1c7171e
storage: fix typo 2019-06-28 10:29:32 +00:00
Rusty Bird
6e592a56d7
storage/reflink: update comment and whitespace 2019-06-28 10:29:31 +00:00
Rusty Bird
1fe2aff643
storage/reflink: _fsync_path(path_from) in _commit()
During regular VM shutdown, the VM should sync() anyway. (And
admin.vm.volume.Import does fdatasync(), which is also fine.) But let's
be extra careful.
2019-06-28 10:29:30 +00:00
Rusty Bird
df23720e4e
storage/reflink: _fsync_dir() -> _fsync_path()
Make it work when passed a file, too.
2019-06-28 10:29:29 +00:00
Rusty Bird
35b4b915e7
storage/reflink: coroutinize pool setup() 2019-06-28 10:29:27 +00:00
Rusty Bird
1d89acf698
app: setup_pools() must be a coroutine
This is needed as a consequence of d8b6d3ef ("Make add_pool/remove_pool
coroutines, allow Pool.{setup,destroy} as coroutines"), but there hasn't
been any problem so far because no storage driver implemented pool
setup() as a coroutine.
2019-06-28 10:29:26 +00:00
Rusty Bird
5b52d23478
factor out utils.void_coros_maybe() 2019-06-28 10:29:25 +00:00
Rusty Bird
fe97a15d11
factor out utils.coro_maybe() 2019-06-28 10:29:24 +00:00
Marek Marczykowski-Górecki
d94e13f753
version 4.1.1 2019-06-25 06:46:06 +02:00
Marek Marczykowski-Górecki
883d324b6c
Revert "tests: do not use lazy unmount"
Revert to use umount -l in storage tests cleanup. With fixed permissions
in 4234fe51 "tests: fix cleanup after reflink tests", it shouldn't cause
issues anymore, but apparently on some systems test cleanup fails
otherwise.

Reported by @rustybird
This reverts commit b6f77ebfa1.
2019-06-25 05:51:33 +02:00
Rusty Bird
66e44e67de
api/admin: always save after self.dest.storage.resize() 2019-06-23 12:48:01 +00:00
Rusty Bird
2b4b45ead8
storage/reflink: preferably get volume size from image size
There were (at least) five ways for the volume's nominal size and the
volume image file's actual size to desynchronize:

- loading a stale qubes.xml if a crash happened right after resizing the
  image but before saving the updated qubes.xml (-> previously fixed)
- restarting a snap_on_start volume after resizing the volume or its
  source volume (-> previously fixed)
- reverting to a differently sized revision
- importing a volume
- user tinkering with image files

Rather than trying to fix these one by one and hoping that there aren't
any others, override the volume size getter itself to always update from
the image file size. (If the getter is called though the storage API, it
takes the volume lock to avoid clobbering the nominal size when resize()
is running concurrently.)
2019-06-23 12:48:00 +00:00
Rusty Bird
de159a1e1e
storage/reflink: don't run verify() under volume lock 2019-06-23 12:47:59 +00:00
Rusty Bird
4c9c0a88d5
storage/reflink: split @_unblock into @_coroutinized @_locked
And change the volume lock from an asyncio.Lock to a threading.Lock -
locking is now handled before coroutinization.

This will allow the coroutinized resize() and a new *not* coroutinized
size() getter from one of the next commits ("storage/reflink: preferably
get volume size from image size") to both run under the volume lock.
2019-06-23 12:47:58 +00:00
Rusty Bird
ab929baa38
Revert "storage/reflink: update volume size in __init__()"
Fixed more generally in one of the next commits ("storage/reflink:
preferably get volume size from image size").

This reverts commit 9f5d05bfde.
2019-06-23 12:47:57 +00:00
Rusty Bird
58a7e0f158
Revert "storage/reflink: update snap_on_start volume size in start()"
Fixed more generally in one of the next commits ("storage/reflink:
preferably get volume size from image size").

This reverts commit c43df968d5.
2019-06-23 12:47:56 +00:00
Marek Marczykowski-Górecki
fa0ae0c921
Merge remote-tracking branch 'origin/pr/257'
* origin/pr/257:
  tests: AdminVM.run_service*()
  tests: QubesVM.run_service function
  vm/adminvm: add run_service* functions
2019-06-23 03:43:35 +02:00
Marek Marczykowski-Górecki
d5bdb5d5e0
Merge remote-tracking branch 'origin/pr/267'
* origin/pr/267:
  lvm: run blkdiscard before remove
2019-06-23 03:26:30 +02:00
Christopher Laprise
62e3d57120
lvm: run blkdiscard before remove
issue #5077
2019-06-21 18:40:03 -04:00
Marek Marczykowski-Górecki
ad8c2d4b09
Merge remote-tracking branch 'origin/pr/266'
* origin/pr/266:
  storage/reflink: volume.resize(): succeed without image
  storage/reflink: volume.resize(): remove safety check
  storage/reflink: simplify volume.usage()
  storage/reflink: update volume size in __init__()
  storage/reflink: update snap_on_start volume size in start()
2019-06-21 20:55:09 +02:00
Marek Marczykowski-Górecki
c468b29490
tests: AdminVM.run_service*() 2019-06-21 20:45:38 +02:00
Marek Marczykowski-Górecki
e352fc25d7
tests: QubesVM.run_service function 2019-06-21 20:45:38 +02:00
Marek Marczykowski-Górecki
2afab43df9
libvirt xml: fix DNS address broadcasted by DHCP server
In R4.0 it is no longer the same as the gateway.

QubesOS/qubes-issues#3585
2019-06-21 20:40:04 +02:00
Marek Marczykowski-Górecki
362e128ec6
libvirt xml: fix timezone syntax
Libvirt XML syntax require lowercase utc as 'basis' attribute.
This option is used only if 'timezone' property is set. And apparently
libvirt prior to 4.1.0 didn't support it for Xen at all. Which means the
setting is ignore prior to Qubes R4.1.
2019-06-21 20:36:24 +02:00
Rusty Bird
ef128156a3
storage/reflink: volume.resize(): succeed without image
Successfully resize volumes without any currently existing image file,
e.g. cleanly stopped volatile volumes: Just update the nominal size in
this case.
2019-06-15 16:03:46 +00:00
Rusty Bird
25e92bd19b
storage/reflink: volume.resize(): remove safety check
It is handled by 'qvm-volume resize', which has a '--force' option that
can't work if the check is duplicated in the storage driver.
2019-06-15 16:03:45 +00:00