Break the ioctl calls from blkdiscard into max 1GB worth of blocks per call. This ensures that the amount of pending discards (both for the virtual lvm thin volume as well as any lower level device recipients) do not dominate the IO streams to the devices. Before this change, IO blocking was leading to both UI and processing freezes in Qubes while very large LVM volumes were being removed. The change will allow other device IOs to be interleaved between the internal ioctl calls from blkdiscard.
Allow user login while some VMs are still starting. This improves UX of
the system startup, since user can start using the system earlier.
FixesQubesOS/qubes-issues#3149
Qubesd limits max payload to 64kb. Do the same in qubesd-query, to avoid
loading to memory potentially unbounded amount of data that would be
refused later anyway.
Reported by @DemiMarie
Theoretically libvirt/libxl should reject too long values anyway, but
lets provide early feedback and reject value that would cause VM start
fail later.
Reported by @DemiMarie
Validate the syntax of net.fake-* features values. While improper values
currently would lead "only" to lack of network connectivity for a VM,
the scripts handling those may assume they are properly formatted
already. Furthermore, this gives better error reporting to the user.
This avoids XML-injection by a malformed property value. If a property
value is controlled by a less privileged entity (like Management VM), it
could lead to a privilege escalation.
Reported by @DemiMarie
umask() modifies the whole process, which could lead to odd permission
problems in concurrent code. Use explicit fchmod() calls instead.
Preserve the legacy storage/file behavior of creating files 0o664 in
FileVolume.create() and 0o644 (from the normal qubesd umask) otherwise.
Get rid of an otherwise unused module-level variable (HOST_MACHINE), and
show the relevant file (reflink.py) in the error message:
Traceback (most recent call last):
File "/usr/bin/qubesd", line 2, in <module>
from qubes.tools.qubesd import main
File "/usr/lib/python3.8/site-packages/qubes/__init__.py", line 801, in <module>
from qubes.app import Qubes
File "/usr/lib/python3.8/site-packages/qubes/app.py", line 64, in <module>
import qubes.storage.reflink
File "/usr/lib/python3.8/site-packages/qubes/storage/reflink.py", line 46, in <module>
FICLONE = {
KeyError: 'armv7l'
qubesd.service: Main process exited, code=exited, status=1/FAILURE
* origin/pr/389:
app: save qubes.xml with utils.replace_file()
app: use suppress() in simple cases
firewall: save firewall.xml with utils.replace_file()
utils: take tweaked helper functions from storage/reflink
storage/reflink: quote logged filenames
'with (yield from alock):' is incompatible with Python 3.9+.
Change it to 'async with alock:', and then change the affected functions
to 'async def'.
This makes the test suite pass again in a Fedora 33 VM.
QubesOS/qubes-issues#2738
is_dirty() returned a false positive if the volume was merely the source
of a currently running volume. For example, if fedora-33:root was the
source volume for myappvm:root and myappvm was running - then is_dirty()
returned True for fedora-33:root, because fedora-33/root-cow.img
contains some allocated blocks (one 256 KiB chunk containing only the
header) in this scenario, even though fedora-33 is shut down.
FixesQubesOS/qubes-issues#6371
replace_file(), rename_file(), and remove_file() now have optional
'logger' and 'log_level' (defaulting to DEBUG) arguments.
replace_file() now has a required 'permissions' and an optional
'close_on_success' (defaulting to True) argument. Also, it doesn't
create any directories; and in case of an exception, the tempfile is
removed even when closing it raises another exception.
remove_file() now returns a value: True if the file was removed, or
False if it already didn't exist.
(fsync_path() is unchanged.)
!!! After cherry-picking for release4.0, consider a fixup !!!
!!! adding 'import qubes.utils' to storage/reflink there !!!
Add support for blocking shutdown call. This adds a symmetry to
admin.vm.Start call which is blocking.
Since the admin.vm.Shutdown has established semantic already, add a
'wait' parameter. It can be combined with 'force' as 'force+wait' (or
the other way around).
Fedora compresses the kernel with zstd now, which isn't suppported in
grub (as of 2.04). Since we don't care about PV that much anymore,
simply skip the test there.
