None of found existing portable locking module does support RW locks.
Use lowlevel system locking support - both Windows and Linux support
such feature.
Drop locking code in write_firewall_conf() b/c is is called with
QubesVmCollection lock held anyway.
Some VM types do not have particular disk image. Instead of enumerating
cases in storage class, signal unused image from VM class by setting
appropriate attr to None.
There are still few uses of direct xenstore access, most of them are
xen-specific (so doesn't need to be portable). For now simply don't
connect to xenstore when no 'xen.lowlevel.xs' module present. It will
break such xen-specific accesses - it must be somehow reworked - either
by adding appropriate conditionals, or moving such code somewhere else
(custom methods of libvirt driver?).
There is still use of it: QubesHost.get_free_xen_memory and
QubesHost.measure_cpu_usage. Will migrate them to libvirt later (for now
some things will be broken - namely qubes-manager).
Mostly done. Things still using xenstore/not working at all:
- DispVM
- qubesutils.py (especially qvm-block and qvm-usb code)
- external IP change notification for ProxyVM (should be done via RPC
service)
This makes easier to import right objects in submodules (only one
object). This also implement lazy connection - at first access, not at
module import, which speeds up tools, which doesn't need runtime
information (like qvm-prefs or qvm-service). In the future this will
ease migration from xenstore to QubesDB.
Also implement "offline mode" - operate on qubes.xml without connecting
to VMM - raise exception at such try.
This is needed to run tools during installation, where only minimal
set of services are started, especially no libvirt.
QubesVmCollection.save() overrides qubes.xml by creating new file, then
renaming it over the old one. If any process has that (old) file open
at the same time - especially while waiting on lock_db_for_writing() -
it will end up in accessing old, already unlinked file.
The exact calls would look like:
P1 P2
lock_db_for_writing
fd = open('qubes.xml')
fcntl(fd, F_SETLK, ...)
lock_db_for_writing
fd = open('qubes.xml')
fcntl(fd, F_SETLK, ...)
...
save():
open(temp-file)
write(temp-file, ...)
...
flush(temp-file)
rename(temp-file, 'qubes.xml')
close(fd) // close old file
lock_db_for_writing succeed
*** fd points at already unlinked
file
unlock_db
close(qubes.xml)
To fix that problem, added a check if (already locked) file is still the
same as qubes.xml.
Since tar multi-archive no longer used, we can simply instruct tar to
pipe output through gzip (or whatever compressor we want). Include used
compressor command in backup header.
Tar multi-volume support is broken when used with sparse files[1], so do
not use it. Instead simply cut the archive manually and concatenate at
restore time. This change require a little modification in restore
process, so make this new backup format ("3"). Also add backup format
version to the header, instead of some guessing code.
For now only cleartext and encrypted backups implemented, compression
will come as a separate commit.
loop device parsing should have "dXpY_style = True" in order to
correctly parse partitions on loop devices.
Reasoning:
==========
Using losetup to create a virtual SD card disk into a loop device and
creating partitions for it results in new devices within an AppVM that
look like: /dev/loop0p1 /dev/loop0p2 and so on.
However as soon as they are created, Qubes Manager rises an exception
and becomes blocked with the following message (redacted):
"QubesException: Invalid device name: loop0p1
at line 639 of file /usr/lib64/python2.7/site-
packages/qubesmanager/main.py
Details:
line: raise QubesException....
func: block_name_to_majorminor
line no.: 181
file: ....../qubes/qubesutils.py
Backups should be safe also for long-term, so change HMAC to SHA512,
which should be usable much longer than SHA1.
See this thread for discussion:
https://groups.google.com/d/msg/qubes-devel/5X-WjdP9VqQ/4zI8-QWd0S4J
Additionally save guessed HMAC in artificial header data (when no real
header exists).
Previously this message goes to /dev/null (unless BACKUP_DEBUG enabled),
so the user got cryptic "Restore failed" message without any clue about
the cause.
