is_dirty() returned a false positive if the volume was merely the source
of a currently running volume. For example, if fedora-33:root was the
source volume for myappvm:root and myappvm was running - then is_dirty()
returned True for fedora-33:root, because fedora-33/root-cow.img
contains some allocated blocks (one 256 KiB chunk containing only the
header) in this scenario, even though fedora-33 is shut down.
FixesQubesOS/qubes-issues#6371
Now Volume.export() may be a coroutine and also may be accompanied by
Volume.export_end() cleaning up after it.
See previous commits for building blocks for this.
This commit adjusts usage of Volume.export() and adds matching
Volume.export_end() throughout the code base.
FixesQubesOS/qubes-issues#5935
This should allow importing a volume and changing the size at the
same time, without performing the resize operation on original
volume first.
The internal API has been renamed to internal.vm.volume.ImportBegin
to avoid confusion, and for symmetry with ImportEnd.
See QubesOS/qubes-issues#5239.
Don't realy on a volume configuration only, it's easy to miss updating
it. Specifically, import_volume() function didn't updated the size based
on the source volume.
The size that the actual VM sees is based on the
file size, and so is the filesystem inside. Outdated size property can
lead to a data loss if the user perform an action based on a incorrect
assumption - like extending size, which actually will shrink the volume.
FixesQubesOS/qubes-issues#4821
Resolve:
- no-else-return
- useless-object-inheritance
- useless-return
- consider-using-set-comprehension
- consider-using-in
- logging-not-lazy
Ignore:
- not-an-iterable - false possitives for asyncio coroutines
Ignore all the above in qubespolicy/__init__.py, as the file will be
moved to separate repository (core-qrexec) - it already has a copy
there, don't desynchronize them.
* devel-storage-fixes:
storage/file: use proper exception instead of assert
storage/file: import data into temporary volume
storage/lvm: check for LVM LV existence and type when creating ThinPool
storage/lvm: fix size reporting just after creating LV
Similar to LVM changes, this fixes/improves multiple things:
- no old data visible in the volume
- failed import do not leave broken volume
- parially imported data not visible to running VM
QubesOS/qubes-issues#3169
It may happen that one pool is inside a volume of other pool. This is
the case for example for varlibqubes pool (file driver,
dir_path=/var/lib/qubes) and default lvm pool (lvm_thin driver). The
latter include whole root filesystem, so /var/lib/qubes too.
This is relevant for proper disk space calculation - to not count some
space twice.
QubesOS/qubes-issues#3240QubesOS/qubes-issues#3241
Do not check for accepted value only in constructor, do that in property
setter. This will allow enforcing the limit regardless of how the value
was set.
This is preparation for dynamic revisions_to_keep change.
QubesOS/qubes-issues#3256
* tests-storage:
tests: register libvirt events
tests: even more agressive cleanup in tearDown
app: do not wrap libvirt_conn.close() in auto-reconnect wrapper
api: keep track of established connections
tests: drop VM cleanup from tearDownClass, fix asyncio usage in tearDown
storage: fix Storage.clone and Storage.clone_volume
tests: more tests fixes
firewall: raise ValueError on invalid hostname in dsthost=
qmemman: don't load qubes.xml
tests: fix AdminVM test
tests: create temporary files in /tmp
tests: remove renaming test - it isn't supported anymore
tests: various fixes for storage tests
tests: fix removing LVM volumes
tests: fix asyncio usage in some tests
tests: minor fixes to api/admin tests
storage/file: create -cow.img only when needed
storage: move volume_config['source'] filling to one place
app: do not create 'default' storage pool
app: add missing setters for default_pool* global properties
This driver isn't used in default Qubes 4.0 installation, but if we do
have it, let it follow defined API and its own documentation. And also
explicitly reject not supported operations:
- support only revisions_to_keep<=1, but do not support revert() anyway
(implemented version were wrong on so many levels...)
- use 'save_on_stop'/'snap_on_start' properties directly instead of
obsolete volume types
- don't call sudo - qubesd is running as root
- consistently use path, path_cow, path_source, path_source_cow
Also, add tests for BlockDevice instance returned by
FileVolume.block_device().
QubesOS/qubes-issues#2256
Add convenient collection wrapper for easier getting selected volume.
Storage pool implementation may still provide only volume listing
function (pool.list_volumes), or, additionally, optimized
pool.get_volume.
This means it is both possible to iterate over volumes:
```python
for volume in pool.volumes:
...
```
And get a single volume:
```python
volume = pool.volumes[vid]
```
QubesOS/qubes-issues#2256
Reference objects, not their IDs - this way when object is modified, it
is visible everywhere where it is used. Main changes:
- volume.pool - Pool object
- volume.source - Volume object
Since volume have Pool object reference now, move volume related
functions into Volume class (from Pool class). This avoids horrible
`storage.get_pool(volume).something(volume)` construct.
One issue here is since volume.source reference a Volume object from a
different VM - VM's template, now VM load order is important. Since we
don't have control over it, initialize vm.storage when needed - possibly
while initializing storage of different VM. Since we don't have cycles
in AppVM-TemplateVM dependencies, it is safe.
Also, since this commit, volume.source (if defined) always points at
volume of the same name from VM's template. Using volumes with something
else as a source is no longer supported.
QubesOS/qubes-issues#2256
When VM is renamed only volume.vid get updated, but not other attributes
calculated from it. Convert them to dynamic properties to not worry
about it.
QubesOS/qubes-issues#2256
Allow importing not only from another volume, but also raw data. In
practice, for all currently implemented storage pools, this is the same
as Pool.export, because path returned there is read-write. But lets not
abuse this fact, some future implementation may need different methods.
QubesOS/qubes-issues#2622QubesOS/qubes-issues#2256
Use the right cow image and apply the second layer to provide read-write
access. The correct setup is:
- base image + base cow -> read-only snapshot (base changes "cached"
until committed)
- read-only snapshot + VM cow -> read-write snapshot (changes discarded
after VM shutdown)
This way, even VM without Qubes-specific startup scripts will can
benefit from Template VMs, while VMs with Qubes-specific startup scripts
may still see original root.img content (for possible signature
verification, when storage domain got implemented).
QubesOS/qubes-issues#2256
* core3-devices:
Fix core2migration and tests for new devices API
tests: more qubes.devices tests
qubes/ext/pci: implement pci-no-strict-reset/BDF feature
qubes/tools: allow calling qvm-device as qvm-devclass (like qvm-pci)
qubes: make pylint happy
qubes/tools: add qvm-device tool (and tests)
tests: load qubes.tests.tools.qvm_ls
tests: PCI devices tests
tests: add context manager to catch stdout
qubes/ext/pci: move PCI devices handling to an extension
qubes/devices: use more detailed exceptions than just KeyError
qubes/devices: allow non-persistent attach
qubes/storage: misc fixes for VM-exposed block devices handling
qubes: new devices API
FixesQubesOS/qubes-issues#2257
