Commit Graph

377 Commits

Author SHA1 Message Date
3hhh
529e4bfbbf
storage/callback: volume callbacks now also rceive the source volume
as argument (if there's any)

This is useful for disposable VMs to identify from which template they
originate.
2020-07-16 14:31:03 +02:00
3hhh
57e7a02912
storage/callback: add a post_volume_start callback 2020-07-16 14:31:03 +02:00
3hhh
bf8ece8a0c
storage/callback: more succinct callback names
[pre|post]_[operation] should be more clear than
on_[operation]
2020-07-16 14:31:03 +02:00
3hhh
9de54ab242
storage/callback: make CallbackVolume a Volume
Unfortunately this appears to be necessary due to
various Qubes OS `assert` checks and to get `__str__()` et al
from the super class. It also means that we have to implement
all methods of the super class (in the future as well).
2020-07-16 14:31:03 +02:00
3hhh
178d4dd997
storage/callback: enforce CallbackPool as the pool attribute of delegated volumes
This fixes a bug preventing the use of the callback pool driver
with disposable VMs.
2020-07-16 14:31:03 +02:00
3hhh
caddc1c499
storage/callback: pylint: disable line-too-long 2020-07-16 14:31:03 +02:00
3hhh
889c9238fe
storage/callback: asyncio implementation 2020-07-16 14:31:03 +02:00
3hhh
170e5f5d7a
storage/callback: fix the rpm build 2020-07-16 14:31:03 +02:00
3hhh
dab41ddcf7
storage/callback: comments 2020-07-16 14:31:03 +02:00
3hhh
a00b2d563a
storage/callback: use Qubes exceptions 2020-07-16 14:31:03 +02:00
3hhh
49dd8250c5
storage/callback: added sphinx attribute comments 2020-07-16 14:31:03 +02:00
3hhh
bbb596e3ee
storage/callback: initialize logger in __init__ 2020-07-16 14:31:02 +02:00
3hhh
5530265b27
storage/callback: make pylint happy 2020-07-16 14:31:02 +02:00
3hhh
efa0d7c257
storage/callback: more readable bash invocation 2020-07-16 14:31:02 +02:00
3hhh
746697ad2c
storage: added the callback pool driver 2020-07-16 14:31:02 +02:00
Marek Marczykowski-Górecki
0bccddf1f5
Adjust code for possibly coroutine Volume.export() and Volume.export_end()
Now Volume.export() may be a coroutine and also may be accompanied by
Volume.export_end() cleaning up after it.

See previous commits for building blocks for this.

This commit adjusts usage of Volume.export() and adds matching
Volume.export_end() throughout the code base.

Fixes QubesOS/qubes-issues#5935
2020-07-08 12:50:10 +02:00
Marek Marczykowski-Górecki
d96480719f
storage: add Volume.export_end() function
This is a counterpart to Volume.export(). Up until now, no driver needed
any cleanup after exporting data, but it doesn't mean there won't be
any. This is especially relevant because Volume.export() is supposed to
return a path of a snapshot from before VM start - which may be a
different one than currently active one.

QubesOS/qubes-issues#5935
2020-07-08 06:05:02 +02:00
Marek Marczykowski-Górecki
d9d55b0586
storage: pass a copy of volume_config to pool.init_volume
Avoid local modification in a pool's init_volume influence
vm.volume_config. Currently every pool driver replaces
volume_config['pool'] with a pool object (instead of name) and it leads
to confusing cases where depending on start stage, it is sometimes an
object and sometimes a string.
Additionally, some pool drivers may modify volume_config in unexpected
way - for example test pool driver removes 'pool' entry entirely. Avoid
this fragile interface by giving pool driver a copy of volume_config,
instead of vm.volume_config directly.

Note one side effect is that 'vid' (and other pool-specific parameters)
is not set into vm.volume_config directly after creating a VM, but
possibly only after loading from XML. This should not be an issue in
theory (no core code should expect it), but if some place use
volume_config instead of Volume instance for getting pool-specific
options, it should be fixed.
2020-07-08 06:05:01 +02:00
Rusty Bird
1b09528740
storage/reflink: clarify comment 2020-07-07 16:25:48 +00:00
Rusty Bird
e188b93c95
storage: move @locked from lvm to Volume base class
And use it in reflink, instead of a synchronous lock.
2020-07-07 15:39:08 +00:00
Rusty Bird
a1b5262426
storeage/reflink: unlock size getter
Don't update _size in the getter, so it can be unlocked (which is
helpful for QubesOS/qubes-issues#5935).

!!! If cherry-picking for release4.0, also adjust import_data() to !!!
!!! use self.size (no underscore) instead of self._get_size()      !!!
2020-07-07 15:39:06 +00:00
Rusty Bird
b98c1814ee
storage/reflink: update _size during volatile volume stop 2020-07-07 15:39:05 +00:00
Rusty Bird
5ed762d152
storage: clean up __init__() of Pool() and subclasses
- Remove **kwargs
- Make all arguments keyword-only
- Use required arguments instead of asserts

QubesOS/qubes-issues#5895
2020-06-22 14:03:21 +00:00
Rusty Bird
652bf3b6d9
storage: modernize super() syntax 2020-06-22 14:03:20 +00:00
Rusty Bird
c2ce28936e
storage/reflink: prefer canonical values for setup_check 2020-06-22 14:03:19 +00:00
Rusty Bird
6f2d82fd20
storage/kernels: fix pool revisions_to_keep setter message 2020-06-22 14:03:18 +00:00
Rusty Bird
3ab789f5d8
storage/file: remove redundant _revisions_to_keep init
FilePool's revisions_to_keep setter - called in the next line via
Pool.__init__() - initializes the variable.
2020-06-22 14:03:17 +00:00
Frédéric Pierret (fepitre)
e4d7df4976
Prevent double hyphens in thin_pool parsing 2020-05-15 15:15:37 +02:00
Marek Marczykowski-Górecki
135eda0582
Merge remote-tracking branch 'origin/pr/320'
* origin/pr/320:
  storage/lvm: don't try to resize non-existing lvm volumes
  storage/lvm: update volume.size attribute on resize
2020-03-09 18:34:49 +01:00
Rusty Bird
8e581108b7
Remove pylint pragma working around very old false positive 2020-02-15 17:33:35 +00:00
Rusty Bird
1f71a06aed
Remove python2 compat __future__ imports 2020-02-15 17:33:34 +00:00
Marek Marczykowski-Górecki
86881afeef
storage/lvm: don't try to resize non-existing lvm volumes
Volume with save_on_stop=False and snap_on_start=False exists only when
VM is running and this is already handled under is_dirty() condition.
For non-running VMs, skip LVM resize step and only update internal size
attribute.

Fixes QubesOS/qubes-issues#5645
2020-02-12 21:57:34 +01:00
Marek Marczykowski-Górecki
6b474e39f6
storage/lvm: update volume.size attribute on resize
When volume is resized, underlying LVM is adjusted if exists. For
persistent volumes it is enough, as generally the driver prefers
underlying LVM size than internal attribute. But volatile volume is
removed at vm shutdown, so that value is lost.

Fix it by updating Volume.size attribute too, to be updated in qubes.xml
later too.

Fixes QubesOS/qubes-issues#5645
2020-02-12 21:44:16 +01:00
Rusty Bird
ada27ee431
storage/reflink: trivial style tweaks
Avoid early return for short and not deeply nested functions.
2020-02-05 17:26:44 +00:00
Rusty Bird
21971d6d0a
storage/reflink: comment on _get_size() use in start() 2020-02-05 17:26:43 +00:00
Rusty Bird
749ce477df
storage/reflink: don't bother using _get_size() in create()
Only the nominal size is available at this point.
2020-01-28 13:40:15 +00:00
Rusty Bird
6659ed8d39
storage/reflink: delete all images at beginning of create()
Ensure that there are no leftover image files for the volume, e.g. from
an unsuccessful removal of a previous incarnation of this vid, or from
an messily restored pool filesystem backup. We don't want to preserve
any stale data (revisions) or metadata (size) in the new incarnation.
2020-01-28 13:40:14 +00:00
Rusty Bird
56f6a6ef65
storage/reflink: get VM dir from less arbitrary-looking path 2020-01-28 13:40:13 +00:00
Rusty Bird
12d882b355
storage/reflink: factor out _remove_all_images() 2020-01-28 13:40:11 +00:00
Rusty Bird
8f4c90c37a
storage/reflink: _remove_incomplete_{files -> images}() 2020-01-28 13:40:10 +00:00
Rusty Bird
9c9d71c069
storage/reflink: omit redundant comment
The is_dirty() one-liner is defined right above.
2020-01-28 13:40:09 +00:00
Marek Marczykowski-Górecki
869f963335
Merge remote-tracking branch 'origin/pr/311'
* origin/pr/311:
  Add a test for loading volume config from XML
  Fix ThinVolume.size initialization from string
2020-01-24 01:38:16 +01:00
Marek Marczykowski-Górecki
edf5977b23
Merge remote-tracking branch 'origin/pr/309'
* origin/pr/309:
  import: check exact size of copied data
  Implement new admin.vm.ImportWithSize API call

Fixes QubesOS/qubes-issues#5239
2020-01-24 01:35:00 +01:00
Pawel Marczewski
49f2c1b78d
Fix ThinVolume.size initialization from string
The size config parameter might be a string coming from XML.
The Volume base class handles the conversion to integer already.

Fixes QubesOS/qubes-issues#5219.
2020-01-23 10:39:47 +01:00
Pawel Marczewski
63ac952803
Implement new admin.vm.ImportWithSize API call
This should allow importing a volume and changing the size at the
same time, without performing the resize operation on original
volume first.

The internal API has been renamed to internal.vm.volume.ImportBegin
to avoid confusion, and for symmetry with ImportEnd.

See QubesOS/qubes-issues#5239.
2020-01-23 09:47:22 +01:00
Rusty Bird
d54e4b0c6e
storage/reflink: fix comment 2020-01-17 16:45:29 +00:00
Rusty Bird
ba662d2819
storage/reflink: bail out early on most FICLONE errnos
Don't fall back on 'cp' if the FICLONE ioctl gives an errno that's not
plausibly reflink specific, because in such a case any fallback could
theoretically mask real but intermittent system/storage errors.

Looking through ioctl_ficlone(2) and the kernel source, it should be
sufficient to do the fallback only on EBADF/EINVAL/EOPNOTSUPP/EXDEV.
(EISDIR/ETXTBSY don't apply to this storage driver, which will never
legitimately attempt to reflink a directory or an active - in the
storage domain - swap file.)
2020-01-17 15:56:51 +00:00
Rusty Bird
90f25890cf
storage/reflink: pool.setup_check -> pool._setup_check 2020-01-17 15:56:50 +00:00
Marek Marczykowski-Górecki
d181c0f354
storage/file: fix resize
Fixes QubesOS/qubes-issues#5518
2019-12-14 15:48:49 +01:00
Rusty Bird
87081d6ee3
storage/reflink: _cleanup() -> _remove_incomplete_files()
"cleanup" sounds related to the concept of a volume being "dirty" - but
it's unrelated. Rename it for clarity.
2019-12-03 18:21:55 +00:00