Marek Marczykowski-Górecki
parent
5971873680
commit
82c3f85042
@ -16,6 +16,7 @@ manpages and API documentation. For primary user documentation, see
|
||||
qubes
|
||||
qubes-vm/index
|
||||
qubes-events
|
||||
qubes-storage
|
||||
qubes-exc
|
||||
qubes-ext
|
||||
qubes-log
|
||||
|
||||
145
doc/qubes-storage.rst
Normal file
145
doc/qubes-storage.rst
Normal file
@ -0,0 +1,145 @@
|
||||
:py:mod:`qubes.storage` -- Qubes data storage
|
||||
=============================================
|
||||
|
||||
Qubes provide extensible API for domains data storage. Each domain have
|
||||
multiple storage volumes, for different purposes. Each volume is provided by
|
||||
some storage pool. Qubes support different storage pool drivers, and it's
|
||||
possible to register additional 3rd-party drivers.
|
||||
|
||||
Domain's storage volumes:
|
||||
|
||||
- `root` - this is where operating system is installed. The volume is
|
||||
available read-write to :py:class:`~qubes.vm.templatevm.TemplateVM` and
|
||||
:py:class:`~qubes.vm.standalonevm.StandaloneVM`, and read-only to others
|
||||
(:py:class:`~qubes.vm.appvm.AppVM` and :py:class:`~qubes.vm.dispvm.DispVM`).
|
||||
- `private` - this is where domain's data live. The volume is available
|
||||
read-write to all domain classes (including :py:class:`~qubes.vm.dispvm.DispVM`,
|
||||
but data written there is discarded on domain shutdown).
|
||||
- `volatile` - this is used for any data that do not to persist. This include
|
||||
swap, copy-on-write layer for `root` volume etc.
|
||||
- `kernel` - domain boot files - operating system kernel, initial ramdisk,
|
||||
kernel modules etc. This volume is provided read-only and should be provided by
|
||||
a storage pool respecting :py:attr:`qubes.vm.qubesvm.QubesVM.kernel` property.
|
||||
|
||||
Storage pool concept
|
||||
--------------------
|
||||
|
||||
Storage pool is responsible for managing its volumes. Qubes have defined
|
||||
storage pool driver API, allowing to put domains storage in various places. By
|
||||
default two drivers are provided: :py:class:`qubes.storage.file.FilePool`
|
||||
(named `file`) and :py:class:`qubes.storage.lvm.ThinPool` (named `lvm_thin`).
|
||||
But the API allow to implement variety of other drivers (like additionally
|
||||
encrypted storage, external disk, drivers using special features of some
|
||||
filesystems like btrfs, etc).
|
||||
|
||||
Most of storage API focus on storage volumes. Each volume have at least those
|
||||
properties:
|
||||
- :py:attr:`~qubes.storage.Volume.rw` - should the volume be available
|
||||
read-only or read-write to the domain
|
||||
- :py:attr:`~qubes.storage.Volume.snap_on_start` - should the domain start
|
||||
with its own state of the volume, or rather a snapshot of its template volume
|
||||
(pointed by a :py:attr:`~qubes.storage.Volume.source` property). This can be
|
||||
set to `True` only if a domain do have `template` property (AppVM and DispVM).
|
||||
If the domain's template is running already, the snapshot should be made out of
|
||||
the template's before its startup.
|
||||
- :py:attr:`~qubes.storage.Volume.save_on_stop` - should the volume state be
|
||||
saved or discarded on domain
|
||||
stop. In either case, while the domain is running, volume's current state
|
||||
should not be committed immediately. This is to allow creating snapshots of the
|
||||
volume's state from before domain start (see
|
||||
:py:attr:`~qubes.storage.Volume.snap_on_start`).
|
||||
- :py:attr:`~qubes.storage.Volume.revisions_to_keep` - number of volume
|
||||
revisions to keep. If greater than zero, at each domain stop (and if
|
||||
:py:attr:`~qubes.storage.Volume.save_on_stop` is `True`) new revision is saved
|
||||
and old ones exceeding :py:attr:`~qubes.storage.Volume.revisions_to_keep` limit
|
||||
are removed.
|
||||
- :py:attr:`~qubes.storage.Volume.source` - source volume for
|
||||
:py:attr:`~qubes.storage.Volume.snap_on_start` volumes
|
||||
- :py:attr:`~qubes.storage.Volume.vid` - pool specific volume identifier, must
|
||||
be unique inside given pool
|
||||
- :py:attr:`~qubes.storage.Volume.pool` - storage pool object owning this volume
|
||||
- :py:attr:`~qubes.storage.Volume.name` - name of the volume inside owning
|
||||
domain (like `root`, or `private`)
|
||||
- :py:attr:`~qubes.storage.Volume.size` - size of the volume, in bytes
|
||||
|
||||
Storage pool driver may define additional properties.
|
||||
|
||||
Storage pool driver API
|
||||
-----------------------
|
||||
|
||||
Storage pool driver need to implement two classes:
|
||||
- pool class - inheriting from :py:class:`qubes.storage.Pool`
|
||||
- volume class - inheriting from :py:class:`qubes.storage.Volume`
|
||||
|
||||
Pool class should be registered with `qubes.storage` entry_point, under the
|
||||
name of storage pool driver. Volume class instances should be returned by
|
||||
:py:meth:`qubes.storage.Pool.init_volume` method of pool class instance.
|
||||
|
||||
Methods required to be implemented by the pool class:
|
||||
- :py:meth:`~qubes.storage.Pool.init_volume` - return instance of appropriate
|
||||
volume class; this method should not alter any persistent disk state, it is
|
||||
used to instantiate both existing volumes and create new ones
|
||||
- :py:meth:`~qubes.storage.Pool.setup` - setup new storage pool
|
||||
- :py:meth:`~qubes.storage.Pool.destroy` - destroy storage pool
|
||||
|
||||
Methods and properties required to be implemented by the volume class:
|
||||
- :py:meth:`~qubes.storage.Volume.create` - create volume on disk
|
||||
- :py:meth:`~qubes.storage.Volume.remove` - remove volume from disk
|
||||
- :py:meth:`~qubes.storage.Volume.start` - prepare the volume for domain start;
|
||||
this include making a snapshot if
|
||||
:py:attr:`~qubes.storage.Volume.snap_on_start` is `True`
|
||||
- :py:meth:`~qubes.storage.Volume.stop` - cleanup after domain shutdown; this
|
||||
include committing changes to the volume if
|
||||
:py:attr:`~qubes.storage.Volume.save_on_stop` is `True`
|
||||
- :py:meth:`~qubes.storage.Volume.export` - return a path to be read to extract
|
||||
volume data; for complex formats, this can be a pipe (connected to some
|
||||
data-extracting process)
|
||||
- :py:meth:`~qubes.storage.Volume.import_data` - return a path the data should
|
||||
be written to, to import volume data; for complex formats, this can be pipe
|
||||
(connected to some data-importing process)
|
||||
- :py:meth:`~qubes.storage.Volume.import_data_end` - finish data import
|
||||
operation (cleanup temporary files etc); this methods is called always after
|
||||
:py:meth:`~qubes.storage.Volume.import_data` regardless if operation was
|
||||
successful or not
|
||||
- :py:meth:`~qubes.storage.Volume.import_volume` - import data from another volume
|
||||
- :py:meth:`~qubes.storage.Volume.resize` - resize volume
|
||||
- :py:meth:`~qubes.storage.Volume.revert` - revert volume state to a given revision
|
||||
- :py:attr:`~qubes.storage.Volume.revisions` - collection of volume revisions (to use
|
||||
with :py:meth:`qubes.storage.Volume.revert`)
|
||||
- :py:meth:`~qubes.storage.Volume.is_dirty` - is volume properly committed
|
||||
after domain shutdown? Applies only to volumes with
|
||||
:py:attr:`~qubes.storage.Volume.save_on_stop` set to `True`
|
||||
- :py:meth:`~qubes.storage.Volume.is_outdated` - have the source volume started
|
||||
since domain startup? applies only to volumes with
|
||||
:py:attr:`~qubes.storage.Volume.snap_on_start` set to `True`
|
||||
- :py:attr:`~qubes.storage.Volume.config` - volume configuration, this should
|
||||
be enough to later reinstantiate the same volume object
|
||||
- :py:meth:`~qubes.storage.Volume.block_device` - return
|
||||
:py:class:`qubes.storage.BlockDevice` instance required to configure volume in
|
||||
libvirt
|
||||
|
||||
Some storage pool drivers can provide limited functionality only - for example
|
||||
support only `volatile` volumes (those with
|
||||
:py:attr:`~qubes.storage.Volume.snap_on_start` is `False`,
|
||||
:py:attr:`~qubes.storage.Volume.save_on_stop` is `False`, and
|
||||
:py:attr:`~qubes.storage.Volume.rw` is `True`). In that case, it should raise
|
||||
:py:exc:`NotImplementedError` in :py:meth:`qubes.storage.Pool.init_volume` when
|
||||
trying to instantiate unsupported volume.
|
||||
|
||||
Note that pool driver should be prepared to recover from power loss before
|
||||
stopping a domain - so, if volume have
|
||||
:py:attr:`~qubes.storage.Volume.save_on_stop` is `True`, and
|
||||
:py:meth:`qubes.storage.Volume.stop` wasn't called, next
|
||||
:py:meth:`~qubes.storage.Volume.start` should pick up previous (not committed)
|
||||
state.
|
||||
|
||||
See specific methods documentation for details.
|
||||
|
||||
Module contents
|
||||
---------------
|
||||
|
||||
.. automodule:: qubes.storage
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. vim: ts=3 sw=3 et
|
||||
@ -78,6 +78,8 @@ class Volume(object):
|
||||
domain = None
|
||||
path = None
|
||||
script = None
|
||||
#: disk space used by this volume, can be smaller than :py:attr:`size`
|
||||
#: for sparse volumes
|
||||
usage = 0
|
||||
|
||||
def __init__(self, name, pool, vid, internal=False, removable=False,
|
||||
@ -85,7 +87,7 @@ class Volume(object):
|
||||
snap_on_start=False, source=None, **kwargs):
|
||||
''' Initialize a volume.
|
||||
|
||||
:param str name: The domain name
|
||||
:param str name: The name of the volume inside owning domain
|
||||
:param Pool pool: The pool object
|
||||
:param str vid: Volume identifier needs to be unique in pool
|
||||
:param bool internal: If `True` volume is hidden when qvm-block ls
|
||||
@ -94,9 +96,12 @@ class Volume(object):
|
||||
run time
|
||||
:param int revisions_to_keep: Amount of revisions to keep around
|
||||
:param bool rw: If true volume will be mounted read-write
|
||||
:param bool snap_on_start: Create a snapshot from source on start
|
||||
:param bool save_on_stop: Write changes to disk in vm.stop()
|
||||
:param Volume source: other volume in same pool, or None
|
||||
:param bool snap_on_start: Create a snapshot from source on
|
||||
start, instead of using volume own data
|
||||
:param bool save_on_stop: Write changes to the volume in
|
||||
vm.stop(), otherwise - discard
|
||||
:param Volume source: other volume in same pool to make snapshot
|
||||
from, required if *snap_on_start*=`True`
|
||||
:param str/int size: Size of the volume
|
||||
|
||||
'''
|
||||
@ -106,16 +111,27 @@ class Volume(object):
|
||||
assert source is None or (isinstance(source, Volume)
|
||||
and source.pool == pool)
|
||||
|
||||
#: Name of the volume in a domain it's attached to (like `root` or
|
||||
#: `private`).
|
||||
self.name = str(name)
|
||||
#: :py:class:`Pool` instance owning this volume
|
||||
self.pool = pool
|
||||
self.internal = internal
|
||||
self.removable = removable
|
||||
#: How many revisions of the volume to keep. Each revision is created
|
||||
# at :py:meth:`stop`, if :py:attr:`save_on_stop` is True
|
||||
self.revisions_to_keep = int(revisions_to_keep)
|
||||
#: Should this volume be writable by domain.
|
||||
self.rw = rw
|
||||
#: Should volume state be saved or discarded at :py:meth:`stop`
|
||||
self.save_on_stop = save_on_stop
|
||||
self._size = int(size)
|
||||
#: Should the volume state be initialized with a snapshot of
|
||||
#: same-named volume of domain's template.
|
||||
self.snap_on_start = snap_on_start
|
||||
#: source volume for :py:attr:`snap_on_start` volumes
|
||||
self.source = source
|
||||
#: Volume unique (inside given pool) identifier
|
||||
self.vid = vid
|
||||
|
||||
def __eq__(self, other):
|
||||
@ -142,6 +158,10 @@ class Volume(object):
|
||||
def create(self):
|
||||
''' Create the given volume on disk.
|
||||
|
||||
This method is called only once in the volume lifetime. Before
|
||||
calling this method, no data on disk should be touched (in
|
||||
context of this volume).
|
||||
|
||||
This can be implemented as a coroutine.
|
||||
'''
|
||||
raise self._not_implemented("create")
|
||||
@ -153,17 +173,37 @@ class Volume(object):
|
||||
raise self._not_implemented("remove")
|
||||
|
||||
def export(self):
|
||||
''' Returns an object that can be `open()`. '''
|
||||
''' Returns a path to read the volume data from.
|
||||
|
||||
Reading from this path when domain owning this volume is
|
||||
running (i.e. when :py:meth:`is_dirty` is True) should return the
|
||||
data from before domain startup.
|
||||
|
||||
Reading from the path returned by this method should return the
|
||||
volume data. If extracting volume data require something more
|
||||
than just reading from file (for example connecting to some other
|
||||
domain, or decompressing the data), the returned path may be a pipe.
|
||||
'''
|
||||
raise self._not_implemented("export")
|
||||
|
||||
def import_data(self):
|
||||
''' Returns an object that can be `open()`. '''
|
||||
''' Returns a path to overwrite volume data.
|
||||
|
||||
This method is called after volume was already :py:meth:`create`-ed.
|
||||
|
||||
Writing to this path should overwrite volume data. If importing
|
||||
volume data require something more than just writing to a file (
|
||||
for example connecting to some other domain, or converting data
|
||||
on the fly), the returned path may be a pipe.
|
||||
'''
|
||||
raise self._not_implemented("import")
|
||||
|
||||
def import_data_end(self, success):
|
||||
''' End data import operation. This may be used by pool
|
||||
''' End the data import operation. This may be used by pool
|
||||
implementation to commit changes, cleanup temporary files etc.
|
||||
|
||||
This method is called regardless the operation was successful or not.
|
||||
|
||||
:param success: True if data import was successful, otherwise False
|
||||
'''
|
||||
# by default do nothing
|
||||
@ -173,19 +213,24 @@ class Volume(object):
|
||||
''' Imports data from a different volume (possibly in a different
|
||||
pool.
|
||||
|
||||
The needs to be create()d first.
|
||||
The volume needs to be create()d first.
|
||||
|
||||
This can be implemented as a coroutine. '''
|
||||
# pylint: disable=unused-argument
|
||||
raise self._not_implemented("import_volume")
|
||||
|
||||
def is_dirty(self):
|
||||
''' Return `True` if volume was not properly shutdown and commited '''
|
||||
''' Return `True` if volume was not properly shutdown and committed.
|
||||
|
||||
This include the situation when domain owning the volume is still
|
||||
running.
|
||||
|
||||
'''
|
||||
raise self._not_implemented("is_dirty")
|
||||
|
||||
def is_outdated(self):
|
||||
''' Returns `True` if the currently used `volume.source` of a snapshot
|
||||
volume is outdated.
|
||||
''' Returns `True` if this snapshot of a source volume (for
|
||||
`snap_on_start`=True) is outdated.
|
||||
'''
|
||||
raise self._not_implemented("is_outdated")
|
||||
|
||||
@ -195,23 +240,33 @@ class Volume(object):
|
||||
given size is less than current_size
|
||||
|
||||
This can be implemented as a coroutine.
|
||||
|
||||
:param int size: new size in bytes
|
||||
'''
|
||||
# pylint: disable=unused-argument
|
||||
raise self._not_implemented("resize")
|
||||
|
||||
def revert(self, revision=None):
|
||||
''' Revert volume to previous revision '''
|
||||
''' Revert volume to previous revision
|
||||
|
||||
:param revision: revision to revert volume to, see :py:attr:`revisions`
|
||||
'''
|
||||
# pylint: disable=unused-argument
|
||||
raise self._not_implemented("revert")
|
||||
|
||||
def start(self):
|
||||
''' Do what ever is needed on start
|
||||
''' Do what ever is needed on start.
|
||||
|
||||
This include making a snapshot of template's volume if
|
||||
:py:attr:`snap_on_start` is set.
|
||||
|
||||
This can be implemented as a coroutine.'''
|
||||
raise self._not_implemented("start")
|
||||
|
||||
def stop(self):
|
||||
''' Do what ever is needed on stop
|
||||
''' Do what ever is needed on stop.
|
||||
|
||||
This include committing data if :py:attr:`save_on_stop` is set.
|
||||
|
||||
This can be implemented as a coroutine.'''
|
||||
|
||||
@ -230,12 +285,14 @@ class Volume(object):
|
||||
|
||||
@property
|
||||
def revisions(self):
|
||||
''' Returns a `dict` containing revision identifiers and paths '''
|
||||
''' Returns a dict containing revision identifiers and time of their
|
||||
creation '''
|
||||
msg = "{!s} has revisions not implemented".format(self.__class__)
|
||||
raise NotImplementedError(msg)
|
||||
|
||||
@property
|
||||
def size(self):
|
||||
''' Volume size in bytes '''
|
||||
return self._size
|
||||
|
||||
@size.setter
|
||||
|
||||
Loading…
Reference in New Issue
Block a user