Go to file
Rusty Bird 1695a732b8
file-reflink, a storage driver optimized for CoW filesystems
This adds the file-reflink storage driver. It is never selected
automatically for pool creation, especially not the creation of
'varlibqubes' (though it can be used if set up manually).

The code is quite small:

               reflink.py  lvm.py      file.py + block-snapshot
    sloccount  334 lines   447 (134%)  570 (171%)

Background: btrfs and XFS (but not yet ZFS) support instant copies of
individual files through the 'FICLONE' ioctl behind 'cp --reflink'.
Which file-reflink uses to snapshot VM image files without an extra
device-mapper layer. All the snapshots are essentially freestanding;
there's no functional origin vs. snapshot distinction.

In contrast to 'file'-on-btrfs, file-reflink inherently avoids
CoW-on-CoW. Which is a bigger issue now on R4.0, where even AppVMs'
private volumes are CoW. (And turning off the lower, filesystem-level
CoW for 'file'-on-btrfs images would turn off data checksums too, i.e.
protection against bit rot.)

Also in contrast to 'file', all storage features are supported,
including

    - any number of revisions_to_keep
    - volume.revert()
    - volume.is_outdated
    - online fstrim/discard

Example tree of a file-reflink pool - *-dirty.img are connected to Xen:

    - /var/lib/testpool/appvms/foo/volatile-dirty.img
    - /var/lib/testpool/appvms/foo/root-dirty.img
    - /var/lib/testpool/appvms/foo/root.img
    - /var/lib/testpool/appvms/foo/private-dirty.img
    - /var/lib/testpool/appvms/foo/private.img
    - /var/lib/testpool/appvms/foo/private.img@2018-01-02T03:04:05Z
    - /var/lib/testpool/appvms/foo/private.img@2018-01-02T04:05:06Z
    - /var/lib/testpool/appvms/foo/private.img@2018-01-02T05:06:07Z
    - /var/lib/testpool/appvms/bar/...
    - /var/lib/testpool/appvms/...
    - /var/lib/testpool/template-vms/fedora-26/...
    - /var/lib/testpool/template-vms/...

It looks similar to a 'file' pool tree, and in fact file-reflink is
drop-in compatible:

    $ qvm-shutdown --all --wait
    $ systemctl stop qubesd
    $ sed 's/ driver="file"/ driver="file-reflink"/g' -i.bak /var/lib/qubes/qubes.xml
    $ systemctl start qubesd
    $ sudo rm -f /path/to/pool/*/*/*-cow.img*

If the user tries to create a fresh file-reflink pool on a filesystem
that doesn't support reflinks, qvm-pool will abort and mention the
'setup_check=no' option. Which can be passed to force a fallback on
regular sparse copies, with of course lots of time/space overhead. The
same fallback code is also used when initially cloning a VM from a
foreign pool, or from another file-reflink pool on a different
mountpoint.

'journalctl -fu qubesd' will show all file-reflink copy/rename/remove
operations on VM creation/startup/shutdown/etc.
2018-02-12 21:20:05 +00:00
ci api/admin: plug backup into Admin API 2017-07-20 12:19:01 +02:00
contrib events: add support for async event handlers 2017-07-04 13:04:22 +02:00
doc file-reflink, a storage driver optimized for CoW filesystems 2018-02-12 21:20:05 +00:00
etc Replace pool config parsing logic 2016-04-25 07:16:37 +02:00
linux Fix starting PCI-having HVMs on early system boot and later 2018-01-29 22:57:32 +01:00
qubes file-reflink, a storage driver optimized for CoW filesystems 2018-02-12 21:20:05 +00:00
qubes-rpc Merge remote-tracking branch 'qubesos/pr/169' 2017-12-06 00:46:15 +01:00
qubes-rpc-policy qubes-rpc: policy.RegisterArgument 2017-11-30 00:26:21 +01:00
qubespolicy Make pylint happy 2017-12-21 18:19:10 +01:00
qvm-tools qubes-hcl-report: detect AMD interrupt remapping 2017-12-05 17:39:33 +01:00
relaxng Change license to LGPL v2.1+ 2017-10-12 00:11:50 +02:00
rpm_spec file-reflink, a storage driver optimized for CoW filesystems 2018-02-12 21:20:05 +00:00
templates Switch to new PVH config variant 2018-01-24 02:48:00 +01:00
test-packages Fix starting VM with kernel=None 2017-12-14 23:26:52 +01:00
tests Change license to LGPL v2.1+ 2017-10-12 00:11:50 +02:00
.coveragerc add config for coverage.py 2015-10-12 16:40:46 +02:00
.gitignore run-tests: Script for running tests in repo 2016-04-20 13:54:56 +02:00
.pylintrc Remove obsolete options from .pylintrc 2016-07-21 12:21:00 +02:00
.travis.yml travis: flag codecov.io uploads with 'unittests' flag 2017-06-02 12:31:07 +02:00
installer.wxs windows/installer: configurable destination directory 2014-11-19 12:50:31 +01:00
LICENSE Change license to LGPL v2.1+ 2017-10-12 00:11:50 +02:00
Makefile doc: link qvm-device man page for qvm-block, qvm-pci, qvm-usb 2018-01-11 03:44:53 +01:00
Makefile.builder rpm: integrate -doc package into main one 2017-05-12 18:43:35 +02:00
README.md Update readme 2017-10-16 04:16:23 +02:00
run-tests tests: rearrage LVM thin tests to run at least some in devel env 2017-05-12 18:10:17 +02:00
setup.cfg Add yapf configuration to setup.cfg 2016-07-21 12:11:34 +02:00
setup.py file-reflink, a storage driver optimized for CoW filesystems 2018-02-12 21:20:05 +00:00
version version 4.0.21 2018-01-30 00:18:20 +01:00

Qubes core, version 3

Build Status

This is master branch of the Qubes OS core.

API documentation is available: https://dev.qubes-os.org/projects/core-admin/en/latest/.