Qubes/core-admin
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
4,866 Commits 1 Branch 0 Tags 15 MiB
3a21e1f1b3
Commit Graph

193 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
4323651afb
storage/lvm: remove duplicated _reset function 
There were two: _reset and _reset_volume. Neither of them was working,
but the later was closer. Remove the other one.

QubesOS/qubes-issues#2256
 2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
37dbf29bc1
storage/lvm: don't fail on removing already removed volumes 
This may happen when removing not fully created VM.

QubesOS/qubes-issues#2256
 2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
400e92b25a
storage/lvm: misc fixes 
- add missing lvm remove call when commiting changes
- delay creating volatile image until domain startup (it will be created
  then anyway)
- reset cache only when really changed anything
- attach VM to the volume (snapshot) created for its runtime - to not
  expose changes (for example in root volume) to child VMs until
  shutdown

QubesOS/qubes-issues#2412
QubesOS/qubes-issues#2256
 2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
0471453773
storage/lvm: call lvm directly, don't use qubes-lvm wrapper 
The wrapper doesn't do anything else than translating command
parameters, but it's load time is significant (because of python imports
mostly). Since we can't use python lvm API from non-root user anyway,
lets drop the wrapper and call `lvm` directly (or through sudo when
necessary).

This makes VM startup much faster - storage preparation is down from
over 10s to about 3s.

QubesOS/qubes-issues#2256
 2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
9197bde76e
storage/lvm: use dd for importing volumes 
...instead of manual copy in python. DD is much faster and when used
with `conv=sparse` it will correctly preserve sparse image.

QubesOS/qubes-issues#2256
 2016-11-04 14:18:54 +01:00
Marek Marczykowski-Górecki
ae42308f5f
storage: improve handling volume export 
1. Add a helper function on vm.storage. This is equivalent of:

    vm.storage.get_pool(vm.volumes[name]).export(vm.volumes[name])

2. Make sure the path returned by `export` on LVM volume is accessible.
 2016-09-29 01:59:54 +02:00
Marek Marczykowski-Górecki
1af1784c69
Merge remote-tracking branch 'qubesos/pr/54' into core3-devel 
* qubesos/pr/54:
  Fix file storage resize
  Fix lvm reset_cache size cache
 2016-09-19 20:36:20 +02:00
Bahtiar `kalkin-` Gadimov
d63d93b0ab
Fix file storage resize 2016-09-05 16:56:57 +02:00
Bahtiar `kalkin-` Gadimov
d72448615d
Fix lvm reset_cache size cache 2016-09-04 23:25:39 +02:00
Marek Marczykowski-Górecki
2285789b49
Merge remote-tracking branch 'qubesos/pr/54' into core3-devel 
* qubesos/pr/54:
  Volume add size property wrapper
 2016-09-04 22:38:06 +02:00
Bahtiar `kalkin-` Gadimov
b863152f73
Volume add size property wrapper 2016-09-04 21:55:56 +02:00
Marek Marczykowski-Górecki
16db68b053
Merge branch 'core3-devices' into core3-devel 
* core3-devices:
  Fix core2migration and tests for new devices API
  tests: more qubes.devices tests
  qubes/ext/pci: implement pci-no-strict-reset/BDF feature
  qubes/tools: allow calling qvm-device as qvm-devclass (like qvm-pci)
  qubes: make pylint happy
  qubes/tools: add qvm-device tool (and tests)
  tests: load qubes.tests.tools.qvm_ls
  tests: PCI devices tests
  tests: add context manager to catch stdout
  qubes/ext/pci: move PCI devices handling to an extension
  qubes/devices: use more detailed exceptions than just KeyError
  qubes/devices: allow non-persistent attach
  qubes/storage: misc fixes for VM-exposed block devices handling
  qubes: new devices API

Fixes QubesOS/qubes-issues#2257
 2016-09-04 21:32:01 +02:00
Marek Marczykowski-Górecki
ae72e294cd
Merge remote-tracking branch 'qubesos/pr/52' into core3-devel 
* qubesos/pr/52:
  qvm-block extend Fix help message
  Fix a few typo bugs in qubes.storage.lvm
  qvm-block extends volumes to NEW_SIZE
  Make pylint happy ♥ qubes.storage.lvm
  Fix ThinVolume _size access
  qvm-block extend use qubes.utils.parse_size
  Make pylint happy ♥ qubes_lvm
  Fix lvm size/usage
  Add ThinPool.resize()
  Document qubes-block extend command in the manpage
  Fix file storage resize
  qvm-block add extend sub command
  qvm-block refactor attach/detach sub_parser init
 2016-09-04 21:21:43 +02:00
Marek Marczykowski-Górecki
6cb624daf3
Merge remote-tracking branch 'qubesos/pr/51' into core3-devel 
* qubesos/pr/51:
  Make pylint happy ♥ qubes.storage.lvm
  Fix lvm AppVM startup from lvm
  qubes.storage.lvm.qubes_lvm use debug not info
  Fix qvm-create lvm based AppVM
  qvm-block & qvm-pool add better Exception handling
  Remove unexpected property
  qvm-start --debug don't catch exceptions
 2016-09-04 21:02:19 +02:00
Marek Marczykowski-Górecki
0af5fdd267
qubes/storage/file: clone file images as normal user 
No need to be root here.

QubesOS/qubes-issues#2256
 2016-09-04 20:56:02 +02:00
Marek Marczykowski-Górecki
70d3f58024
qubes/storage: misc fixes for VM-exposed block devices handling 
Add 'backenddomain' element when source (not target) domain is not dom0.
Fix XML elemenet name. Actually set volume.domain when listing
VM-exposed devices.

QubesOS/qubes-issues#2256
 2016-09-03 20:41:02 +02:00
Bahtiar `kalkin-` Gadimov
e4433f9a65
Make pylint happy ♥ qubes.storage.lvm 2016-09-02 20:16:03 +02:00
Bahtiar `kalkin-` Gadimov
46eb4a72dd
Fix a few typo bugs in qubes.storage.lvm 2016-09-02 19:57:35 +02:00
Bahtiar `kalkin-` Gadimov
be48d48e56
Make pylint happy ♥ qubes.storage.lvm 2016-09-02 19:46:11 +02:00
Bahtiar `kalkin-` Gadimov
8974b2cbc0
Fix ThinVolume _size access 2016-09-02 19:45:31 +02:00
Bahtiar `kalkin-` Gadimov
1ca3c00797
Fix lvm size/usage 
- Uses a size_cache, because it's faster than doing a call for each volume
 2016-08-29 00:15:47 +02:00
Bahtiar `kalkin-` Gadimov
0561dfce60
Add ThinPool.resize() 2016-08-29 00:13:50 +02:00
Bahtiar `kalkin-` Gadimov
e7caa14a85
Fix file storage resize 2016-08-29 00:09:55 +02:00
Bahtiar `kalkin-` Gadimov
355ea0c2a8
Fix lvm AppVM startup from lvm 2016-08-29 00:05:19 +02:00
Bahtiar `kalkin-` Gadimov
24bbc971b0
qubes.storage.lvm.qubes_lvm use debug not info 2016-08-29 00:05:19 +02:00
Bahtiar `kalkin-` Gadimov
e0bf4bdb8e
Fix qvm-create lvm based AppVM 2016-08-29 00:05:18 +02:00
Marek Marczykowski-Górecki
06f46243ec
qubes/storage: add resize to Pool API 
It was already implemented by FilePool and used by Storage object, but
wasn't included in base class.
 2016-08-18 11:42:00 +02:00
Marek Marczykowski-Górecki
42d6fddcd3
qubes/storage: support on-line resize 
Call `qubes.ResizeDisk` qrexec service after resizing volume, to adjust
filesystem size there.
 2016-08-18 11:40:38 +02:00
Marek Marczykowski-Górecki
6e383c703d
qubes/storage: allow passing just volume name to storage.resize 
It makes much more sense to call:

    vm.storage.resize('private', 10000000000)

instead of:

    vm.storage.resize(vm.volumes['private'], 10000000000)
 2016-08-17 22:42:15 +02:00
Marek Marczykowski-Górecki
ab6f961931
qubes/dispvm: misc fixes, make it actually working 
- DispVM is no longer a special case for storage
- Add missing 'rw=True' for volatile volume
- Handle storage initialization (copy&paste from AppVM)
- Clone properties from DispVM template

QubesOS/qubes-issues#2253
 2016-08-17 00:42:17 +02:00
Marek Marczykowski-Górecki
6490767278
qubes/storage: update vm.storage.commit 
Better name it 'commit', as it isn't only for template. And also fix to
call `pool.commit`
 2016-08-09 05:14:11 +02:00
Bahtiar `kalkin-` Gadimov
02c8fc999c
ThinPool.import() skips not saveable volumes 2016-08-03 02:57:46 +02:00
Bahtiar `kalkin-` Gadimov
064629b63a
Fix qubes.lvm _reset 2016-08-03 02:57:44 +02:00
Bahtiar `kalkin-` Gadimov
9d30bb132a
Fix umask 2016-08-03 02:57:44 +02:00
Bahtiar `kalkin-` Gadimov
824c359319
Fix ThinPool revision handling 2016-08-03 02:57:43 +02:00
Bahtiar `kalkin-` Gadimov
3ae6530cd8
Add lvm thin pool storage implementation 2016-08-03 02:57:42 +02:00
Bahtiar `kalkin-` Gadimov
1467f1ede5
Storage add clone support 2016-07-13 22:35:57 +02:00
Bahtiar `kalkin-` Gadimov
95fed1eb71
qubes.linux.kernel use new storage api 2016-07-13 22:34:34 +02:00
Bahtiar `kalkin-` Gadimov
f60ccb235d
qubes.storage.domain use new storage API 2016-07-13 22:34:34 +02:00
Bahtiar `kalkin-` Gadimov
d1c606b952
qubes.storage.file use new storage API 2016-07-13 22:34:33 +02:00
Bahtiar `kalkin-` Gadimov
7e1563c88d
Add handling for old volume config 2016-07-13 22:34:31 +02:00
Bahtiar `kalkin-` Gadimov
1bccb146d8
Add qubes.storage.isodate() helper function 2016-07-13 22:34:31 +02:00
Bahtiar `kalkin-` Gadimov
7841e3f6c0
qubes.storage rework api 2016-07-13 22:34:30 +02:00
Wojtek Porczyk
1ff1ca37a1 Merge remote-tracking branch 'woju/pull/21/head' into core3-devel 
Conflicts:
	qubes/storage/__init__.py
 2016-07-02 11:54:56 +02:00
Wojtek Porczyk
c9accc258c Merge remote-tracking branch 'woju/pull/15/head' into core3-devel 
Conflicts:
	qubes/storage/__init__.py
	qubes/storage/file.py
	qubes/vm/qubesvm.py
 2016-06-28 15:15:00 +02:00
Bahtiar `kalkin-` Gadimov
e26932a81a
Fix bug in FilesPool.target_dir() 2016-06-23 14:23:34 +02:00
Bahtiar `kalkin-` Gadimov
7081bd433a
Make pylint happy 2016-06-23 14:23:34 +02:00
Bahtiar `kalkin-` Gadimov
b584d1a4bb
Remove QubesVM.is_netvm() and is_disposablevm() 2016-06-23 14:23:33 +02:00
Bahtiar `kalkin-` Gadimov
e08ca1ff57
Remove QubesVM.is_template() 2016-06-23 14:23:31 +02:00
Bahtiar `kalkin-` Gadimov
4db84c42a6
Fix qubes.storage.file _remove_if_exists 2016-06-23 13:17:00 +02:00
Bahtiar `kalkin-` Gadimov
8fc3772017
Add Volume.__str__() 2016-06-23 13:16:59 +02:00
Bahtiar `kalkin-` Gadimov
3dd77719c1
Pool.create source_volume argument is optional 2016-06-23 13:16:59 +02:00
Bahtiar `kalkin-` Gadimov
b9ddc00741
Add qvm-block 2016-06-23 13:16:08 +02:00
Bahtiar `kalkin-` Gadimov
90c882610e
Storage attach volumes from other pools 
- Already attached volumes are ignored
 2016-06-23 13:16:07 +02:00
Bahtiar `kalkin-` Gadimov
0ac40ddd4f
Volume.__repr__ returns POOL_NAME:VOLUME_ID 2016-06-23 13:16:04 +02:00
Bahtiar `kalkin-` Gadimov
09727b1cbd
Add Volume.internal field 
- Linux kernel volumes are always internal volumes
 2016-06-23 13:16:03 +02:00
Bahtiar `kalkin-` Gadimov
07800a0e67
Add Volume.eq, neq and hash 2016-06-23 13:16:03 +02:00
Bahtiar `kalkin-` Gadimov
f08ce2cb79
A Pool should always have a volumes property 
NOTE: FilesPool need some way to dynamically discover volumes
 2016-06-23 13:15:54 +02:00
Bahtiar `kalkin-` Gadimov
88198fb7ac
Storage verification is part of the pool interface 
- LinuxKernel pool add verify method
- FilePool implent verification
 2016-06-23 13:11:59 +02:00
Bahtiar `kalkin-` Gadimov
e7732f8ad0
Storage.verify fires domain-verify-files event 2016-06-21 14:58:11 +02:00
Bahtiar `kalkin-` Gadimov
732d2b33f4
Add LinuxKernel.is_outdated() 2016-06-21 14:58:10 +02:00
Bahtiar `kalkin-` Gadimov
3b441ebaad
Add Storage.outdated_volumes and Pool.is_outdated 2016-06-21 14:58:10 +02:00
Bahtiar `kalkin-` Gadimov
db608f6e01
Pool add str, eq & neq 2016-06-21 12:53:31 +02:00
Bahtiar `kalkin-` Gadimov
803efa76ff
Merge FilePool._resize_loop_device() with resize() 2016-06-21 12:48:57 +02:00
Bahtiar `kalkin-` Gadimov
72df863bb9
Fix qubes.storage.file pylint warnings 2016-06-21 12:47:47 +02:00
Bahtiar `kalkin-` Gadimov
20282c17fe
Volume add docstrings 2016-06-21 12:44:21 +02:00
Bahtiar `kalkin-` Gadimov
b830cb5544
Volume add eq, neq & hash methods 2016-06-21 12:39:47 +02:00
Bahtiar `kalkin-` Gadimov
9f3385296b qubes.storage.file fix pylint warnings 2016-06-16 16:59:27 +02:00
Bahtiar `kalkin-` Gadimov
38dc74587d Rewrote file rename_target_dir 2016-06-16 16:59:27 +02:00
Bahtiar `kalkin-` Gadimov
a4577c0dce Rename Storage.create_on_disk to Storage.create 2016-06-16 16:59:27 +02:00
Bahtiar `kalkin-` Gadimov
930fe417a8 Remove storage size and usage methods from QubesVM 2016-06-16 16:59:26 +02:00
Wojtek Porczyk
6ade5736d7 pylint fixes 2016-06-10 21:27:29 +02:00
Wojtek Porczyk
a615a45ecd Merge remote-tracking branch 'woju/pull/13/head' into core3-devel 2016-06-02 13:24:15 +02:00
Marek Marczykowski-Górecki
bb2e1f2870 qubes/vm: handle VM toplevel directory in QubesVM object not Storage 
This directory is not only for disk images (in fact disk images may be
elsewhere depending on choosen volume pool), so it would be cleaner to
handle (create/remove) it directly in QubesVM class.
 2016-06-02 13:22:07 +02:00
Marek Marczykowski-Górecki
0a21300872 qubes/storage: unify _remove_if_exists argument type 
In some places full volume object was called, in others - just file
path. Since this function is also used in some volume init/teardown, use
path everywhere.
 2016-06-02 13:07:35 +02:00
Marek Marczykowski-Górecki
b119b2c36b storage/file: simplify search for origin volume of snapshot/read-only 
Check directly vm.template, throwing AttributeError when not found.
There may be some value in converting it to more descriptive error, but
since that's mostly for internal users (not user facing actions) don't
bother for now.

QubesOS/qubes-issues#1842
 2016-06-02 12:38:07 +02:00
Marek Marczykowski-Górecki
54c70766a4 qubes/storage: allow snapshots of snapshots 
It may make sense to create 'snapshot' volume out of already 'snapshot',
not only 'origin'. In pracice it will exactly the same as 'snapshot
connected directly to 'origin'.

QubesOS/qubes-issues#866
 2016-06-02 12:37:00 +02:00
Marek Marczykowski-Górecki
48176d51f1 qubes/storage: minor error message fix 2016-06-02 12:29:39 +02:00
Marek Marczykowski-Górecki
d80f34888d qubes/storage: fix getting origin pool 
volume_config parameter is about target volume, not origin.

QubesOS/qubes-issues#1842
 2016-06-02 12:29:30 +02:00
Bahtiar `kalkin-` Gadimov
17790c32bb
Fix DomainPool missing a continue 2016-06-01 17:28:55 +02:00
Bahtiar `kalkin-` Gadimov
35974a5dbf
DomainPool check the untrusted data from qubes-db 2016-05-25 17:39:34 +02:00
Bahtiar `kalkin-` Gadimov
ddf040ae64
Do not serialize the domain pool config 2016-05-22 22:09:55 +02:00
Bahtiar `kalkin-` Gadimov
baaac858bc
Add DomainPool 
- All domain pool volumes are removable volumes
- DomainVolume uses device name as vid
 2016-05-22 22:09:54 +02:00
Bahtiar `kalkin-` Gadimov
e30f894df9
Add Volume.removable field 2016-05-22 21:42:27 +02:00
Bahtiar `kalkin-` Gadimov
116ba64e51
Storage.remove() catch IO/OSError and log it 2016-05-21 01:35:32 +02:00
Bahtiar `kalkin-` Gadimov
c5810758c5
FilePool fix origin volume removale 2016-05-21 01:35:31 +02:00
Bahtiar `kalkin-` Gadimov
8959e5a77e
Implement qvm-remove 
- Remove old qvm-remove
- Remove a log line from Storage, because it prints confusing lines, like:
    Removing volume kernel: /var/lib/qubes/vm-kernels/4.1.13-6/modules.img
 2016-05-21 01:35:30 +02:00
Bahtiar `kalkin-` Gadimov
1d5b89f0d5 LinuxKernel keep track of own volumes 
This is squashed woju/qubes-core-admin#8 by @kalkin

- LinuxKernel.volumes() lists all available kernels
- LinuxKernel use kernel version as vid
- LinuxKernel add docstrings
- Linux.kernel use os.listdir instead of os.walk
- LinuxKernel dynamically list available kernels
 2016-05-04 17:21:05 +02:00
Bahtiar `kalkin-` Gadimov
7200e6153b
Rename default storage driver from xen to file 
- Rename XenPool   ⇒ FilePool
- Rename XenVolume ⇒ FileVolume
 2016-04-30 20:42:46 +02:00
Bahtiar `kalkin-` Gadimov
b2c1017488 Fix missing parameter in Pool.__init__() 2016-04-30 20:42:00 +02:00
Bahtiar `kalkin-` Gadimov
04a3e80311 SizeMixIn first assert than call super() 2016-04-25 07:17:21 +02:00
Bahtiar `kalkin-` Gadimov
37ca33b0d1 Add docstring to xen volumes implementations 2016-04-25 07:17:21 +02:00
Bahtiar `kalkin-` Gadimov
8f060a8746 Fix Pool and Volume __init__ 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
591134833b Replace Volume.__str__ with enhanced __repr__ 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
d7ff4b9057 Move volume xml config from QubesVM to Volume 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
2e28849c90 Move pool xml config from Qubes to Pool 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
49b4951389 Storage move rename() logic to XenPool 
- Fix config renaming
 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
fe6a35155e Move kernel file checks to LinuxKernel pool 2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov
62c81044c5 Add XenPool.setup/destroy 2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov
97d04791b7 After add/remove_pool execute Pool.setup/destroy 2016-04-25 07:17:18 +02:00