Commit Graph

4976 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
dddd94b339
Merge branch 'core3-storage3' 2017-07-04 16:44:53 +02:00
Marek Marczykowski-Górecki
6db39345fb
Merge remote-tracking branch 'origin/core3-quick-fixes-20170703' 2017-07-04 16:11:18 +02:00
Marek Marczykowski-Górecki
51022cada5
Merge remote-tracking branch 'origin/policy-adminvm' 2017-07-04 15:51:30 +02:00
Marek Marczykowski-Górecki
cfbccc0a74
vm: drop old is_outdated/commit_changes
This code is unused now. Theoretically this is_outdated implementation
should be moved to FileVolume, but since we don't have VM reference
there, it isn't possible to read appropriate xenstore entry. As we're
phasing out file pool, simply ignore it for now.

QubesOS/qubes-issues#2256
2017-07-04 14:25:10 +02:00
Marek Marczykowski-Górecki
12adf8bede
storage/lvm: major cleanup, update
- remove obsolete volume types, use snap_on_start/save_on_stop directly
- handle multiple revisions
- implement is_outdated()

QubesOS/qubes-issues#2256
2017-07-04 14:25:09 +02:00
Marek Marczykowski-Górecki
c7ca4a445e
storage/kernels: support only save_on_stop=False volumes
LinuxKernel pool support only read-only volumes, so save_on_stop=True
doesn't make sense. Make it more explicit - raise NotImplementedError
otherwise.
Also, migrate old configs where snap_on_start=True, but no source was
given.

QubesOS/qubes-issues#2256
2017-07-04 14:25:08 +02:00
Marek Marczykowski-Górecki
0e554296e3
storage: drop 'internal' and 'removable' volume properties
Since dynamic volumes (qvm-block) are moved to devices API, those two
are not needed anymore.

QubesOS/qubes-issues#2256
2017-07-04 14:25:08 +02:00
Marek Marczykowski-Górecki
075d991114
config: eliminate duplicated qubes_base_dir
Remove it from system_path dict, have it in one place.
2017-07-04 14:25:07 +02:00
Marek Marczykowski-Górecki
317d140f46
storage/file: major FilePool/FileVolume cleanup and documentation
This driver isn't used in default Qubes 4.0 installation, but if we do
have it, let it follow defined API and its own documentation. And also
explicitly reject not supported operations:
 - support only revisions_to_keep<=1, but do not support revert() anyway
 (implemented version were wrong on so many levels...)
 - use 'save_on_stop'/'snap_on_start' properties directly instead of
 obsolete volume types
 - don't call sudo - qubesd is running as root
 - consistently use path, path_cow, path_source, path_source_cow

Also, add tests for BlockDevice instance returned by
FileVolume.block_device().

QubesOS/qubes-issues#2256
2017-07-04 14:25:07 +02:00
Marek Marczykowski-Górecki
1a1dd3dba2
storage: make default pool configurable
Do not always use pool named 'default'. Instead, have global
`default_pool` property to specify default storage pools.
Additionally add `default_pool_*` properties for each VM property, so
those can be set separately.

QubesOS/qubes-issues#2256
2017-07-04 14:25:06 +02:00
Marek Marczykowski-Górecki
ca63c71030
doc: typo fix 2017-07-04 13:04:25 +02:00
Marek Marczykowski-Górecki
8318038a73
tests: workaround asyncio.iscouroutinefunction(Mock)=True
It's already fixed in Python 3.5.3, but on Travis we have Python 3.5.2.
https://bugs.python.org/issue28703
2017-07-04 13:04:24 +02:00
Marek Marczykowski-Górecki
b5ad245614
vm/qubesvm: have VM lifecycle related events async
Those functions are coroutines anyway, so allow event handlers to be
too.
Some of this (`domain-create-on-disk`, `domain-remove-from-disk`) will
be useful for appmenus handling.
2017-07-04 13:04:24 +02:00
Marek Marczykowski-Górecki
d00e4538bf
devices: make attach/detach related events async
This will allow starting processes and calling RPC services in those
events. This if required for usb devices, which are attached using RPC
services.
Intentionally keep device listing events synchronous only - to
discourage putting long-running actions there.

This change also require some not-async attach method version for
loading devices from qubes.xml - have `load_persistent` for this.
2017-07-04 13:04:23 +02:00
Marek Marczykowski-Górecki
e5a9c46e3d
typos in docstrings 2017-07-04 13:04:22 +02:00
Marek Marczykowski-Górecki
ea1a04cb19
events: add support for async event handlers
See documentation for details.
2017-07-04 13:04:22 +02:00
Marek Marczykowski-Górecki
6238254f49
events: merge fire_event and fire_event_pre functions
Those functions really do very similar things - lets merge them and add
simple parameter.
2017-07-04 13:04:21 +02:00
Marek Marczykowski-Górecki
aed6de4937
events: typos in docstring 2017-07-04 12:58:39 +02:00
Marek Marczykowski-Górecki
3c4b60fa45
vm/adminvm: (re-)add vm.qdb property
Dom0 also expose some information in QubesDB - like list of block
devices.
2017-07-04 12:58:39 +02:00
Wojtek Porczyk
55c9370454 qubespolicy: make commas exchangeable for spaces
Commas in policy action are optional, if used with spaces.
2017-07-04 12:49:26 +02:00
Marek Marczykowski-Górecki
9d17c52a8d
tests: add missing import 2017-07-04 05:22:39 +02:00
Marek Marczykowski-Górecki
36d56010af
qubespolicy: add a commend about 'deny' being default action 2017-07-04 04:27:37 +02:00
Marek Marczykowski-Górecki
f393ce62dc
Fix docstrings formatting
Fix warnings/errors reported by Sphinx
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
e7f717ec3d
doc: fix formating of policy-related documentation
Fix Sphinx warnings and errors in both doc/ and docstrings.
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
e8e30c8bdf
qubespolicy: fix handling allow rule to '$dispvm'
When rule does not specify forced target (`target=...`), generic
`$dispvm` wasn't resolved to specific Disposable VM (based on
`default_dispvm` property).
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
a96a85bdc9
qubespolicy: add a tool to analyze policy in form of graph
Output possible connections between VMs in form of dot file.

Fixes QubesOS/qubes-issues#2873
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
8afb425271
qubespolicy: allow non-default policy directory
This will allow to evaluate policy extracted from other system.
And also ease tests.

QubesOS/qubes-issues#2873
2017-07-04 04:27:35 +02:00
Marek Marczykowski-Górecki
68f046cd31
Include qubespolicy in coverage report 2017-07-04 04:27:35 +02:00
Marek Marczykowski-Górecki
26ea836f67
qubespolicy: add $adminvm keyword for specifying dom0 aka AdminVM
Fixes QubesOS/qubes-issues#2872
2017-07-04 04:27:35 +02:00
Marek Marczykowski-Górecki
a937bb173a
qubespolicy: allow spaces in action arguments
This is natural to write space after coma.
2017-07-04 04:27:34 +02:00
Marek Marczykowski-Górecki
291a338e73
api: add missing docstring 2017-07-04 04:27:34 +02:00
Marek Marczykowski-Górecki
3d803acfde
Generate policy for Admin API calls based on annotations on actual methods
This ease Admin API administration, and also adds checking if qrexec
policy + scripts matches actual Admin API methods implementation.
The idea is to classify every Admin API method as either local
read-only, local read-write, global read-only or global read-write.
Where local/global means affecting a single VM, or the whole system.

See QubesOS/qubes-issues#2871 for details.

Fixes QubesOS/qubes-issues#2871
2017-07-04 04:27:34 +02:00
Marek Marczykowski-Górecki
82c3f85042
storage: add API documentation
QubesOS/qubes-issues#2256
2017-07-04 04:11:14 +02:00
Marek Marczykowski-Górecki
5971873680
storage: drop functions not being part of the API
commit/recover/reset should really be handled in start/stop. Nothing
stops specific pool implementation to define such functions privately.

QubesOS/qubes-issues#2256
2017-07-04 04:11:14 +02:00
Marek Marczykowski-Górecki
820539e909
storage: make volume snap_on_start/save_on_stop explicit
Always define those properties, always include them in volume config.
Also simplify overriding pool based on volume type defined by those:
override pool unless snap_on_start=True.

QubesOS/qubes-issues#2256
2017-07-04 04:11:14 +02:00
Marek Marczykowski-Górecki
697eb05c20
storage: drop rename support
Since VM name is immutable, rename method can be dropped from storage
API.

QubesOS/qubes-issues#2868
2017-07-04 04:11:14 +02:00
Marek Marczykowski-Górecki
dd1e05dc83
vm: drop rename related methods
Since VM name in immutable now, drop rename-related methods.

QubesOS/qubes-issues#2868
2017-07-04 04:11:13 +02:00
Marek Marczykowski-Górecki
2dbb51a38d
exc/block: fix attaching block device exposed by dom0
Don't set <backenddomain> element in libvirt XML when device is provided
by dom0.
2017-07-04 04:09:28 +02:00
Marek Marczykowski-Górecki
75608795b0
Disable SMAP in VMs
Linux kernel bug cause hypercall fails from HVM userspace (see
referenced issue). As a simple workaround, do not advertise SMAP to VMs
- but still use it as hypervisor level.

Fixes QubesOS/qubes-issues#2881
2017-07-04 04:09:27 +02:00
Marek Marczykowski-Górecki
3721ae3e8b
vm/adminvm: setup libvirt_domain property
Since libvirt do provide object for dom0 too, return it here.
It's much easier than special-casing AdminVM everywhere. And in fact
sometimes it is actually useful (for example attaching devices from/to
dom0, adjusting memory).
2017-07-04 04:09:27 +02:00
Marek Marczykowski-Górecki
e5de8f4115
devices: better handle exceptions in device extension
Do not fail app.save() just because listing devices failed, for any
reason.
2017-07-04 04:09:27 +02:00
Marek Marczykowski-Górecki
77e84b9ce4
Add qvm-block symlink 2017-07-04 04:09:27 +02:00
Marek Marczykowski-Górecki
3748eb3e2b
Merge branch 'core3-firewall2' 2017-07-04 03:38:59 +02:00
Marek Marczykowski-Górecki
f83c516082
Merge branch 'two-stage-clone' 2017-07-04 03:37:41 +02:00
Marek Marczykowski-Górecki
d0663940a7
qubes/api/admin: annotate API methods
Second attempt: this time use full words for scope, read, write,
execute.

QubesOS/qubes-issues#2871
2017-06-27 06:01:58 +02:00
Wojtek Porczyk
3e0d01cfb9
qubes/admin: Add listing of API methods 2017-06-27 06:01:58 +02:00
Marek Marczykowski-Górecki
2abdbc4628
firewall: minor simplification for old firewall.xml loading
Have `default_policy_is_accept` variable of type bool, instead of
`policy`, which is only compared to a constant value (`accept`).
Suggested by @woju
2017-06-26 18:45:59 +02:00
Marek Marczykowski-Górecki
2b963be9c7
firewall: pass untrusted values as keyword arguments
Use keyword arguments to pass untrusted arguments to make sure the
function parameter also have `untrusted_` prefix.
Suggested by @woju
2017-06-26 18:41:27 +02:00
Marek Marczykowski-Górecki
26a9974432
api/admin: split vm.volume.Clone to CloneFrom and CloneTo
The first operation returns a token, which can be passed to the second
one to actually perform clone operation. This way the caller needs have
power over both source and destination VMs (or at least appropriate
volumes), so it's easier to enforce appropriate qrexec policy.

The pending tokens are stored on Qubes() instance (as QubesAdminAPI is
not persistent). It is design choice to keep them in RAM only - those
are one time use and this way restarting qubesd is a simple way to
invalidate all of them. Otherwise we'd need some additional calls like
CloneCancel or such.

QubesOS/qubes-issues#2622
2017-06-26 13:27:48 +02:00
Marek Marczykowski-Górecki
3dcd29afea
api/admin: remove admin.vm.Clone operation
The same can be achieved with Create+volume.Clone

QubesOS/qubes-issues#2622
2017-06-26 13:10:00 +02:00