Commit Graph

5139 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
f2f89c7b0c
storage: do not use deepcopy on volume configs
There may be Pool or Volume object references, which is intentional to
keep them as is - not copy whole Pool/Volume objects.
2017-07-12 21:33:14 +02:00
Marek Marczykowski-Górecki
e6d5337fa7
api: cleanup already started servers when some later failed 2017-07-12 21:33:14 +02:00
Marek Marczykowski-Górecki
d7b968d867
tests: fix block devices tests when running on real system 2017-07-12 21:33:14 +02:00
Marek Marczykowski-Górecki
7aa8d74ad4
tests: fix some FD leaks
There are still much more of them...
2017-07-12 21:33:14 +02:00
Marek Marczykowski-Górecki
58f4369a7e
make pylint happy 2017-07-12 21:14:23 +02:00
Marek Marczykowski-Górecki
dce0b28cf4
clock sync: drop untrusted_ prefix after validation, fix error msg 2017-07-12 20:57:52 +02:00
Marta Marczykowska-Górecka
ab88cd9644
minor amends to clock synchronization
renamed date_out variable to untrusted_date_out
2017-07-11 21:39:01 +02:00
Marek Marczykowski-Górecki
c32f0db582
vm/dispvm: convert DispVM related function to coroutines
Some functions used there (create_on_disk, remove_from_disk, kill) are
coroutines, so callers needs to be too.

Fixes QubesOS/qubes-issues#2896
2017-07-08 03:04:55 +02:00
Marek Marczykowski-Górecki
148d1cda78
vm/dispvm: fix DispVM storage definition
Specify empty 'source' field, so it gets filled with appropriate
template's images. Then also fix recursive 'source' handling - DispVM
root volume should point at TemplateVM's root volume as a source, not a
AppVM's one - which is also only a snapshot.

Fixes QubesOS/qubes-issues#2896
2017-07-08 02:58:21 +02:00
Marek Marczykowski-Górecki
a9ece6f706
Remove qubes-netvm.service
Since qubesd properly handle chained startup of sys-net->sys-firewall
etc, we don't need a separate service to start netvm explicitly earlier.

Fixes QubesOS/qubes-issues#2533
2017-07-07 20:35:13 +02:00
Marta Marczykowska-Górecka
6da06d424e
clock synchronization rewrite
clock synchronization mechanism rewritten to use systemd-timesync instead of NtpDate; at the moment, requires:
- modifying /etc/qubes-rpc/policy/qubes.GetDate to redirect GetDate to designated clockvm
- enabling clocksync service in clockvm ( qvm-features clockvm-name service/clocksync true )

Works as specified in issue listed below, except for:
- each VM synces with clockvm after boot and every 6h
- clockvm synces time with the Internet using systemd-timesync
- dom0 synces itself with clockvm every 1h (using cron)

fixes QubesOS/qubes-issues#1230
2017-07-06 23:37:26 +02:00
Wojtek Porczyk
b04f612374 qubes: have "service" features' keys separated by period 2017-07-05 04:16:16 +02:00
Marek Marczykowski-Górecki
d9db4f747c
version 4.0.1 2017-07-04 23:15:10 +02:00
Marek Marczykowski-Górecki
dddd94b339
Merge branch 'core3-storage3' 2017-07-04 16:44:53 +02:00
Marek Marczykowski-Górecki
6db39345fb
Merge remote-tracking branch 'origin/core3-quick-fixes-20170703' 2017-07-04 16:11:18 +02:00
Marek Marczykowski-Górecki
51022cada5
Merge remote-tracking branch 'origin/policy-adminvm' 2017-07-04 15:51:30 +02:00
Marek Marczykowski-Górecki
cfbccc0a74
vm: drop old is_outdated/commit_changes
This code is unused now. Theoretically this is_outdated implementation
should be moved to FileVolume, but since we don't have VM reference
there, it isn't possible to read appropriate xenstore entry. As we're
phasing out file pool, simply ignore it for now.

QubesOS/qubes-issues#2256
2017-07-04 14:25:10 +02:00
Marek Marczykowski-Górecki
12adf8bede
storage/lvm: major cleanup, update
- remove obsolete volume types, use snap_on_start/save_on_stop directly
- handle multiple revisions
- implement is_outdated()

QubesOS/qubes-issues#2256
2017-07-04 14:25:09 +02:00
Marek Marczykowski-Górecki
c7ca4a445e
storage/kernels: support only save_on_stop=False volumes
LinuxKernel pool support only read-only volumes, so save_on_stop=True
doesn't make sense. Make it more explicit - raise NotImplementedError
otherwise.
Also, migrate old configs where snap_on_start=True, but no source was
given.

QubesOS/qubes-issues#2256
2017-07-04 14:25:08 +02:00
Marek Marczykowski-Górecki
0e554296e3
storage: drop 'internal' and 'removable' volume properties
Since dynamic volumes (qvm-block) are moved to devices API, those two
are not needed anymore.

QubesOS/qubes-issues#2256
2017-07-04 14:25:08 +02:00
Marek Marczykowski-Górecki
075d991114
config: eliminate duplicated qubes_base_dir
Remove it from system_path dict, have it in one place.
2017-07-04 14:25:07 +02:00
Marek Marczykowski-Górecki
317d140f46
storage/file: major FilePool/FileVolume cleanup and documentation
This driver isn't used in default Qubes 4.0 installation, but if we do
have it, let it follow defined API and its own documentation. And also
explicitly reject not supported operations:
 - support only revisions_to_keep<=1, but do not support revert() anyway
 (implemented version were wrong on so many levels...)
 - use 'save_on_stop'/'snap_on_start' properties directly instead of
 obsolete volume types
 - don't call sudo - qubesd is running as root
 - consistently use path, path_cow, path_source, path_source_cow

Also, add tests for BlockDevice instance returned by
FileVolume.block_device().

QubesOS/qubes-issues#2256
2017-07-04 14:25:07 +02:00
Marek Marczykowski-Górecki
1a1dd3dba2
storage: make default pool configurable
Do not always use pool named 'default'. Instead, have global
`default_pool` property to specify default storage pools.
Additionally add `default_pool_*` properties for each VM property, so
those can be set separately.

QubesOS/qubes-issues#2256
2017-07-04 14:25:06 +02:00
Marek Marczykowski-Górecki
ca63c71030
doc: typo fix 2017-07-04 13:04:25 +02:00
Marek Marczykowski-Górecki
8318038a73
tests: workaround asyncio.iscouroutinefunction(Mock)=True
It's already fixed in Python 3.5.3, but on Travis we have Python 3.5.2.
https://bugs.python.org/issue28703
2017-07-04 13:04:24 +02:00
Marek Marczykowski-Górecki
b5ad245614
vm/qubesvm: have VM lifecycle related events async
Those functions are coroutines anyway, so allow event handlers to be
too.
Some of this (`domain-create-on-disk`, `domain-remove-from-disk`) will
be useful for appmenus handling.
2017-07-04 13:04:24 +02:00
Marek Marczykowski-Górecki
d00e4538bf
devices: make attach/detach related events async
This will allow starting processes and calling RPC services in those
events. This if required for usb devices, which are attached using RPC
services.
Intentionally keep device listing events synchronous only - to
discourage putting long-running actions there.

This change also require some not-async attach method version for
loading devices from qubes.xml - have `load_persistent` for this.
2017-07-04 13:04:23 +02:00
Marek Marczykowski-Górecki
e5a9c46e3d
typos in docstrings 2017-07-04 13:04:22 +02:00
Marek Marczykowski-Górecki
ea1a04cb19
events: add support for async event handlers
See documentation for details.
2017-07-04 13:04:22 +02:00
Marek Marczykowski-Górecki
6238254f49
events: merge fire_event and fire_event_pre functions
Those functions really do very similar things - lets merge them and add
simple parameter.
2017-07-04 13:04:21 +02:00
Marek Marczykowski-Górecki
aed6de4937
events: typos in docstring 2017-07-04 12:58:39 +02:00
Marek Marczykowski-Górecki
3c4b60fa45
vm/adminvm: (re-)add vm.qdb property
Dom0 also expose some information in QubesDB - like list of block
devices.
2017-07-04 12:58:39 +02:00
Wojtek Porczyk
55c9370454 qubespolicy: make commas exchangeable for spaces
Commas in policy action are optional, if used with spaces.
2017-07-04 12:49:26 +02:00
Marek Marczykowski-Górecki
9d17c52a8d
tests: add missing import 2017-07-04 05:22:39 +02:00
Marek Marczykowski-Górecki
36d56010af
qubespolicy: add a commend about 'deny' being default action 2017-07-04 04:27:37 +02:00
Marek Marczykowski-Górecki
f393ce62dc
Fix docstrings formatting
Fix warnings/errors reported by Sphinx
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
e7f717ec3d
doc: fix formating of policy-related documentation
Fix Sphinx warnings and errors in both doc/ and docstrings.
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
e8e30c8bdf
qubespolicy: fix handling allow rule to '$dispvm'
When rule does not specify forced target (`target=...`), generic
`$dispvm` wasn't resolved to specific Disposable VM (based on
`default_dispvm` property).
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
a96a85bdc9
qubespolicy: add a tool to analyze policy in form of graph
Output possible connections between VMs in form of dot file.

Fixes QubesOS/qubes-issues#2873
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
8afb425271
qubespolicy: allow non-default policy directory
This will allow to evaluate policy extracted from other system.
And also ease tests.

QubesOS/qubes-issues#2873
2017-07-04 04:27:35 +02:00
Marek Marczykowski-Górecki
68f046cd31
Include qubespolicy in coverage report 2017-07-04 04:27:35 +02:00
Marek Marczykowski-Górecki
26ea836f67
qubespolicy: add $adminvm keyword for specifying dom0 aka AdminVM
Fixes QubesOS/qubes-issues#2872
2017-07-04 04:27:35 +02:00
Marek Marczykowski-Górecki
a937bb173a
qubespolicy: allow spaces in action arguments
This is natural to write space after coma.
2017-07-04 04:27:34 +02:00
Marek Marczykowski-Górecki
291a338e73
api: add missing docstring 2017-07-04 04:27:34 +02:00
Marek Marczykowski-Górecki
3d803acfde
Generate policy for Admin API calls based on annotations on actual methods
This ease Admin API administration, and also adds checking if qrexec
policy + scripts matches actual Admin API methods implementation.
The idea is to classify every Admin API method as either local
read-only, local read-write, global read-only or global read-write.
Where local/global means affecting a single VM, or the whole system.

See QubesOS/qubes-issues#2871 for details.

Fixes QubesOS/qubes-issues#2871
2017-07-04 04:27:34 +02:00
Marek Marczykowski-Górecki
82c3f85042
storage: add API documentation
QubesOS/qubes-issues#2256
2017-07-04 04:11:14 +02:00
Marek Marczykowski-Górecki
5971873680
storage: drop functions not being part of the API
commit/recover/reset should really be handled in start/stop. Nothing
stops specific pool implementation to define such functions privately.

QubesOS/qubes-issues#2256
2017-07-04 04:11:14 +02:00
Marek Marczykowski-Górecki
820539e909
storage: make volume snap_on_start/save_on_stop explicit
Always define those properties, always include them in volume config.
Also simplify overriding pool based on volume type defined by those:
override pool unless snap_on_start=True.

QubesOS/qubes-issues#2256
2017-07-04 04:11:14 +02:00
Marek Marczykowski-Górecki
697eb05c20
storage: drop rename support
Since VM name is immutable, rename method can be dropped from storage
API.

QubesOS/qubes-issues#2868
2017-07-04 04:11:14 +02:00
Marek Marczykowski-Górecki
dd1e05dc83
vm: drop rename related methods
Since VM name in immutable now, drop rename-related methods.

QubesOS/qubes-issues#2868
2017-07-04 04:11:13 +02:00