Commit Graph

4770 Commits

Author SHA1 Message Date
Wojtek Porczyk
b212a75ba3 qubes/vm/qubesvm: don't use .run_service() for .run()
QubesOS/qubes-issues#2622
2017-05-29 17:34:15 +02:00
Wojtek Porczyk
a9755ed76a qubes/ext/gui: remove is-fully-usable and is_guid_running 2017-05-29 17:34:15 +02:00
Wojtek Porczyk
0fdcbfc237 qubes-rpc/admin.vm.volume.Import: fix dd flags
QubesOS/qubes-issues#2622
2017-05-29 17:34:15 +02:00
Wojtek Porczyk
b256af3bfb qubes.tests asyncio
QubesOS/qubes-issues#2622
2017-05-29 17:34:15 +02:00
Wojtek Porczyk
345c16aa47 doc/manpages: fix qubes-create.1
The --property option went out in
83eef56f9d.
2017-05-26 17:19:30 +02:00
Marek Marczykowski-Górecki
7f3dd8b3d7
storage: don't fail on import if lvm is not installed 2017-05-26 15:08:20 +02:00
Marek Marczykowski-Górecki
e54cc11a2c
vm: expose to VM only features with 'service/' prefix
And place them in /qubes-service/ QubesDB directory. This allows
extensions to easily store some data not exposed to VM, but also have
control what VM will see. And at the same time, it make it compatible
with existing services framework

QubesOS/qubes-issues#1637
2017-05-26 15:08:19 +02:00
Marek Marczykowski-Górecki
2c01504a50
Add default policy for qubes.UpdatesProxy service
QubesOS/qubes-issues#1854
2017-05-26 15:08:19 +02:00
Marek Marczykowski-Górecki
6297919a72
rpm: drop dependency on now obsolete qubes-core-dom0-doc package 2017-05-26 15:08:18 +02:00
Marek Marczykowski-Górecki
607dcbaf37
admin-api: fix handling admin.vm.property.Set with None VM value
Setting VMProperty to None VM should be encoded as '' value (according
to VMProperty._none_value). But value validation rejected this value.

QubesOS/qubes-issues#2622
2017-05-26 15:08:18 +02:00
Marek Marczykowski-Górecki
305fb5b6d0
Move qvm-template-postprocess tool to core-admin-client
Since implementing admin.vm.volume.Import, this migrated to Admin API.
2017-05-26 15:08:17 +02:00
Marek Marczykowski-Górecki
d42197e94c
make 'mgmt-permission:' event compatible with check-events script
It doesn't support str.format().
2017-05-26 15:08:17 +02:00
Marek Marczykowski-Górecki
afdfbe4eed
Update contrib/check-events for python 3 2017-05-26 15:08:16 +02:00
Marek Marczykowski-Górecki
3177fc504f
tests: basic tests for admin.vm.volume.Import 2017-05-26 15:08:16 +02:00
Marek Marczykowski-Górecki
d7af67d875
storage: minor error handling fix 2017-05-26 15:08:15 +02:00
Marek Marczykowski-Górecki
6c29c653a0
tests: improve qubes_basdir_patching
File storage pool use qubes.config.qubes_base_dir, not
qubes.config.system_path, so patch that too.
2017-05-26 15:08:15 +02:00
Marek Marczykowski-Górecki
3cacf290bb
admin: implement admin.vm.volume.Import
Implement this in two parts:
1. Permissions checks, getting a path from appropriate storage pool
2. Actual data import

The first part is done by qubesd in a standard way, but then, instead of
accepting all the data (which may be several GB), return a path to which
a shell script (in practice: `dd` command) will write the data.
Then the script call back to qubesd again to report success/failure and
qubesd response from that call is actually returned to the user.

This way we do not pass all the data through qubesd, but still can
control the process from there in a meaningful way. Note that the last
part (second call to qubesd) may perform all kind of verification (like
a signature check on the data, or so) and can also prevent VM from
starting (hooking also domain-pre-start event) from not verified image.

QubesOS/qubes-issues#2622
2017-05-26 15:08:14 +02:00
Marek Marczykowski-Górecki
46b60dbf42
storage: add Pool.import_data to the API
Allow importing not only from another volume, but also raw data. In
practice, for all currently implemented storage pools, this is the same
as Pool.export, because path returned there is read-write. But lets not
abuse this fact, some future implementation may need different methods.

QubesOS/qubes-issues#2622
QubesOS/qubes-issues#2256
2017-05-23 15:35:55 +02:00
Marek Marczykowski-Górecki
1692601fcd
typo fix 2017-05-23 15:35:21 +02:00
Marek Marczykowski-Górecki
74689dd998
qubesd: reduce verbosity
Remove debug prints, log full traceback (of handled exception) only when
debug mode enabled (--debug, introduce in this commit too).
--debug option also enables sending tracebacks to the API clients.

QubesOS/qubes-issues#853
2017-05-23 04:47:50 +02:00
Marek Marczykowski-Górecki
64b83fa95a
admin: raise QubesNoSuchPropertyError for non-existing properties
Accessing non-existing property is a common action (for example
hasattr() do try to access the property). So, introduce specific
exception, inheriting from AttributeError. It will behave very similar
to standard (non-Admin-API) property access.

This exception is reported to the Admin API user, so it will be possible
to distinguish between non-existing property and access denied. But it
isn't any significant information leak, as list of valid properties is
publicly available in the source code.

QubesOS/qubes-issues#853
2017-05-23 04:32:59 +02:00
Marek Marczykowski-Górecki
f42cd28901
admin-api: create and install actual Admin API RPC endpoints
Install files in /etc/qubes-rpc for all methods defined in API
documentation, even if not yet implemented (qubesd will handle it
raising appropriate exception).
Use minimal program written in C (qubesd-query-fast), instead of
qubesd-query in python for performance reasons:
 - a single qubesd-query run: ~300ms
 - equivalent in shell (echo | nc -U): ~40ms
 - qubesd-query-fast: ~20ms

Many tools makes multiple API calls, so performance here do matter. For
example qvm-ls (from VM) currently takes about 60s on a system with 24
VMs.

Also make use of `$include:` directive in policy file, to make it easier
defining a VM with full Admin API access.

QubesOS/qubes-issues#853
2017-05-23 03:24:15 +02:00
Marek Marczykowski-Górecki
f93583e2be
devices: adjust XML serialization of device options
Use '<option name="option_name">option_value</option>' instead of
'<options option_name="option_value"/>'. It's more consistent with the
rest of qubes.xml - have one thing per element.

Also, add options deserialization test.
2017-05-22 17:11:20 +02:00
Marek Marczykowski-Górecki
29f3c9b58f
vm: don't try to define libvirt domain when just checking its state
When libvirt domain is not defined, it isn't running for sure.
This commit fixes the case when vm.is_running() appears anywhere in the
code used during libvirt xml building. In this case, it's mostly about
PCI device description for libvirt.
2017-05-22 17:11:19 +02:00
Marek Marczykowski-Górecki
6bc44b43de
pci: adjust PCIDeviceExtension device identifier syntax
':' is not allowed in device identifier, replace it with '_'.

Warning: this breaks existing qubes.xml
2017-05-22 17:11:19 +02:00
Marek Marczykowski-Górecki
fe37e0933e
Devices-related formating fixes
QubesOS/qubes-issues#
2017-05-22 03:21:13 +02:00
Marek Marczykowski-Górecki
dde9085920
vm: fix autostart-related events handling
event signature contains 'name', not 'prop' (it's no longer positional
argument).
2017-05-22 03:21:13 +02:00
Marek Marczykowski-Górecki
402afa1925
pci: use device attach options for disabling strict reset
Since we have now per-device options, it's more logical to use it here,
instead of features with device identifier encoded into feature name.
2017-05-22 03:21:13 +02:00
Marek Marczykowski-Górecki
d0b8933374
devices: add assignment.device property
Make it easy to retrieve DeviceInfo object out of DeviceAssignment
object. The only missing piece of information for that is device class,
so add it. Make it optional, as it can be filled on demand when passing
the object through DeviceCollection (either by listing devices, or
attaching/detaching).

This is mostly to ease handling options in libvirt template - to get
them, you need to use `assignments()`, istead of `persistent()` or
`attached()`, but there were no _simple_ way of getting actual device
object.

This also makes DeviceCollection._device method not needed anymore.
2017-05-22 03:21:07 +02:00
Marek Marczykowski-Górecki
227010d433
pci: fix device-pre-attach event signature
Now it contains also options. PCI extension do not use it, yet.
2017-05-19 18:49:22 +02:00
Marek Marczykowski-Górecki
f9fa27dacc
property: add property.__str__ 2017-05-19 18:49:21 +02:00
Marek Marczykowski-Górecki
f7cb8f27c5
admin: add admin.vm.Remove method
QubesOS/qubes-issues#2622
2017-05-19 18:49:21 +02:00
Marek Marczykowski-Górecki
880c190587
systemd: prevent killing VM-daemons on service restart/stop
Kill only main qubesd process, not all the control group (which include
all qrexec-daemon and qubesdb-daemon processes).
2017-05-18 11:11:18 +02:00
Marek Marczykowski-Górecki
0ef010cc70
qmemman: fix list/filter handling on python3
Filter in python3 returns a generator, can be iterated only once.
This is about list of existing domains - store it as a list, otherwise
domains will "disappear" after being discovered.
2017-05-18 09:44:48 +02:00
Marek Marczykowski-Górecki
58e58e0ca7
qubespolicy: improve logging
Include actual target to which service was allowed (either overriden by
policy, or chosen by user).

QubesOS/qubes-issues#910
2017-05-17 14:25:54 +02:00
Marek Marczykowski-Górecki
1831088c9b
api/internal: don't try to start dom0
qrexec-policy do try to start a VM before executing service. Avoid this
here, instead of qrexec-policy to keep the later clean.
2017-05-17 14:24:59 +02:00
Marek Marczykowski-Górecki
4c7dafe85e
rpm: cleanup scripts, use %systemd_* macros to handle services 2017-05-17 11:17:23 +02:00
Marek Marczykowski-Górecki
ae28d32af7
vm: make sure qmemman connection is closed even on failed VM startup
Qmemman hold global lock while serving requests, so not closing
connection will prevent any further action.
2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki
e5daf902b3
vm/net: fix IP address calculation
7 is not the same as 7 bits...
2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki
4d6d6c913a
qubesd: make sure NOTIFY_SOCKET variable is not propagated
qubesd do start other daemons - make sure they will not try to signal
systemd about it. In some cases such daemons (qubesdb-daemon) behave
differently based on this variable.
2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki
dc2563271c
tests: very basic tests for LinuxKernel pool 2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki
0b64e3fbe0
storage: make LinuxKernel pool track vm kernel version
Do not initialize it only at qubes.xml load time, but re-read vm.kernel
property each time the path is constructed. While at it, add support for
vm.kernel set to 'None' - simply don't include modules.img (xvdd) then.
2017-05-17 11:17:21 +02:00
Marek Marczykowski-Górecki
595d983659
storage: make verify() asyncio aware 2017-05-17 11:17:21 +02:00
Marek Marczykowski-Górecki
d57aef96e5
admin.vm.Start: report libvirt errors to caller
Convert them to QubesException for that.
2017-05-17 11:17:21 +02:00
Marek Marczykowski-Górecki
207d497bfe
Remove block-cleaner-daemon
This service currently does more harm (desync libvirt state with
reality) than good. Since we have qubesd, we may come back to
implementing it properly using libvirt events.
2017-05-17 01:33:30 +02:00
Marek Marczykowski-Górecki
774f4c8d49
tests: admin.vm.devices.* Admin API
QubesOS/qubes-issues#2622
2017-05-16 09:05:22 +02:00
Marek Marczykowski-Górecki
661752bbc1
devices: list persistent devices also when not currently attached
When device extension do not return some "persistent" device as
currently attached, still return it, as it will be attached at next
domain startup. User can distinguish such devices by having
frontend_domain=None (or other VM).
Also, return a set from DeviceCollection.assignments().
2017-05-16 09:05:22 +02:00
Marek Marczykowski-Górecki
2c0d79af4a
devices: pass attach options to device handling extension
Otherwise options would be ignored...
2017-05-16 09:05:22 +02:00
Marek Marczykowski-Górecki
75ac611c90
admin: admin.vm.device.* methods
QubesOS/qubes-issues#2622
2017-05-16 09:05:13 +02:00
Marek Marczykowski-Górecki
f3c7fb8129
admin: add missing parts to admin.vm.Clone
Clone all the things.
2017-05-16 09:05:00 +02:00