Commit Graph

567 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
6297919a72
rpm: drop dependency on now obsolete qubes-core-dom0-doc package 2017-05-26 15:08:18 +02:00
Marek Marczykowski-Górecki
305fb5b6d0
Move qvm-template-postprocess tool to core-admin-client
Since implementing admin.vm.volume.Import, this migrated to Admin API.
2017-05-26 15:08:17 +02:00
Marek Marczykowski-Górecki
f42cd28901
admin-api: create and install actual Admin API RPC endpoints
Install files in /etc/qubes-rpc for all methods defined in API
documentation, even if not yet implemented (qubesd will handle it
raising appropriate exception).
Use minimal program written in C (qubesd-query-fast), instead of
qubesd-query in python for performance reasons:
 - a single qubesd-query run: ~300ms
 - equivalent in shell (echo | nc -U): ~40ms
 - qubesd-query-fast: ~20ms

Many tools makes multiple API calls, so performance here do matter. For
example qvm-ls (from VM) currently takes about 60s on a system with 24
VMs.

Also make use of `$include:` directive in policy file, to make it easier
defining a VM with full Admin API access.

QubesOS/qubes-issues#853
2017-05-23 03:24:15 +02:00
Marek Marczykowski-Górecki
4c7dafe85e
rpm: cleanup scripts, use %systemd_* macros to handle services 2017-05-17 11:17:23 +02:00
Marek Marczykowski-Górecki
dc2563271c
tests: very basic tests for LinuxKernel pool 2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki
207d497bfe
Remove block-cleaner-daemon
This service currently does more harm (desync libvirt state with
reality) than good. Since we have qubesd, we may come back to
implementing it properly using libvirt events.
2017-05-17 01:33:30 +02:00
Marek Marczykowski-Górecki
cd489f46e1
Rename MgmtAPI to AdminAPI - part 1: classes
QubesOS/qubes-issues#853
2017-05-12 19:28:07 +02:00
Marek Marczykowski-Górecki
e5596fe1ba
rpm: integrate -doc package into main one
It doesn't really make sense to keep man pages in separate package.
Previously it was done to avoid some build dependencies (pandoc require
a lot of them), but it isn't a problem anymore.
2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki
81162a699b
Remove unused helper scripts 2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki
2e7f483a60
Remove old pvusb handling code 2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki
a6df6df449
doc: remove man pages of just removed tools 2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki
83eef56f9d
Remove tools that are moved to -client repository
QubesOS/qubes-issues#853
2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki
8992e71f85
Remove core2 code
This all either have been migrated to core3, or is not needed anymore.

There is still qvm-tools directory with a few tools that needs to be
migrated, or installed as is.
2017-05-12 18:36:29 +02:00
Marek Marczykowski-Górecki
59426b2dbc
tests: tools.qubesd
For now only QubesDaemonProtocol test. main function still untested

QubesOS/qubes-issues#2622
2017-05-12 14:27:47 +02:00
Marek Marczykowski-Górecki
68c3d7309a
tools: remove qubes_monitor_layout_notify tool - moved to mgmt-client 2017-05-12 14:25:30 +02:00
Marek Marczykowski-Górecki
55037e0446
tests: tools.qubesd
For now only QubesDaemonProtocol test. main function still untested

QubesOS/qubes-issues#2622
2017-05-12 14:25:29 +02:00
Marek Marczykowski-Górecki
2aa0de3d5b
tools: remove qvm-ls tool and related integration in qubes.property
qvm-ls tool (as all other tools) will be accessing properties through
API, so no need (nor sense) for this tool-specific attributes in
qubes.property. The only somehow used was ls_width, and in fact it made
the output unnecessary wide.

The tool itself is already moved to core-mgmt-client repository.

QubesOS/qubes-issues#853
2017-05-12 14:25:27 +02:00
Marek Marczykowski-Górecki
a3da85bfda
qubespolicy: run GUI code inside user session and expose it as dbus object
This way it will work independently from where qrexec-policy tool will
be called (in most cases - from a system service, as root).
This is also very similar architecture to what we'll need when moving to
GUI domain - there GUI part will also be separated from policy
evaluation logic.

QubesOS/qubes-issues#910
2017-04-07 17:07:30 +02:00
Marek Marczykowski-Górecki
44e9ce4189
rpm: add rpc-window related files to package
QubesOS/qubes-issues#910
2017-04-07 17:07:29 +02:00
Marek Marczykowski-Górecki
83526a28d3
qubesd: add second socket for in-dom0 internal calls
This socket (and commands) are not exposed to untrusted input, so no
need to extensive sanitization. Also, there is no need to provide a
stable API here, as those methods are used internally only.

QubesOS/qubes-issues#853
2017-04-07 17:07:28 +02:00
Marek Marczykowski-Górecki
c9b5d0ab15
policy: qrexec-policy cli tool
This is the tool called by qrexec-daemon.
2017-04-06 15:43:17 +02:00
Marek Marczykowski-Górecki
e705a04cc5
qubespolicy: initial version for core3
This is rewritten version of core-admin-linux/qrexec/qrexec-policy.

It's placed outside of `qubes` module on purpose - to avoid imporing it,
which require a lot of time.

QubesOS/qubes-issues#865
QubesOS/qubes-issues#910
2017-04-06 15:43:17 +02:00
Wojtek Porczyk
1b9479837a Merge remote-tracking branch 'marmarek/core3-mgmt-methods1' into core3-devel
Conflicts:
	qubes/mgmt.py
2017-04-01 18:33:54 +02:00
Wojtek Porczyk
8defc883a4 spec: update and tidy up python dependencies
QubesOS/qubes-issues#2622
2017-03-31 00:46:37 +02:00
Wojtek Porczyk
8e3621c4e5 Move libviraio to core-libvirt, may be upstreamed
QubesOS/qubes-issues#2622
2017-03-22 15:30:55 +01:00
Marek Marczykowski-Górecki
944bb26369
tests/mgmt: VM property related functions 2017-03-16 20:04:05 +01:00
Marek Marczykowski-Górecki
1363251438
Revert "Revert "backup: use 'scrypt' tool for backup encryption and integrity protection""
This reverts commit 0f1672dc63.

Bring it back. Lets not revert the whole feature just because required
package exists only in qubes-builder, not in some online repository.
Also, this revert didn't go as planned - there was a reference to a
'passphrase' local variable, but it wasn't assigned any value.

Cc: @woju
2017-02-27 02:37:50 +01:00
Wojtek Porczyk
25d81b8ab6 Merge remote-tracking branch 'origin/pull/88/head' into core3-devel 2017-02-15 12:17:41 +01:00
Marek Marczykowski-Górecki
c3fc4062d8
tests: add basic test for qvm-features 2017-02-15 00:01:33 +01:00
Wojtek Porczyk
249d8c08e2 qubes/tools/qubesd-query: low-level interrogation tool 2017-02-10 23:25:45 +01:00
Wojtek Porczyk
c12fc744a2 qubes/mgmt: move mgmt api to separate module
QubesOS/qubes-issues#2622
2017-02-09 23:29:05 +01:00
Wojtek Porczyk
0be3b1fbb1 qubes/tools/qubesd: initial version 2017-02-07 17:07:53 +01:00
Wojtek Porczyk
1be75d9c83 misc python3 fixes 2017-02-07 17:07:52 +01:00
Wojtek Porczyk
9f7ba53eec fix Makefile, setup.py, spec and travis
QubesOS/qubes-issues#2074
2017-01-20 18:31:09 +01:00
Wojtek Porczyk
0f1672dc63 Revert "backup: use 'scrypt' tool for backup encryption and integrity protection"
This reverts commit 418d749680.

Package `scrypt` is currently not installable (not present in any repo).

Cc: @marmarek
2016-12-05 18:36:13 +01:00
Wojtek Porczyk
25912f5787 qubes/tools: add qvm-tags
QubesOS/qubes-issues#865
2016-12-02 14:17:09 +01:00
Marek Marczykowski-Górecki
194fb9b99b
rpm: disable prelink only when present
It isn't installed by default.

QubesOS/qubes-issues#2412
2016-11-26 04:08:59 +01:00
Marek Marczykowski-Górecki
badc58837a
Add qvm-template-postprocess tool
This is intended to call to finish template installation/removal.
Template RPM package is basically container for root.img, nothing more.
Other parts needs to be generated after root.img extraction. Previously
it was open coded in rpm post-install script, but lets keep it as qvm
tool to ease supporting multiple version in template builder

QubesOS/qubes-issues#2412
2016-11-26 04:08:08 +01:00
Marek Marczykowski-Górecki
408606e0f0
rpm: adjust post-installation script for new tools
QubesOS/qubes-issues#2412
2016-11-26 04:08:08 +01:00
Wojtek Porczyk
a4d50409df Merge remote-tracking branch 'marmarek/core3-storage-fixes' into core3-devel 2016-11-15 17:36:53 +01:00
Marek Marczykowski-Górecki
b011cef8af
tests/storage: add tests for basic volumes properties
Things like if read-only volume is really read-only, volatile is
volatile etc.

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
0471453773
storage/lvm: call lvm directly, don't use qubes-lvm wrapper
The wrapper doesn't do anything else than translating command
parameters, but it's load time is significant (because of python imports
mostly). Since we can't use python lvm API from non-root user anyway,
lets drop the wrapper and call `lvm` directly (or through sudo when
necessary).

This makes VM startup much faster - storage preparation is down from
over 10s to about 3s.

QubesOS/qubes-issues#2256
2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
418d749680
backup: use 'scrypt' tool for backup encryption and integrity protection
`openssl dgst` and `openssl enc` used previously poorly handle key
stretching - in case of `openssl enc` encryption key is derived using
single MD5 iteration, without even any salt. This hardly prevent
brute force or even rainbow tables attacks. To make things worse, the
same key is used for encryption and integrity protection which ease
brute force even further.
All this is still about brute force attacks, so when using long, high
entropy passphrase, it should be still relatively safe. But lets do
better.
According to discussion in QubesOS/qubes-issues#971, scrypt algorithm is
a good choice for key stretching (it isn't the best of all existing, but
a good one and widely adopted). At the same time, lets switch away from
`openssl` tool, as it is very limited and apparently not designed for
production use. Use `scrypt` tool, which is very simple and does exactly
what we need - encrypt the data and integrity protect it. Its archive
format have own (simple) header with data required by the `scrypt`
algorithm, including salt. Internally data is encrypted with AES256-CTR
and integrity protected with HMAC-SHA256. For details see:
https://github.com/tarsnap/scrypt/blob/master/FORMAT

This means change of backup format. Mainly:

1. HMAC is stored in scrypt header, so don't use separate file for it.
Instead have data in files with `.enc` extension.
2. For compatibility leave `backup-header` and `backup-header.hmac`. But
`backup-header.hmac` is really scrypt-encrypted version of `backup-header`.
3. For each file, prepend its identifier to the passphrase, to
authenticate filename itself too. Having this we can guard against
reordering archive files within a single backup and across backups. This
identifier is built as:

        backup ID (from backup-header)!filename!

For backup-header itself, there is no backup ID (just 'backup-header!').

Fixes QubesOS/qubes-issues#971
2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki
36eb7f923f
qubes/tarwriter: add simple sparse-tar writer module
tar can't write archive with _contents_ of block device. We need this to
backup LVM-based disk images. To avoid dumping image to a file first,
create a simple tar archiver just for this purpose.

Python is not the fastest possible technology, it's 3 times slower than
equivalent written in C. But it's much easier to read, much less
error-prone, and still process 1GB image under 1s (CPU time, leaving
along actual disk reads). So, it's acceptable.
2016-10-05 01:54:41 +02:00
Marek Marczykowski-Górecki
96a4bb650b
qubes/tools: qvm-backup and qvm-backup-restore tools
Fixes QubesOS/qubes-issues#1213
Fixes QubesOS/qubes-issues#1214
2016-09-21 16:02:50 +02:00
Marek Marczykowski-Górecki
d4f29bb8b7
qubes: add qvm-firewall tool
Fixes QubesOS/qubes-issues#1815
2016-09-19 20:36:32 +02:00
Marek Marczykowski-Górecki
1da75a676f
qubes/firewall: new firewall interface
First part - handling firewall.xml and rules formatting.
Specification on https://qubes-os.org/doc/vm-interface/

TODO (for dom0):
 - plug into QubesVM object
 - expose rules in QubesDB (including reloading)
 - drop old functions (vm.get_firewall_conf etc)

QubesOS/qubes-issues#1815
2016-09-19 20:36:30 +02:00
Marek Marczykowski-Górecki
e8d011b83f
qubes/tools: add qvm-device tool (and tests)
Add a tool to manipulate various devices.

QubesOS/qubes-issues#2257
2016-09-03 20:41:05 +02:00
Marek Marczykowski-Górecki
fbb086aed5
tests: PCI devices tests
Just basic things for now.

QubesOS/qubes-issues#2257
2016-09-03 20:41:04 +02:00
Marek Marczykowski-Górecki
aa67a4512e
qubes/ext/pci: move PCI devices handling to an extension
Implement required event handlers according to documentation in
qubes.devices.

A modification of qubes.devices.DeviceInfo is needed to allow dynamic,
read-only properties.

QubesOS/qubes-issues#2257
2016-09-03 20:41:03 +02:00