Commit Graph

4836 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
ba4dec5383
qvm-block: fix checking attached status of disk image files
Libvirt do not show actual block device (loop*) choosen for the device -
only original (file) path. But file path is available in device
description. Please note that VM can provide any description (withing
allowed limits), effectively breaking this check again (hidding the
attachment status). But even without this bug it could do that - by
hidding the whole device from QubesDB.

Fixes QubesOS/qubes-issues#2453
2016-11-23 03:25:03 +01:00
Jean-Philippe Ouellet
c4f30bc3ad
Don't allow VM name ending in -dm
This is used internally for device-model stubdomains (untrusted qemu)
for HVMs, and causes conflicts when trying to boot foo (HVM) and foo-dm
(any type).

Partially fixes https://github.com/QubesOS/qubes-issues/issues/2422
2016-11-21 22:38:59 -05:00
Marek Marczykowski-Górecki
68a116e8a3
tests: check '$anyvm' parsing in qrexec policy
Regression test for QubesOS/qubes-issues#2031
2016-11-18 03:15:08 +01:00
Wojtek Porczyk
2b0ad51b18 Merge remote-tracking branch 'origin/pull/68/head' into core3-devel 2016-11-15 17:41:47 +01:00
Wojtek Porczyk
37bfd0d2a3 Merge remote-tracking branch 'marmarek/core3-fake-ip' into core3-devel 2016-11-15 17:40:30 +01:00
Wojtek Porczyk
a4d50409df Merge remote-tracking branch 'marmarek/core3-storage-fixes' into core3-devel 2016-11-15 17:36:53 +01:00
Wojtek Porczyk
fd953f4f27 Merge remote-tracking branch 'marmarek/core3-backup2' into core3-devel 2016-11-15 17:34:12 +01:00
Marek Marczykowski-Górecki
85f6ff9ded
Merge remote-tracking branch 'qubesos/pr/47'
* qubesos/pr/47:
  Move /var/run/shm.id to /var/run/qubes/shm.id
2016-11-11 16:13:51 +01:00
Jean-Philippe Ouellet
fadaa64985
Keep Makefile DRY 2016-11-10 06:25:30 -05:00
Bahtiar `kalkin-` Gadimov
5db67fca8d
Fix init property swap in DomainVolumes 2016-11-07 23:26:53 +01:00
Marek Marczykowski-Górecki
b011cef8af
tests/storage: add tests for basic volumes properties
Things like if read-only volume is really read-only, volatile is
volatile etc.

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
b59463e8e8
qvm-block: fix listing non-internal volumes
In case of LVM (at least), "internal" flag is initialized only when
listing volume attached to given VM, but not when listing them from the
pool. This looks like a limitation (bug?) of pool driver, it looks like
much nicer fix is to handle the flag in qvm-block tool (which list VMs
volumes anyway), than in LVM storage pool driver (which would need to
keep second copy of volumes list - just like file driver).

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
1a7f2892d1
storage/lvm: fix logic regarding snapshots, start, stop etc
There are mutiple cases when snapshots are inconsistently created, for
example:
 - "-back" snapshot created from the "new" data, instead of old one
 - "-snap" created even when volume.snap_on_start=False
 - probably more

Fix this by following volume.snap_on_start and volume.save_on_stop
directly, instead of using abstraction of old volume types.

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
ab9d7fbb76
storage: improve/fix handling extra volumes
Just calling pool.init_volume isn't enough - a lot of code depends on
additional data loaded into vm.storage object. Provide a convenient
wrapper for this.

At the same time, fix loading extra volumes from qubes.xml - don't fail
on volume not mentioned in initial vm.volume_config.

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
4323651afb
storage/lvm: remove duplicated _reset function
There were two: _reset and _reset_volume. Neither of them was working,
but the later was closer. Remove the other one.

QubesOS/qubes-issues#2256
2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
37dbf29bc1
storage/lvm: don't fail on removing already removed volumes
This may happen when removing not fully created VM.

QubesOS/qubes-issues#2256
2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
400e92b25a
storage/lvm: misc fixes
- add missing lvm remove call when commiting changes
- delay creating volatile image until domain startup (it will be created
  then anyway)
- reset cache only when really changed anything
- attach VM to the volume (snapshot) created for its runtime - to not
  expose changes (for example in root volume) to child VMs until
  shutdown

QubesOS/qubes-issues#2412
QubesOS/qubes-issues#2256
2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
0471453773
storage/lvm: call lvm directly, don't use qubes-lvm wrapper
The wrapper doesn't do anything else than translating command
parameters, but it's load time is significant (because of python imports
mostly). Since we can't use python lvm API from non-root user anyway,
lets drop the wrapper and call `lvm` directly (or through sudo when
necessary).

This makes VM startup much faster - storage preparation is down from
over 10s to about 3s.

QubesOS/qubes-issues#2256
2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
9197bde76e
storage/lvm: use dd for importing volumes
...instead of manual copy in python. DD is much faster and when used
with `conv=sparse` it will correctly preserve sparse image.

QubesOS/qubes-issues#2256
2016-11-04 14:18:54 +01:00
Marek Marczykowski-Górecki
38fc504ca0
qubes/vm/net: set mapped IP info before attaching network
Set parameters for possibly hiding domain's real IP before attaching
network to it, otherwise we'll have race condition with vif-route-qubes
script.

QubesOS/qubes-issues#1143
2016-11-01 00:37:43 +01:00
Marek Marczykowski-Górecki
b4fa8cdce3
qubes/vm/net: use domain's "visible IP" for a gateway address
This is the IP known to the domain itself and downstream domains. It may
be a different one than seen be its upstream domain.

Related to QubesOS/qubes-issues#1143`
2016-11-01 00:30:11 +01:00
Marek Marczykowski-Górecki
ec81b3046f
tests: add missing app.save() before starting a domain
Otherwise domain will be unknown to other processes (like qrexec
services).
2016-11-01 00:28:37 +01:00
Marek Marczykowski-Górecki
d999d91049
tests: few more tests for fake/custom IP
QubesOS/qubes-issues#1143
QubesOS/qubes-issues#1477
2016-10-31 03:39:46 +01:00
Marek Marczykowski-Górecki
ea33fef9cc
tests: drop dispvm_netvm tests
This property no longer exists in core3.
2016-10-31 03:10:12 +01:00
Marek Marczykowski-Górecki
5072acc8f2
tests: custom VM IP
QubesOS/qubes-issues#1477
2016-10-31 03:09:45 +01:00
Marek Marczykowski-Górecki
b8145595a9
qubes/vm/net: allow setting custom IP
Fixes QubesOS/qubes-issues#1477
2016-10-31 03:04:13 +01:00
Marek Marczykowski-Górecki
4585f2b503
tests: add tests for fake IP feature
QubesOS/qubes-issues#1143
2016-10-31 02:17:21 +01:00
Marek Marczykowski-Górecki
2c6c476410
qubes/vm/net: add feature of hiding real IP from the VM
This helps hiding VM IP for anonymous VMs (Whonix) even when some
application leak it. VM will know only some fake IP, which should be set
to something as common as possible.
The feature is mostly implemented at (Proxy)VM side using NAT in
separate network namespace. Core here is only passing arguments to it.
It is designed the way that multiple VMs can use the same IP and still
do not interfere with each other. Even more: it is possible to address
each of them (using their "native" IP), even when multiple of them share
the same "fake" IP.

Original approach (marmarek/old-qubes-core-admin#2) used network script
arguments by appending them to script name, but libxl in Xen >= 4.6
fixed that side effect and it isn't possible anymore. So use QubesDB
instead.

From user POV, this adds 3 "features":
 - net/fake-ip - IP address visible in the VM
 - net/fake-gateway - default gateway in the VM
 - net/fake-netmask - network mask
The feature is enabled if net/fake-ip is set (to some IP address) and is
different than VM native IP. All of those "features" can be set on
template, to affect all of VMs.
Firewall rules etc in (Proxy)VM should still be applied to VM "native"
IP.

Fixes QubesOS/qubes-issues#1143
2016-10-31 02:06:01 +01:00
Marek Marczykowski-Górecki
b91714b204
qubes/features: handle recursive templates
Have features.check_with_template() check the template recursively.
The longest path (currently) is: DispVM -> AppVM -> TemplateVM.
2016-10-31 02:04:27 +01:00
Marek Marczykowski-Górecki
bd35a1061b
version 3.2.12 2016-10-30 02:28:41 +02:00
Marek Marczykowski-Górecki
800fcfd880
Do not start guid if DISPLAY is not set
Fixes QubesOS/qubes-issues#2408
2016-10-30 01:31:53 +02:00
Marek Marczykowski-Górecki
3b680ebc94
Merge remote-tracking branch 'qubesos/pr/63'
* qubesos/pr/63:
  Better order for plymouth disabling stanza, as well as modern initrd plymouth disabling.
2016-10-28 23:52:01 +02:00
Rudd-O
63fd200abf Better order for plymouth disabling stanza, as well as modern initrd plymouth disabling. 2016-10-28 20:53:49 +00:00
Wojtek Porczyk
0141e1ac73 qubes/app: Allow keeping lock after load
QubesOS/qubes-issues#1729
2016-10-28 15:43:43 +02:00
Marek Marczykowski-Górecki
8c270d8061
vm-config: drop '3' from kernel command line
We set default systemd target anyway, so it isn't needed. But it breaks
Fedora recommended system upgrade path.

Fixes QubesOS/qubes-issues#2404
2016-10-28 13:53:24 +02:00
Marek Marczykowski-Górecki
8cf19e3c92
tests/backupcompatibility: verify restored VM properties 2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
36bd834c01
core2migration: try to set properties to "default" when possible
Core3 keep information whether property have default value for all the
properties (not only few like netvm or kernel). Try to use this feature
as much as possible.
2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
64ac7f6e8d
tests/backup: check non-ASCII passphrase
QubesOS/qubes-issues#2398
2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
af182c4fd1
backup: fixup restore options just before restoring VMs
When user included/excluded some VMs for restoration, it may be
neceesarry to fix dependencies between them (for example when default
template is no longer going to be restored).
Also fix handling conflicting names.
2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
043d20c05d
backup: fix handling non-ascii characters in backup passphrase
Fixes QubesOS/qubes-issues#2398
2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki
fc00dd211e
tests/backup: test backup with non-ASCII passphrase 2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki
49e718cf57
backup: mark 'encryption' option as deprecated - all backups are encrypted
QubesOS/qubes-issues#971
2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki
51b66208f3
backup: verify if archive chunks are not reordered
Now, when file name is also integrity protected (prefixed to the
passphrase), we can make sure that input files are given in the same
order. And are parts of the same VM.

QubesOS/qubes-issues#971
2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki
4ad15c082b
backup: add 'backup_id' to integrity protection
This prevent switching parts of backup of the same VM between different
backups made by the same user (or actually: with the same passphrase).

QubesOS/qubes-issues#971
2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki
418d749680
backup: use 'scrypt' tool for backup encryption and integrity protection
`openssl dgst` and `openssl enc` used previously poorly handle key
stretching - in case of `openssl enc` encryption key is derived using
single MD5 iteration, without even any salt. This hardly prevent
brute force or even rainbow tables attacks. To make things worse, the
same key is used for encryption and integrity protection which ease
brute force even further.
All this is still about brute force attacks, so when using long, high
entropy passphrase, it should be still relatively safe. But lets do
better.
According to discussion in QubesOS/qubes-issues#971, scrypt algorithm is
a good choice for key stretching (it isn't the best of all existing, but
a good one and widely adopted). At the same time, lets switch away from
`openssl` tool, as it is very limited and apparently not designed for
production use. Use `scrypt` tool, which is very simple and does exactly
what we need - encrypt the data and integrity protect it. Its archive
format have own (simple) header with data required by the `scrypt`
algorithm, including salt. Internally data is encrypted with AES256-CTR
and integrity protected with HMAC-SHA256. For details see:
https://github.com/tarsnap/scrypt/blob/master/FORMAT

This means change of backup format. Mainly:

1. HMAC is stored in scrypt header, so don't use separate file for it.
Instead have data in files with `.enc` extension.
2. For compatibility leave `backup-header` and `backup-header.hmac`. But
`backup-header.hmac` is really scrypt-encrypted version of `backup-header`.
3. For each file, prepend its identifier to the passphrase, to
authenticate filename itself too. Having this we can guard against
reordering archive files within a single backup and across backups. This
identifier is built as:

        backup ID (from backup-header)!filename!

For backup-header itself, there is no backup ID (just 'backup-header!').

Fixes QubesOS/qubes-issues#971
2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki
d7c355eadb
backup: make wait_backup_feedback/handle_streams less ugly
Have a generic function `handle_streams`, instead of
`wait_backup_feedback` with open coded process names and manual
iteration over them.

No functional change, besides minor logging change.
2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki
6ee200236c
tests/backup: verify migration into LVM thin pool 2016-10-28 11:53:31 +02:00
Marek Marczykowski-Górecki
673fe4423a
tests: handle LVM thin pool 2016-10-28 11:53:31 +02:00
Marek Marczykowski-Górecki
fbecd08a58
tests/backup: exclude some VMs during restore 2016-10-28 11:53:31 +02:00
Wojtek Porczyk
3553b2e1d4 Make pylint happy 2016-10-25 17:27:02 +02:00