Commit Graph

769 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
3b209515c2
qubes/vm/dispvm: don't crash when DispVM is already killed
This is regression of QubesOS/qubes-issues#1660

Fixes QubesOS/qubes-issues#1660
2016-11-26 04:09:00 +01:00
Marek Marczykowski-Górecki
6ff836dfa4
qubes/log: add FD_CLOEXEC to log files
Don't leak log file descriptors. At least 'lvm' complains.

QubesOS/qubes-issues#2412
2016-11-26 04:08:59 +01:00
Marek Marczykowski-Górecki
210cb65d1c
qubes/tools: drop requirement of qubes-prefs --force-root
None of properties set there do any "dangerous thing" for filesystem
permissions (at least for now), so do not require it. This is mostly to
keep compatibility with %post rpm scripts (kernel-qubes-vm at least).

QubesOS/qubes-issues#2412
2016-11-26 04:08:59 +01:00
Marek Marczykowski-Górecki
a318d5cea9
Don't fail on DBus connection error or opening log
Especially in offline mode - like during installation, tests etc.

QubesOS/qubes-issues#2412
2016-11-26 04:08:59 +01:00
Marek Marczykowski-Górecki
02a0713665
qubes/tools: better handle qvm-template-postprocess called as root
This tool by design is called as root, so try to:
 - switch to normal user if possible
 - fix file permissions afterwards - if not

QubesOS/qubes-issues#2412
2016-11-26 04:08:58 +01:00
Marek Marczykowski-Górecki
28d4feb0d0
qubes: fix network-related functions
- cleanup_vifs crash when non-networked VM is running
- type error in get_vms_connected_to (store VM objects, not qid)
2016-11-26 04:08:58 +01:00
Marek Marczykowski-Górecki
e85b0663f6
tools: fire 'template-postinstall' event for extensions
Allow extensions to finish template setup. This include retrieving
appmenus, settings defaults etc.

QubesOS/qubes-issues#2412
2016-11-26 04:08:58 +01:00
Marek Marczykowski-Górecki
0041063b8e
qubes/config: set default labels
There was a comment '# Set later', but actually values were never set.
This break adding just installed template (qvm-template-postprocess).

QubesOS/qubes-issues#2412
2016-11-26 04:08:57 +01:00
Marek Marczykowski-Górecki
c4e85a81fb
qubes/app: automatically enable offline mode when running in chroot
Do not spray --offline-mode over every installer-related script.

QubesOS/qubes-issues#2412
2016-11-26 04:08:50 +01:00
Marek Marczykowski-Górecki
d8a1216daf
Fix qubes-create in offline mode
QubesOS/qubes-issues#2412
2016-11-26 04:08:09 +01:00
Marek Marczykowski-Górecki
5e15db4176
qubes/tools: accept qvm-start --no-guid
Lets keep compatibility with older scripts.

QubesOS/qubes-issues#2412
2016-11-26 04:08:08 +01:00
Marek Marczykowski-Górecki
964955758c
qubes/app: create 'default' storage pool as LVM when present
When system is installed with LVM thin pool, it should be used by
default. But lets keep file-based on for /var/lib/qubes for some corner
cases, migration etc.

QubesOS/qubes-issues#2412
2016-11-26 04:08:08 +01:00
Marek Marczykowski-Górecki
badc58837a
Add qvm-template-postprocess tool
This is intended to call to finish template installation/removal.
Template RPM package is basically container for root.img, nothing more.
Other parts needs to be generated after root.img extraction. Previously
it was open coded in rpm post-install script, but lets keep it as qvm
tool to ease supporting multiple version in template builder

QubesOS/qubes-issues#2412
2016-11-26 04:08:08 +01:00
Marek Marczykowski-Górecki
1418555346
qubes/vm: don't fail on removing VM without files
VM files may be already removed. Don't fail on this while removing a
VM, it's probably the reason why domain is being removed.

qvm-remove tool have its own guard for this, but it isn't enough - if
rmtree(dir_path) fails, storage.remove() would not be called, so
non-file storages would not be cleaned up.

This is also needed to correctly handle template reinstallation - where
VM directory is moved away to call create_on_disk again.

QubesOS/qubes-issues#2412
2016-11-26 04:08:07 +01:00
Marek Marczykowski-Górecki
cc440c62f6
qubes/tools: accept properties with '-'
'-' is invalid character in python identifier, so all the properties
have '_'. But in previous versions qvm-* tools accepted names with '-',
so lets not break this.

QubesOS/qubes-issues#2412
2016-11-26 04:08:07 +01:00
Marek Marczykowski-Górecki
880566a387
qubes/tools: do not reject --set/--get in *-prefs tools
Those options are no longer needed, but lets not reject them to preserve
compatibility with older scripts

QubesOS/qubes-issues#2412
2016-11-26 04:08:07 +01:00
Marek Marczykowski-Górecki
91727389c4
qubes/log: ensure logs are group writable
/var/log/qubes directory have setgid set, so all the files will be owned
by qubes group (that's ok), but there is no enforcement of creating it
group writable, which undermine group ownership (logs created by root
would not be writable by normal user)

QubesOS/qubes-issues#2412
2016-11-26 04:08:07 +01:00
Marek Marczykowski-Górecki
80c0093c77
qubes/tools/qubes-create: reject overriding existing qubes.xml
If someone really want do to this, need to manually remove the file
first.

QubesOS/qubes-issues#2412
2016-11-26 04:08:06 +01:00
Marek Marczykowski-Górecki
c08766e157
qubes/features: rename 'services/ntpd' to 'service/ntpd'
It makes much more sense to use singular form here - ntpd is a single
service.
2016-11-26 04:08:06 +01:00
Wojtek Porczyk
2b0ad51b18 Merge remote-tracking branch 'origin/pull/68/head' into core3-devel 2016-11-15 17:41:47 +01:00
Wojtek Porczyk
37bfd0d2a3 Merge remote-tracking branch 'marmarek/core3-fake-ip' into core3-devel 2016-11-15 17:40:30 +01:00
Wojtek Porczyk
a4d50409df Merge remote-tracking branch 'marmarek/core3-storage-fixes' into core3-devel 2016-11-15 17:36:53 +01:00
Wojtek Porczyk
fd953f4f27 Merge remote-tracking branch 'marmarek/core3-backup2' into core3-devel 2016-11-15 17:34:12 +01:00
Bahtiar `kalkin-` Gadimov
5db67fca8d
Fix init property swap in DomainVolumes 2016-11-07 23:26:53 +01:00
Marek Marczykowski-Górecki
b011cef8af
tests/storage: add tests for basic volumes properties
Things like if read-only volume is really read-only, volatile is
volatile etc.

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
b59463e8e8
qvm-block: fix listing non-internal volumes
In case of LVM (at least), "internal" flag is initialized only when
listing volume attached to given VM, but not when listing them from the
pool. This looks like a limitation (bug?) of pool driver, it looks like
much nicer fix is to handle the flag in qvm-block tool (which list VMs
volumes anyway), than in LVM storage pool driver (which would need to
keep second copy of volumes list - just like file driver).

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
1a7f2892d1
storage/lvm: fix logic regarding snapshots, start, stop etc
There are mutiple cases when snapshots are inconsistently created, for
example:
 - "-back" snapshot created from the "new" data, instead of old one
 - "-snap" created even when volume.snap_on_start=False
 - probably more

Fix this by following volume.snap_on_start and volume.save_on_stop
directly, instead of using abstraction of old volume types.

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
ab9d7fbb76
storage: improve/fix handling extra volumes
Just calling pool.init_volume isn't enough - a lot of code depends on
additional data loaded into vm.storage object. Provide a convenient
wrapper for this.

At the same time, fix loading extra volumes from qubes.xml - don't fail
on volume not mentioned in initial vm.volume_config.

QubesOS/qubes-issues#2256
2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki
4323651afb
storage/lvm: remove duplicated _reset function
There were two: _reset and _reset_volume. Neither of them was working,
but the later was closer. Remove the other one.

QubesOS/qubes-issues#2256
2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
37dbf29bc1
storage/lvm: don't fail on removing already removed volumes
This may happen when removing not fully created VM.

QubesOS/qubes-issues#2256
2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
400e92b25a
storage/lvm: misc fixes
- add missing lvm remove call when commiting changes
- delay creating volatile image until domain startup (it will be created
  then anyway)
- reset cache only when really changed anything
- attach VM to the volume (snapshot) created for its runtime - to not
  expose changes (for example in root volume) to child VMs until
  shutdown

QubesOS/qubes-issues#2412
QubesOS/qubes-issues#2256
2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
0471453773
storage/lvm: call lvm directly, don't use qubes-lvm wrapper
The wrapper doesn't do anything else than translating command
parameters, but it's load time is significant (because of python imports
mostly). Since we can't use python lvm API from non-root user anyway,
lets drop the wrapper and call `lvm` directly (or through sudo when
necessary).

This makes VM startup much faster - storage preparation is down from
over 10s to about 3s.

QubesOS/qubes-issues#2256
2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki
9197bde76e
storage/lvm: use dd for importing volumes
...instead of manual copy in python. DD is much faster and when used
with `conv=sparse` it will correctly preserve sparse image.

QubesOS/qubes-issues#2256
2016-11-04 14:18:54 +01:00
Marek Marczykowski-Górecki
38fc504ca0
qubes/vm/net: set mapped IP info before attaching network
Set parameters for possibly hiding domain's real IP before attaching
network to it, otherwise we'll have race condition with vif-route-qubes
script.

QubesOS/qubes-issues#1143
2016-11-01 00:37:43 +01:00
Marek Marczykowski-Górecki
b4fa8cdce3
qubes/vm/net: use domain's "visible IP" for a gateway address
This is the IP known to the domain itself and downstream domains. It may
be a different one than seen be its upstream domain.

Related to QubesOS/qubes-issues#1143`
2016-11-01 00:30:11 +01:00
Marek Marczykowski-Górecki
ec81b3046f
tests: add missing app.save() before starting a domain
Otherwise domain will be unknown to other processes (like qrexec
services).
2016-11-01 00:28:37 +01:00
Marek Marczykowski-Górecki
d999d91049
tests: few more tests for fake/custom IP
QubesOS/qubes-issues#1143
QubesOS/qubes-issues#1477
2016-10-31 03:39:46 +01:00
Marek Marczykowski-Górecki
ea33fef9cc
tests: drop dispvm_netvm tests
This property no longer exists in core3.
2016-10-31 03:10:12 +01:00
Marek Marczykowski-Górecki
5072acc8f2
tests: custom VM IP
QubesOS/qubes-issues#1477
2016-10-31 03:09:45 +01:00
Marek Marczykowski-Górecki
b8145595a9
qubes/vm/net: allow setting custom IP
Fixes QubesOS/qubes-issues#1477
2016-10-31 03:04:13 +01:00
Marek Marczykowski-Górecki
4585f2b503
tests: add tests for fake IP feature
QubesOS/qubes-issues#1143
2016-10-31 02:17:21 +01:00
Marek Marczykowski-Górecki
2c6c476410
qubes/vm/net: add feature of hiding real IP from the VM
This helps hiding VM IP for anonymous VMs (Whonix) even when some
application leak it. VM will know only some fake IP, which should be set
to something as common as possible.
The feature is mostly implemented at (Proxy)VM side using NAT in
separate network namespace. Core here is only passing arguments to it.
It is designed the way that multiple VMs can use the same IP and still
do not interfere with each other. Even more: it is possible to address
each of them (using their "native" IP), even when multiple of them share
the same "fake" IP.

Original approach (marmarek/old-qubes-core-admin#2) used network script
arguments by appending them to script name, but libxl in Xen >= 4.6
fixed that side effect and it isn't possible anymore. So use QubesDB
instead.

From user POV, this adds 3 "features":
 - net/fake-ip - IP address visible in the VM
 - net/fake-gateway - default gateway in the VM
 - net/fake-netmask - network mask
The feature is enabled if net/fake-ip is set (to some IP address) and is
different than VM native IP. All of those "features" can be set on
template, to affect all of VMs.
Firewall rules etc in (Proxy)VM should still be applied to VM "native"
IP.

Fixes QubesOS/qubes-issues#1143
2016-10-31 02:06:01 +01:00
Marek Marczykowski-Górecki
b91714b204
qubes/features: handle recursive templates
Have features.check_with_template() check the template recursively.
The longest path (currently) is: DispVM -> AppVM -> TemplateVM.
2016-10-31 02:04:27 +01:00
Wojtek Porczyk
0141e1ac73 qubes/app: Allow keeping lock after load
QubesOS/qubes-issues#1729
2016-10-28 15:43:43 +02:00
Marek Marczykowski-Górecki
8cf19e3c92
tests/backupcompatibility: verify restored VM properties 2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
36bd834c01
core2migration: try to set properties to "default" when possible
Core3 keep information whether property have default value for all the
properties (not only few like netvm or kernel). Try to use this feature
as much as possible.
2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
64ac7f6e8d
tests/backup: check non-ASCII passphrase
QubesOS/qubes-issues#2398
2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
af182c4fd1
backup: fixup restore options just before restoring VMs
When user included/excluded some VMs for restoration, it may be
neceesarry to fix dependencies between them (for example when default
template is no longer going to be restored).
Also fix handling conflicting names.
2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki
043d20c05d
backup: fix handling non-ascii characters in backup passphrase
Fixes QubesOS/qubes-issues#2398
2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki
fc00dd211e
tests/backup: test backup with non-ASCII passphrase 2016-10-28 11:53:33 +02:00