Commit Graph

211 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
adfc4e0ac9 core: disks handling cleanup, fix them for TemplateHVM
Move rootcow_img to storage class, remove clean_volatile_img. And most
importantly - set source_template in QubesHVm.create_on_disk.
2015-02-09 06:02:20 +01:00
Marek Marczykowski-Górecki
2def43517a core/hvm: handle verbose option for guid 2015-02-09 05:39:44 +01:00
Marek Marczykowski-Górecki
393bb00471 core: update reporting for missing VT-x 2015-02-09 03:46:53 +01:00
Marek Marczykowski-Górecki
869675c15c core: convert memory/cpu stats to libvirt API 2015-02-09 03:28:01 +01:00
Marek Marczykowski-Górecki
4e26588bb3 core/hvm: remove xenstore code
QubesDB does not require setting up directory (and permissions), so just
remove the function.
2015-02-07 01:12:29 +01:00
Marek Marczykowski-Górecki
89f8f219bf core: changes in libvirt config for libvirt-1.2.12 2015-02-05 06:31:00 +01:00
Marek Marczykowski-Górecki
72cf3a8201 core: remove kernel properties from DispVM and Dom0 (#948)
Qubes does not keep track of those kernel versions.

Conflicts:
	core-modules/01QubesDisposableVm.py
2015-01-30 01:40:40 +01:00
Marek Marczykowski-Górecki
49d510dc65 core: prevent permissions error when VM was started by root
When VM is started by root, config file is created with root owner and
user has no write access to it. As the directory is user-writable,
delete the file first.

Conflicts:
	core-modules/000QubesVm.py
2015-01-30 01:39:57 +01:00
Marek Marczykowski-Górecki
52334bc414 core: fix firewall update code
Do not load qubes.xml again, it can cause race conditions between two
instances of the same VM objects.
Especially when VM is starting ProxyVM to which it is connected,
firewall rules could not be loaded.
2015-01-30 01:38:56 +01:00
Marek Marczykowski-Górecki
73301a67c8 core: fix vm.run(..., passio=False) handling
Long time ago passio=True was used to replace current process with
qrexec-client directly (qvm-run --pass-io was the called), but this
behaviour is not used anymore (qvm-run was the only user). And this
option was left untouched, with misleading name - one would assume that
using passio=False should disallow any I/O, but this isn't the case.

Especially qvm-sync-clock is calling clockvm.run('...', wait=True),
default value for passio=False. This causes to output data from
untrusted VM, without sanitising terminal sequences, which can be fatal.

This patch changes passio semantic to actually do what it means - when
set to True - VM process will be able to interact with
stdin/stdout/stderr. But when set to False, all those FDs will be
connected to /dev/null.

Conflicts:
	core-modules/000QubesVm.py
2015-01-30 01:38:52 +01:00
Marek Marczykowski-Górecki
7a3bce6c61 core: fix is_paused method 2014-11-29 02:58:47 +01:00
Marek Marczykowski-Górecki
592a4901c9 core: import monitorlayoutnotify instead of calling it as external script
Otherwise deadlock could happen - the script will try to get read lock
on qubes.xml, while the calling tool can already hold the lock. If that
was write lock (which is in case of qfile-daemon-dvm), the deadlock
occurs.
2014-11-21 21:45:03 +01:00
Marek Marczykowski-Górecki
1df73d31c6 core: xid is no longer local variable here 2014-11-19 12:50:32 +01:00
Marek Marczykowski-Górecki
9205c5c054 core: fix imports 2014-11-19 12:50:32 +01:00
Marek Marczykowski-Górecki
479ac1e42d core: check libvirt error on specific connection
Not global last one.
2014-11-19 12:50:32 +01:00
Rafał Wojdyła
97c793ed16 QubesVm.run(): wait for client to exit on Windows 2014-11-19 12:50:31 +01:00
Marek Marczykowski-Górecki
3ba424e6ac Use VM name as argument to qrexec-client
This is the only place where ID was used - all other places uses name.
Linux qrexec-client accepts both ID and name, but sticking to one option
will simplify things (especially Windows qrexec-client/daemon).
2014-11-19 12:50:31 +01:00
Marek Marczykowski-Górecki
def58ab911 core: typo fix 2014-11-19 12:50:30 +01:00
Marek Marczykowski-Górecki
687e004b1d windows: workaround for windows "behavior" regarding parsing exec() arguments 2014-11-19 12:50:30 +01:00
Marek Marczykowski-Górecki
27b031c59f Check for None before calling method (VM rename fix) 2014-11-19 12:50:29 +01:00
Marek Marczykowski-Górecki
c3d9b1971a Handle the case when libvirt object doesn't exists for given VM
This can be some "virtual" VM (like dom0).
2014-11-19 12:50:29 +01:00
Marek Marczykowski-Górecki
5763beb898 HVM: do not fail on non-Xen systems (without xenstore)
Currently getting Stubdom XID is (the last one?) read directly from
Xenstore as there is no libvirt function for it.
This means that even if HVM is running it can have not connection to
Xenstore. For now give -1 in such situation.
2014-11-19 12:50:29 +01:00
Marek Marczykowski-Górecki
7e355c5dad core: ignore error when PCI device already "detached"
This can be the case at startup, when all network devices are connected
to pciback module by initramfs.
2014-11-19 12:50:29 +01:00
Marek Marczykowski-Górecki
11047bf427 Use platform specific locking method
None of found existing portable locking module does support RW locks.
Use lowlevel system locking support - both Windows and Linux support
such feature.

Drop locking code in write_firewall_conf() b/c is is called with
QubesVmCollection lock held anyway.
2014-11-19 12:50:28 +01:00
Marek Marczykowski-Górecki
392b70a4d8 Create missing private.img when needed
Not only for HVM, but any VM type which support private.img.
2014-11-19 12:50:28 +01:00
Marek Marczykowski-Górecki
21c908b9b2 Move storage-related VM rename code to storage class 2014-11-19 12:50:28 +01:00
Marek Marczykowski-Górecki
396011e227 Fix initialization of storage class
Move it earlier - before first access to *_img attrs.
2014-11-19 12:50:27 +01:00
Marek Marczykowski-Górecki
7b5dd67b58 Completely move *_img attrs to QubesVmStorage class 2014-11-19 12:50:27 +01:00
Marek Marczykowski-Górecki
2457a705d0 Load Qubes DB python module only when needed.
This is temporary band aid for the python module problems.
This should also slightly speed up some tools (not all uses Qubes DB).
2014-11-19 12:50:27 +01:00
Marek Marczykowski-Górecki
a17f6ef779 Update QubesHVM to use QubesVmStorage classes
Also add external drive support to QubesXenVmStorage (move from
QubesHVM).
2014-11-19 12:50:27 +01:00
Marek Marczykowski-Górecki
66b39ac2a2 Typo fix in error message. 2014-11-19 12:50:26 +01:00
Marek Marczykowski-Górecki
4ae720956d Use file copy instead of symlink on Windows 2014-11-19 12:50:26 +01:00
Marek Marczykowski-Górecki
d91162e74d minor comment fix 2014-11-19 12:50:26 +01:00
Marek Marczykowski-Górecki
0a1f3d0a44 core: split VM images handling to separate class
This will ease handling different types of VMM (which can require
different image types, location etc).
2014-11-19 12:50:25 +01:00
Marek Marczykowski-Górecki
69d1ae645f Do not use qmemman when not present (installed) on particular VMM 2014-11-19 12:50:25 +01:00
Marek Marczykowski-Górecki
0009805041 rpm+makefile: move build/install code to Makefile files
This makes build "scripts" not tied to Fedora-specific files. Especially
ease porting to other platforms.
2014-11-19 12:50:24 +01:00
Marek Marczykowski
fb353ea615 core: fix regression in iptables rule generator
This commit:
eb11a2f Use QubesDB instead of Xenstore.
introduced regression in iptables rules generator (sed misuse...).
2014-11-19 12:50:24 +01:00
Marek Marczykowski
ff2c925e18 dispvm: use qubesdb instead of xenstore 2014-11-19 12:50:24 +01:00
Marek Marczykowski
457737b6cc QubesVm: ignore errors during debug VM config file creation
Currently <vm-dir>/<vm-name>.conf file is used only for debugging
purposes - the real one is passed directly to libvirt, without storing
on disk for it.
In some cases (e.g. qvm-clone) QubesVM.create_config_file() can be
called before VM directory exists and in this case it would fail.
Because it isn't critical fail in any means (the config file will be
recreated on next occasion) just ignore this error.

Final version most likely will have this part of code removed
completely.
2014-11-19 12:48:28 +01:00
Marek Marczykowski
f159f3e168 Use QubesDB instead of Xenstore.
Mostly done. Things still using xenstore/not working at all:
 - DispVM
 - qubesutils.py (especially qvm-block and qvm-usb code)
 - external IP change notification for ProxyVM (should be done via RPC
   service)
2014-11-19 12:48:28 +01:00
Marek Marczykowski
3f6947b87e dispvm: fix domain ID handling
Do not use obsoleted get_xid() function.
2014-11-19 12:48:27 +01:00
Marek Marczykowski
9393b5f931 Fix domain rename
libvirt_domain object needs to be recreated, so force it. Also fix
config path setting (missing extension) - create_config_file
uses it as custom config indicator (if such detected, VM settings -
especially name, would not be updated).
2014-11-19 12:48:27 +01:00
Marek Marczykowski
062c3afcf5 Remove unused 'source_template' parameter from some QubesVm internal functions 2014-11-19 12:48:27 +01:00
Marek Marczykowski
25ee58217d Set umask before possibly creating new files - make them group-writable 2014-11-19 12:48:27 +01:00
Marek Marczykowski
b3af858502 Indentation fix - check for custom config file 2014-11-19 12:48:27 +01:00
Marek Marczykowski
5a28074c2b qvm-template-commit: add --offline-mode option
Allow force offline mode (disable check if VM isn't running) - useful
for running from anaconda (inside of chroot).
2014-11-19 12:48:27 +01:00
Marek Marczykowski
8e9c59e7f8 Minor changes to make VMM connection really lazy
1. Fake dom0 object doesn't need proper maxmem nor vcpus - set
statically to 0 instead of getting from physical host.
2. QubesHVM doesn't preserve maxmem setting, so set it to self.memory
earlier (to suppress default total_memory/2 calculation).
2014-11-19 12:48:27 +01:00
Marek Marczykowski
b8c62c0279 Wrap all VMM connection related object into QubesVMMConnection class
This makes easier to import right objects in submodules (only one
object). This also implement lazy connection - at first access, not at
module import, which speeds up tools, which doesn't need runtime
information (like qvm-prefs or qvm-service). In the future this will
ease migration from xenstore to QubesDB.

Also implement "offline mode" - operate on qubes.xml without connecting
to VMM - raise exception at such try.
This is needed to run tools during installation, where only minimal
set of services are started, especially no libvirt.
2014-11-19 12:48:26 +01:00
Marek Marczykowski
201cd509e1 QubesDom0NetVm: provide get_mem*
As libvirt doesn't keep dom0 domain object, so add special cases to get
memory information.
2014-11-19 12:48:26 +01:00
Marek Marczykowski
550d8cac68 Create permanent libvirt domain objects
Do not recreate them at each startup. This will save some time and also
solve some problems from invalidated libvirt handles after domain
shutdown (e.g. causes qubes-manager crashes).
This requires storing uuid in qubes.xml.
2014-11-19 12:48:26 +01:00
Marek Marczykowski
1fda0502ab Check for invalid XID 2014-11-19 12:48:26 +01:00
Marek Marczykowski
895ca0633d Fix current memory reporting 2014-11-19 12:48:26 +01:00
Marek Marczykowski
4f090cdb5e pass VM name to guid via cmdline 2014-11-19 12:48:26 +01:00
Marek Marczykowski
107ebad9d5 Migration to libvirt - DispVM
Move DispVM creation to qfile-daemon-dvm/QubesDisposableVm from
qubes-restore. As actual restore is handled by libvirt, we don't get
much from separate qubes-restore process.
This code still needs some improvements, especially on performance.
2014-11-19 12:48:26 +01:00
Marek Marczykowski
f44dc40858 Migration to libvirt - HVM 2014-11-19 12:47:00 +01:00
Marek Marczykowski
f3a7d5f6e6 Remove debug_console QubesVm.start() argument
It isn't working for a long time, so remove it to not confuse reader.
2014-11-19 12:47:00 +01:00
Marek Marczykowski
a880483092 Migration to libvirt - core part
Still not all code migrated, added appropriate TODO/FIXME comments.
2014-11-19 12:47:00 +01:00
Marek Marczykowski-Górecki
aa958eeff6 core: really quiet mode if vm.start()
Pass '-q' option to gui-daemon and qrexec-daemon.
2014-10-25 01:45:01 +02:00
Victor Lopez
bf2173073b Linux VM specific: maxmem limited by init mem
Check maxmem taking into account the minimum init memory that allows
that requested maximum memory.

Explanation:
Linux kernel needs space for memory-related structures created at boot.
If init_mem is just 400MB, then max_mem can't balloon above 4.3GB (at
which poing it yields "add_memory() failed: -17" messages and apps
crash), regardless of the max_mem_size value.

Based on Marek's findings and my tests on a 16GB PC, using several
processes like:
   stress -m 1 --vm-bytes 1g --vm-hang 100

result in the following points:
init_mem  ==> actual max memory
400             4300
700             7554
800             8635
1024            11051
1200            12954
1300            14038
1500            14045 <== probably capped on my 16GB system

The actual ratio of max_mem_size/init_mem is surprisingly constant at
10.79

If less init memory is set than that ratio allows, then the set
maxmem is unreachable and the VM becomes unstable (app crashes)

Based on qubes-devel discussion titled "Qubes Dom0 init memory against
Xen best practices?" at:
https://groups.google.com/d/msg/qubes-devel/VRqkFj1IOtA/UgMgnwfxVSIJ
2014-10-06 04:02:07 +02:00
Marek Marczykowski-Górecki
ed15e15f50 core: make create_on_disk verbose parameter optional 2014-10-01 03:50:55 +02:00
Marek Marczykowski-Górecki
9ad4e15c66 core: inherit default_user from template 2014-09-16 01:46:41 +02:00
Marek Marczykowski-Górecki
17cf9f7a9b Remove duplicated line 2014-09-16 01:33:40 +02:00
Marek Marczykowski-Górecki
355ed640b2 hvm: reserve memory for stubdom
Previously it was allocated from 50MB memory margin left by qmemman.
Which wasn't reliable enough.
2014-09-16 01:18:55 +02:00
Wojciech Zygmunt Porczyk
d7958625c6 core+modules: provide meaingful repr()s for some classes 2014-08-11 16:34:33 +02:00
Marek Marczykowski-Górecki
279dc78a02 core: do not reject already existing VMs with invalid name
It would break the system when the rules for names change. Warn instead.
2014-07-28 01:07:02 +02:00
Marek Marczykowski-Górecki
98540ac513 core: reject VM names longer than 31 chars 2014-07-28 01:00:51 +02:00
Wojciech Zygmunt Porczyk
b2162b1af9 Forbid renaming of RPM-borne VMs (mainly templates) (#839)
User can still clone VM and remove old via rpm/yum.
2014-06-26 13:55:35 +02:00
Marek Marczykowski-Górecki
44f38fe076 Declare file encoding for all python files, fill missing copyright headers
Without that, python do not accept UTF-8 even in comments.
2014-05-18 21:03:27 +02:00
Marek Marczykowski-Górecki
6566d4f57c core: allow clone_attrs() to not fail on errors
... and continue setting other properties. Currently not used, but can
be considered for backup-restore (to restore as much properties as
possible).
2014-05-12 19:47:48 +02:00
Marek Marczykowski-Górecki
86bae8d276 core: verify kernel property when its modified
Do not assume that frontend have verified that setting.
2014-05-12 19:45:30 +02:00
Marek Marczykowski-Górecki
e5feb3169a core: cleanup remains of failed domain startup (#825) 2014-05-10 21:23:04 +02:00
Wojciech Zygmunt Porczyk
8ef5f4c665 QubesTemplateVm: fire post-rename hooks 2014-05-07 14:15:15 +02:00
Wojciech Zygmunt Porczyk
a63a8069b1 QubesVm: setter hook passes old value
This is part of #827 bug.
2014-05-05 22:23:43 +02:00
Wojciech Zygmunt Porczyk
72f74b6b76 QubesVm: add attribute setter hook mechanism
This is part of #827 bug.
2014-05-05 16:40:44 +02:00
Marek Marczykowski-Górecki
dfb6b384cb core: fix typo in firewall.xml root element 2014-05-05 05:22:25 +02:00
Marek Marczykowski-Górecki
14f3094036 core: fix resize of root.img for HVM template (#837) 2014-04-29 02:13:16 +02:00
Marek Marczykowski-Górecki
8ac5981fdd Merge remote-tracking branch 'woju/bug-822' 2014-04-28 14:31:29 +02:00
Wojciech Zygmunt Porczyk
8c26f19103 core-modules: remove duplicate get_disk_usage 2014-04-28 13:33:18 +02:00
Wojciech Zygmunt Porczyk
ee059df63e qubesutils.py: get_disk_usage
get_disk_usage is moved from code-modules and backup.py and rewritten not to
call external du

This commit is part of #822 fix.
2014-04-24 21:50:12 +02:00
Marek Marczykowski-Górecki
cac866ed1b core: not not fail network reconnect on HVM without qrexec installed 2014-04-24 12:57:41 +02:00
Marek Marczykowski-Górecki
047cbb3443 core: fix tempate HVM creation 2014-04-23 04:17:40 +02:00
Marek Marczykowski-Górecki
113fe3391d core: fix vm.is_outdated() for template-based HVM
HVM template can't be started while any its VM is running, so VM is
never outdated.
2014-04-18 01:34:09 +02:00
Marek Marczykowski-Górecki
ca1be8683c core: Fix creation of template-based HVM 2014-04-18 01:23:39 +02:00
Marek Marczykowski-Górecki
1ac3f81848 core: increase default netvm memory to 300MB (#818) 2014-04-16 16:44:42 +02:00
Marek Marczykowski-Górecki
f4deddbbdf core: do not run commands on paused VM 2014-04-16 16:44:42 +02:00
Marek Marczykowski-Górecki
b17bf72b8a hvm: do not kill stubdom GUI in debug mode 2014-04-16 12:54:24 +02:00
Marek Marczykowski-Górecki
ac7746feed qvm-run: add color output and filtering escape sequences
This makes VM output clearly distinguishable
2014-04-15 03:19:48 +02:00
Marek Marczykowski-Górecki
5704b41a45 hvm: seamless_gui_mode setting, including runtime change support (#810) 2014-04-01 01:12:35 +02:00
Marek Marczykowski-Górecki
12d897cd3e hvm: fix startup of HVM without Qubes Tools
Do not pollute environment of calling process, otherwise all VMs started
from Qubes Manager afterwards will get QREXEC_STARTUP_NOWAIT, which
will cause wait_for_session not working.
2014-04-01 01:07:57 +02:00
Marek Marczykowski-Górecki
2eaf649eed core: add simple 'run_service' function 2014-04-01 01:07:23 +02:00
Marek Marczykowski-Górecki
72d277c56b core: add QubesVm.get_prefmem (#788)
Expose 'prefmem' property used by qmemman.
2014-03-31 03:45:16 +02:00
Marek Marczykowski-Górecki
242590902a firewall: minor improvements
Do not require ports specified in rule - useful for "any" protocol where
ports doesn't have sense.
2014-03-28 02:55:35 +01:00
Marek Marczykowski-Górecki
e90e1c62ec proxyvm: add support for rules with expire time (#760) 2014-03-28 02:54:59 +01:00
Marek Marczykowski-Górecki
d1fbd9c59d proxyvm: use "conntrack" iptables module instead of deprecated "state" 2014-03-27 17:16:36 +01:00
Marek Marczykowski-Górecki
04f86c7059 core: use functions instead of evals for storing attributes to qubes.xml 2014-03-27 17:15:15 +01:00
Marek Marczykowski-Górecki
e9fe890acb core: rewrite "eval" to "func" attribute handlers 2014-03-26 04:41:28 +01:00
Marek Marczykowski-Górecki
5141aba741 core: support functions instead of evals in attribute config
eval still supported, but once all the code will be converted to
functions, the support will be removed.
2014-03-26 04:40:45 +01:00
Marek Marczykowski-Górecki
91428ebaa1 core: method to resize root.img (#699) 2014-03-21 18:43:13 +01:00
Marek Marczykowski-Górecki
9768b38ffc core: handle errors with xenstore access during VM shutdown
When netvm and firewallvm is shut down, netvm handling code will
try to revoke firewallvm access to external IP. But if netvm shutdown
happens in the meantime, xenstore will throw ENOENT error.
2014-03-13 18:32:13 +01:00
Marek Marczykowski-Górecki
840dc38730 core: do not mark DispVMs as included in backups 2014-03-10 04:29:59 +01:00