Commit Graph

3308 Commits

Author SHA1 Message Date
Marek Marczykowski
f159f3e168 Use QubesDB instead of Xenstore.
Mostly done. Things still using xenstore/not working at all:
 - DispVM
 - qubesutils.py (especially qvm-block and qvm-usb code)
 - external IP change notification for ProxyVM (should be done via RPC
   service)
2014-11-19 12:48:28 +01:00
Marek Marczykowski
31424603fa rpm: move R: xen-runtime inside vmm-xen deps block
Do not depend on xen package unconditionally.
2014-11-19 12:48:28 +01:00
Marek Marczykowski
6c345aaccf hvm: enable missing features
Apparently libvirt driver have own defaults for machine features - all
disabled (instead of using libxl defaults). So enable such features
explicitly.
2014-11-19 12:48:28 +01:00
Marek Marczykowski
5644656859 hvm: add UUID to libvirt config
Add missing entry. Also fix indentation of first few lines.
2014-11-19 12:48:28 +01:00
Marek Marczykowski
3f6947b87e dispvm: fix domain ID handling
Do not use obsoleted get_xid() function.
2014-11-19 12:48:27 +01:00
Marek Marczykowski
9393b5f931 Fix domain rename
libvirt_domain object needs to be recreated, so force it. Also fix
config path setting (missing extension) - create_config_file
uses it as custom config indicator (if such detected, VM settings -
especially name, would not be updated).
2014-11-19 12:48:27 +01:00
Marek Marczykowski
062c3afcf5 Remove unused 'source_template' parameter from some QubesVm internal functions 2014-11-19 12:48:27 +01:00
Marek Marczykowski
fc7d200452 add vim swapfiles to gitignore 2014-11-19 12:48:27 +01:00
Marek Marczykowski
c95dc298a1 rpm: set sgid for data directories
Make sure that contents belong to qubes group, even when created by root
user.
2014-11-19 12:48:27 +01:00
Marek Marczykowski
25ee58217d Set umask before possibly creating new files - make them group-writable 2014-11-19 12:48:27 +01:00
Marek Marczykowski
b3af858502 Indentation fix - check for custom config file 2014-11-19 12:48:27 +01:00
Marek Marczykowski
5db1957086 rpm: improve deps on libvirt
Use metapackage to install all required libvirt modules.
2014-11-19 12:48:27 +01:00
Marek Marczykowski
5a28074c2b qvm-template-commit: add --offline-mode option
Allow force offline mode (disable check if VM isn't running) - useful
for running from anaconda (inside of chroot).
2014-11-19 12:48:27 +01:00
Marek Marczykowski
8e9c59e7f8 Minor changes to make VMM connection really lazy
1. Fake dom0 object doesn't need proper maxmem nor vcpus - set
statically to 0 instead of getting from physical host.
2. QubesHVM doesn't preserve maxmem setting, so set it to self.memory
earlier (to suppress default total_memory/2 calculation).
2014-11-19 12:48:27 +01:00
Marek Marczykowski
b8c62c0279 Wrap all VMM connection related object into QubesVMMConnection class
This makes easier to import right objects in submodules (only one
object). This also implement lazy connection - at first access, not at
module import, which speeds up tools, which doesn't need runtime
information (like qvm-prefs or qvm-service). In the future this will
ease migration from xenstore to QubesDB.

Also implement "offline mode" - operate on qubes.xml without connecting
to VMM - raise exception at such try.
This is needed to run tools during installation, where only minimal
set of services are started, especially no libvirt.
2014-11-19 12:48:26 +01:00
Marek Marczykowski
9f90106db4 rpm: Add libvirt to dependencies 2014-11-19 12:48:26 +01:00
Marek Marczykowski
0f6b878664 rpm: update build dependencies 2014-11-19 12:48:26 +01:00
Marek Marczykowski
201cd509e1 QubesDom0NetVm: provide get_mem*
As libvirt doesn't keep dom0 domain object, so add special cases to get
memory information.
2014-11-19 12:48:26 +01:00
Marek Marczykowski
550d8cac68 Create permanent libvirt domain objects
Do not recreate them at each startup. This will save some time and also
solve some problems from invalidated libvirt handles after domain
shutdown (e.g. causes qubes-manager crashes).
This requires storing uuid in qubes.xml.
2014-11-19 12:48:26 +01:00
Marek Marczykowski
1fda0502ab Check for invalid XID 2014-11-19 12:48:26 +01:00
Marek Marczykowski
895ca0633d Fix current memory reporting 2014-11-19 12:48:26 +01:00
Marek Marczykowski
4f090cdb5e pass VM name to guid via cmdline 2014-11-19 12:48:26 +01:00
Marek Marczykowski
3b5c89d9ce makefile: change build order
Move docs to the end of list - build errors in core code can be spotted
earlier.
2014-11-19 12:48:26 +01:00
Marek Marczykowski
107ebad9d5 Migration to libvirt - DispVM
Move DispVM creation to qfile-daemon-dvm/QubesDisposableVm from
qubes-restore. As actual restore is handled by libvirt, we don't get
much from separate qubes-restore process.
This code still needs some improvements, especially on performance.
2014-11-19 12:48:26 +01:00
Marek Marczykowski
f44dc40858 Migration to libvirt - HVM 2014-11-19 12:47:00 +01:00
Marek Marczykowski
f3a7d5f6e6 Remove debug_console QubesVm.start() argument
It isn't working for a long time, so remove it to not confuse reader.
2014-11-19 12:47:00 +01:00
Marek Marczykowski
a880483092 Migration to libvirt - core part
Still not all code migrated, added appropriate TODO/FIXME comments.
2014-11-19 12:47:00 +01:00
Marek Marczykowski
4a4fb5e243 qmemman: change process model from forking to single process + sd_notify
Forking daemon after initializing hypervisor connection can cause
problems (and actually does in case of libvirt).
To notify systemd when daemon is ready use notify socket (previously it
was termination of parent process).
2014-11-19 12:47:00 +01:00
Marek Marczykowski
b242680cc1 spec: fix typi in %post 2014-11-19 12:46:59 +01:00
Marek Marczykowski-Górecki
7fbbcd58bf dispvm: print diagnostics to stderr 2014-11-10 11:42:47 +01:00
Marek Marczykowski-Górecki
49ca49017e tests: DispVM tests 2014-11-10 11:42:30 +01:00
Marek Marczykowski-Górecki
256e35a62c tests: annotate qrexec EOF tests with expected failure
EOF transmission actually doesn't work in some cases in R2. It is rather
hard to change without introducing big changes (which we don't want to
do in R2). It should be fixed in R3.

BTW The said changes are in my qrexec-sockets branch in core-agent-linux
and linux-utils (then application should use shutdown(2) instead of
close). Alternatively qrexec-agent can use PAM directly (instead of
calling "su"). This is also implemented (pam branch in
core-agent-linux), but IMO it isn't the way to go (reimplementing "su"
in qrexec-agent code).
2014-11-10 02:28:18 +01:00
Marek Marczykowski-Górecki
19d064ca66 qvm-tools/qvm-run: do not echo empty line, remove unneeded imports 2014-11-10 02:25:27 +01:00
Marek Marczykowski-Górecki
ac155705d1 qvm-tools/qvm-run: remove --shutdown option
There is separate tool for this operation (qvm-shutdown). qvm-run
options related to shutdown (--wait, --force) can be confusing to the
user.
2014-11-10 02:24:43 +01:00
Marek Marczykowski-Górecki
1b99011bbb dispvm: use getopt to parse qubes-restore cmdline
It will ease further enhancements, especially adding additional options.
2014-11-04 02:55:35 +01:00
Marek Marczykowski-Górecki
fa6e6da211 tests: qrexec VM-VM EOF transmission tests
Similar to previous commit - test_053_qrexec_vm_service_eof_reverse
fails for now.
2014-10-30 06:37:41 +01:00
Marek Marczykowski-Górecki
d202ec7bc4 tests: qrexec EOF transmission tests
Currently EOF VM->dom0 test fails...
2014-10-29 22:00:20 +01:00
Marek Marczykowski-Górecki
c3b0b4f75f tests: separate test for qrexec "loopback" call 2014-10-29 21:38:54 +01:00
Marek Marczykowski-Górecki
91425d2e0b version 2.1.64 2014-10-25 01:46:58 +02:00
Marek Marczykowski-Górecki
9e62b77ecd rpm: require qubes-core-dom0-linux >= 2.0.24 for qrexec '-q' option 2014-10-25 01:46:26 +02:00
Marek Marczykowski-Górecki
aa958eeff6 core: really quiet mode if vm.start()
Pass '-q' option to gui-daemon and qrexec-daemon.
2014-10-25 01:45:01 +02:00
Marek Marczykowski-Górecki
247cff335f core: fix race condition in qubes.xml locking (#906)
QubesVmCollection.save() overrides qubes.xml by creating new file, then
renaming it over the old one. If any process has that (old) file open
at the same time - especially while waiting on lock_db_for_writing() -
it will end up in accessing old, already unlinked file.

The exact calls would look like:
P1                                      P2
lock_db_for_writing
  fd = open('qubes.xml')
  fcntl(fd, F_SETLK, ...)

                                      lock_db_for_writing
                                          fd = open('qubes.xml')
                                          fcntl(fd, F_SETLK, ...)
...
save():
    open(temp-file)
    write(temp-file, ...)
    ...
    flush(temp-file)
    rename(temp-file, 'qubes.xml')
    close(fd) // close old file

                                      lock_db_for_writing succeed
                                      *** fd points at already unlinked
                                          file
unlock_db
    close(qubes.xml)

To fix that problem, added a check if (already locked) file is still the
same as qubes.xml.
2014-10-22 03:53:30 +02:00
Marek Marczykowski-Górecki
9324cdb175 tests: add test for race condition during VM creation (#906) 2014-10-22 03:52:43 +02:00
Marek Marczykowski-Górecki
d5817eb6c5 tests: add some basic tests for running VM, qrexec, GUI 2014-10-22 02:14:25 +02:00
Victor Lopez
bf2173073b Linux VM specific: maxmem limited by init mem
Check maxmem taking into account the minimum init memory that allows
that requested maximum memory.

Explanation:
Linux kernel needs space for memory-related structures created at boot.
If init_mem is just 400MB, then max_mem can't balloon above 4.3GB (at
which poing it yields "add_memory() failed: -17" messages and apps
crash), regardless of the max_mem_size value.

Based on Marek's findings and my tests on a 16GB PC, using several
processes like:
   stress -m 1 --vm-bytes 1g --vm-hang 100

result in the following points:
init_mem  ==> actual max memory
400             4300
700             7554
800             8635
1024            11051
1200            12954
1300            14038
1500            14045 <== probably capped on my 16GB system

The actual ratio of max_mem_size/init_mem is surprisingly constant at
10.79

If less init memory is set than that ratio allows, then the set
maxmem is unreachable and the VM becomes unstable (app crashes)

Based on qubes-devel discussion titled "Qubes Dom0 init memory against
Xen best practices?" at:
https://groups.google.com/d/msg/qubes-devel/VRqkFj1IOtA/UgMgnwfxVSIJ
2014-10-06 04:02:07 +02:00
Marek Marczykowski-Górecki
a457a3e5ca tests: initial VM mgmt tests 2014-10-02 10:56:04 +02:00
Marek Marczykowski-Górecki
c58fc767a3 tests: fix test_r1 backup test 2014-10-02 02:38:51 +02:00
Marek Marczykowski-Górecki
8193d3dd61 tests: add test restore of R1 backup 2014-10-01 05:42:33 +02:00
Marek Marczykowski-Górecki
bd4cd9baa2 tests: minor improvements to backupcompatibility/test_r2b2 2014-10-01 05:36:47 +02:00
Marek Marczykowski-Górecki
58586a0e38 tests: add test for restoring R2B2 backup
One of "format 1" backup.
2014-10-01 03:50:56 +02:00