Commit Graph

76 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
56ec271606
Fix initial value of dom0 static-max entry
This value needs to be set to actual static max for qmemman to work
properly. If it's set higher than real static-max, qmemman will try to
assign more memory to dom0, which dom0 could not use - will be wasted.
Since this script is executed before any VM is started, simply
take the current dom0 memory usage, instead of parsing dom0_mem Xen
argument. There doesn't seem to be nice API to get this value from Xen
directly.

Fixes QubesOS/qubes-issues#4891
2019-04-05 05:00:11 +02:00
Marek Marczykowski-Górecki
a88733149d
Add logrotate for qubes logs
Fixes QubesOS/qubes-issues#3603
2019-01-19 03:25:20 +01:00
Marek Marczykowski-Górecki
7feed2f680
Handle qubes.skip_autostart option on kernel command line
This allows to prevent automatically starting VMs at boot, mostly for
troubleshooting.

Fixes QubesOS/qubes-issues#4312
2018-09-16 05:22:30 +02:00
Rusty Bird
d3a5799245
Order qubesd before systemd-user-sessions
qubes-vm@.service would already cause this ordering, but not every user
has any autostart=True VMs.

Also needed to maybe f*x QubesOS/qubes-issues#3149 at some point.
2018-09-06 16:23:24 +00:00
Rusty Bird
6886e70aa9
backup.py: add vmN/empty file if no other files to backup 2018-02-24 22:43:53 +00:00
Marek Marczykowski-Górecki
5e89b23288
firewall: use asyncio's call_later instead of systemd to reload rules
When some expiring rules are present, it is necessary to reload firewall
when those rules expire. Previously systemd timer was used to trigger
this action, but since we have own daemon now, it isn't necessary
anymore - use this daemon for that.
Additionally automatically removing expired rules was completely broken
in R4.0.

Fixes QubesOS/qubes-issues#1173
2018-02-07 02:48:11 +01:00
Marek Marczykowski-Górecki
86026e364f
Fix starting PCI-having HVMs on early system boot and later
1. Make sure VMs are started after dom0 actual memory usage is reported
to qmemman, otherwise dom0 will hold 4GB, even if just a little over 1GB
is needed at that time.

2. Request only vm.memory MB from qmemman, instead of vm.maxmem. While
HVM with PCI devices indeed do not support populate-on-demand, this is
already handled in libvirt XML.

The later may often cause VM startup fail on systems with 8GB of memory,
because maxmem is 4GB there and with dom0 keeping the other 4GB (see
point 1) there is not enough memory to start any sych VM.

Fixes QubesOS/qubes-issues#3462
2018-01-29 22:57:32 +01:00
Rusty Bird
8506547923
block-snapshot: fix ShellCheck warnings (3/3)
These were the remaining warnings that only affected the newer
block-snapshot in this git repo.
2018-01-16 06:20:02 +00:00
Rusty Bird
50f35b3ec1
block-snapshot: fix ShellCheck warnings (2/3)
Forward-ported from qubes-core-agent-linux:

    commit aad6fa6d190d24393e326a4c2ff7ebc3b5921641
    Author: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
    Date:   Sat Sep 30 04:56:02 2017 +0200

        Hint shellcheck where to look for sourced files, if in repository

        This will ease running shellcheck from the repository.
2018-01-16 06:20:01 +00:00
Rusty Bird
45a06e7fad
block-snapshot: fix ShellCheck warnings (1/3)
Forward-ported from qubes-core-agent-linux:

    commit e95b6f8d0357ac9a32fb7e35e5734781460d8f24
    Author: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
    Date:   Sat Sep 30 04:53:38 2017 +0200

        Fix shellcheck warnings in block-snapshot script
2018-01-16 06:20:00 +00:00
Rusty Bird
342c36ed36
block-snapshot: remove trailing spaces
Forward-ported from qubes-core-agent-linux:

    commit 2eb0ed2be14350d6df1fce2af855805133a4a416
    Author: Patrick Schleizer <adrelanos@riseup.net>
    Date:   Thu Oct 15 04:34:55 2015 +0200

        removed trailing spaces
2018-01-16 06:19:58 +00:00
Rusty Bird
9dade5c219
block-snapshot: remove R3.2 template commit code
qvm-template-commit does not exist in R4.0, where FileVolume.stop()
takes care of committing.
2018-01-16 06:19:57 +00:00
Marek Marczykowski-Górecki
f738f5d4d8
Fix cleanup-dispvms script
qubesadmin.Qubes().domains do not support indexing by VM object.

QubesOS/qubes-issues#3037
2017-12-14 23:26:52 +01:00
Marek Marczykowski-Górecki
e209e448f2
Fix script for DispVM cleanup
Cleanup DispVMs after non-clean shutdown

Fixes QubesOS/qubes-issues#3037
2017-12-05 17:41:22 +01:00
Marek Marczykowski-Górecki
2cc5e4ce9a
qmemman: remove cyclic service ordering
Fixes QubesOS/qubes-issues#2224
2017-11-21 13:01:51 +01:00
Marek Marczykowski-Górecki
08584ce370
Fix qubes-core.service ordering
The service needs to be started after qubesd and qubes-qmemman to be
able to start domains. The same applies on shutdown - should be ordered
before shutting down qubesd.
2017-11-07 02:59:20 +01:00
Marek Marczykowski-Górecki
8d8bbb99e9
Order qubesd.service after lvm2-pvscan@.service
qubesd needs to list LVM volumes, so start it when the list is
available.

Fixes QubesOS/qubes-issues#2960
2017-08-12 19:50:50 +02:00
Marek Marczykowski-Górecki
5a9f478466
Automatically restart qubesd when it crashes
qubesd service is critical for Qubes usage, so even in case of critical
error crashing the whole service, make sure it is restarted.
Set delay to 1s (default 100ms), to allow other services to restart too,
if crash was caused by some other service (like libvirtd crash).

QubesOS/qubes-issues#2960
2017-08-12 15:51:28 +02:00
Marek Marczykowski-Górecki
c035f8fc35
systemd: order VMs autostart after qubesd
Fixes QubesOS/qubes-issues#2919
2017-07-18 04:04:26 +02:00
Marek Marczykowski-Górecki
a9ece6f706
Remove qubes-netvm.service
Since qubesd properly handle chained startup of sys-net->sys-firewall
etc, we don't need a separate service to start netvm explicitly earlier.

Fixes QubesOS/qubes-issues#2533
2017-07-07 20:35:13 +02:00
Marek Marczykowski-Górecki
79a8d9c58f
systemd: fix qubes-vm@.service - qvm-start no longer have --no-guid arg 2017-06-05 23:33:58 +02:00
Marek Marczykowski-Górecki
0816f002ba
systemd: use appropriate meminfo-writer service for dom0 2017-06-05 23:33:58 +02:00
Marek Marczykowski-Górecki
ae66d29aff
Cleanup qubes-block-cleaner from one more place 2017-06-05 23:33:58 +02:00
Marek Marczykowski-Górecki
880c190587
systemd: prevent killing VM-daemons on service restart/stop
Kill only main qubesd process, not all the control group (which include
all qrexec-daemon and qubesdb-daemon processes).
2017-05-18 11:11:18 +02:00
Marek Marczykowski-Górecki
207d497bfe
Remove block-cleaner-daemon
This service currently does more harm (desync libvirt state with
reality) than good. Since we have qubesd, we may come back to
implementing it properly using libvirt events.
2017-05-17 01:33:30 +02:00
Marek Marczykowski-Górecki
eaf5c27b27
Merge branch 'core3-devel'
This concludes over two years of rewriting Qubes' core.
Thank God it's now merged to master. --@woju

QubesOS/qubes-issues#1825
2017-05-12 19:59:14 +02:00
Marek Marczykowski-Górecki
81162a699b
Remove unused helper scripts 2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki
2e7f483a60
Remove old pvusb handling code 2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki
8992e71f85
Remove core2 code
This all either have been migrated to core3, or is not needed anymore.

There is still qvm-tools directory with a few tools that needs to be
migrated, or installed as is.
2017-05-12 18:36:29 +02:00
Marek Marczykowski-Górecki
a3da85bfda
qubespolicy: run GUI code inside user session and expose it as dbus object
This way it will work independently from where qrexec-policy tool will
be called (in most cases - from a system service, as root).
This is also very similar architecture to what we'll need when moving to
GUI domain - there GUI part will also be separated from policy
evaluation logic.

QubesOS/qubes-issues#910
2017-04-07 17:07:30 +02:00
M. Vefa Bicakci
5e2bd5ea64
qubes-core.service: Do not shut down/reboot before all VMs have shut down
Prior to this commit, the qubes-core.service inherited systemd's default
timeout value of 90 seconds. With slow hard disk drives, this caused the
dom0 shut-down sequence to proceed even if some VMs were still not fully
shut down at the time of dom0 shut down.

This commit aims to avoid this issue by setting the service stop timeout to
180 seconds.

Signed-off-by: M. Vefa Bicakci <m.v.b@runbox.com>
2017-03-16 12:11:14 +01:00
Marek Marczykowski-Górecki
c5a8135fdb
fix misplaced comment 2017-02-27 02:37:51 +01:00
Marek Marczykowski-Górecki
b89689e278
storage: implement two-layers of dm-snapshot in block-snapshot script
Have dm-snapshot of dm-snapshot. The first layer is to "cache" changes
done by base volume holder (TemplateVM in case of root.img), the second
layer is to hold changes do by snapshot volume holder (AppVM in case of
root.img). In case of Linux VMs the second layer is normally done inside
of VM (original volume is exposed read-only). But this does not work for
non-Linux VMs, orr even Linux but without qubes-specific startup
scripts.

This is first part of the change - actual construction of two layers of
dm-snapshot, not plugged in to core scripts yet.

QubesOS/qubes-issues#2256
2017-02-14 23:59:07 +01:00
Wojtek Porczyk
0be3b1fbb1 qubes/tools/qubesd: initial version 2017-02-07 17:07:53 +01:00
Marek Marczykowski-Górecki
28475f7214
systemd: adjust qubes-prefs options
As the service is called as root, it needs --force-root.

QubesOS/qubes-issues#2412
2016-11-26 04:08:58 +01:00
Marek Marczykowski-Górecki
d954bb82a3
block-snapshot: set qubes.xml path for test VMs
Since it is running with totally different environment, QUBES_XML_PATH
isn't passed from test runner. Apply heuristic based on VM name.
2016-08-09 05:14:11 +02:00
Marek Marczykowski-Górecki
3ff7281e97
systemd: don't setup xenstore dom0 entries
Should be done by xen-init-dom0 upstream service.
2016-08-08 01:18:26 +02:00
Marek Marczykowski-Górecki
4e022382a5
Merge remote-tracking branch 'origin/master' into core3-devel 2016-08-08 00:11:46 +02:00
Marek Marczykowski-Górecki
2908992741
prepare-volatile-img.sh: create volatile.img group accessible
Otherwise qvm-create-default-dvm may fail to include it in
saved-cows.tar, which will lead to DispVM being not really disposable.

Fixes QubesOS/qubes-issues#2200
2016-08-07 16:07:30 +02:00
HW42
09b49feea6
prepare-volatile-img.sh: don't run as root
This is no longer necessary since volatile.img is formated inside the
VM. This also fixes DispVM creation if the user sets a restrictive umask
for root. Maybe related to #2200.
2016-08-07 00:41:59 +02:00
Wojtek Porczyk
9eafa57539 Merge remote-tracking branch 'marmarek/master' into core3-devel
The following list is bollocks. There were many, many more.

Conflicts:
    core-modules/003QubesTemplateVm.py
    core-modules/005QubesNetVm.py
    core/qubes.py
    core/storage/__init__.py
    core/storage/xen.py
    doc/qvm-tools/qvm-pci.rst
    doc/qvm-tools/qvm-prefs.rst
    qubes/tools/qmemmand.py
    qvm-tools/qvm-create
    qvm-tools/qvm-prefs
    qvm-tools/qvm-start
    tests/__init__.py
    vm-config/xen-vm-template-hvm.xml

This commit took 2 days (26-27.01.2016) and put our friendship to test.
    --Wojtek and Marek
2016-03-03 01:13:51 +01:00
Marek Marczykowski-Górecki
169eac6412
systemd: do not fail qubes-vm@ service when the VM is already running
Fixes QubesOS/qubes-issues#1528
2015-12-26 11:34:40 +01:00
Marek Marczykowski-Górecki
a30e8d1f33
systemd: fix starting VMs _before_ allowing user login
systemd-user-sessions.service is specicically for that, do not use hack
(plymouth-quit.service), which doesn't work when the service is
disabled.

Fixes QubesOS/qubes-issues#1250
2015-12-23 19:08:01 +01:00
Marek Marczykowski-Górecki
de275cc917
core: leave volatile.img structure creation for the VM
This is already handled by a script in initramfs.

Fixes QubesOS/qubes-issues#1308
2015-11-25 01:35:04 +01:00
Marek Marczykowski-Górecki
bae493d015
Kill remaining qubesdb-daemon instances after shutting down all the VMs
It may happen (especially when VM doesn't close cleanly and needs to be
killed) that qubesdb-daemon will not notice VM shutdown immediately.
Normally it would stop after 60s timeout, but speed it up in case of
system shutdown

QubesOS/qubes-issues#1425
2015-11-15 02:06:19 +01:00
Marek Marczykowski-Górecki
46cbb4a133
Prevent stopping xenconsoled before shutting down all the VMs
Otherwise still running VMs will hang, consuming 100% CPU.

QubesOS/qubes-issues#1425
2015-11-14 23:29:21 +01:00
Marek Marczykowski-Górecki
06f5922da6
Adjust dom0 sched-credit weight based on default libvirt value for VMs
While default value for VMs is normally 256, libvirt set it to
(hardcoded) 1000. So adjust dom0 accordingly.

Fixes QubesOS/qubes-issues#1404
2015-11-11 04:30:01 +01:00
Wojtek Porczyk
c538d536c8 core3: move qmemman
This is part of fixing qvm-start.

qmemman was moved with minimal touching, mainly module names.

Moved function parsing human-readable sizes from core2. This function is
wrong, because it treats k/M/G as 1024-based, but leave it for now.
2015-10-05 12:46:14 +02:00
Marek Marczykowski-Górecki
d27d22a3cf
Merge remote-tracking branch 'qubesos/pr/5'
* qubesos/pr/5:
  fix VM autostart race condition

Fixes qubesos/qubes-issues#1075

Conflicts:
	linux/aux-tools/prepare-volatile-img.sh
2015-08-01 22:04:18 +02:00
Marek Marczykowski-Górecki
c4a28a943a
prepare-volatile-img: ensure that /usr/sbin in in the PATH
Build environment of LiveUSB does not contain /usr/sbin in path.
2015-08-01 22:02:07 +02:00