Commit Graph

621 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
ccd7e285a1
Remove unused core2migration.py
It is part of backup-restore code, but it was moved to
core-admin-client repo.
2018-12-04 16:12:33 +01:00
Marek Marczykowski-Górecki
d23636fa02
tests: migrate qvm-block tests to core3 2018-10-10 00:44:15 +02:00
Marek Marczykowski-Górecki
02f9661169
tests: migrate mime handlers test to core3 2018-10-07 15:46:52 +02:00
Marek Marczykowski-Górecki
b88fa39894
Fix mock-based build 2018-09-29 02:40:28 +02:00
Rusty Bird
797bbc43a0
tests/storage_reflink: test some file-reflink helpers
Tested:

    - _copy_file()
    - _create_sparse_file()
    - _resize_file()
    - _update_loopdev_sizes()

Smoke tested by calls from the functions above:

    - _replace_file()
    - _rename_file()
    - _make_dir()
    - _fsync_dir()
2018-09-11 23:50:32 +00:00
Marek Marczykowski-Górecki
af7d54d388
Update windows-related feature requests
Handle 'os' feature - if it's Windows, then set rpc-clipboard feature.
Handle 'gui-emulated' feature - request for specifically stubdomain GUI.
With 'gui' feature it is only possible to enable gui-agent based on, or
disable GUI completely.
Handle 'default-user' - verify it for weird characters and set
'default_user' property (if wasn't already set).

QubesOS/qubes-issues#3585
2018-07-16 22:02:58 +02:00
Frédéric Pierret
ecdaa157a9
spec.in: remove useless condition on Fedora 17 which induces problem with Travis 2018-04-04 16:24:21 -04:00
Frédéric Pierret
6fb14e8198
spec.in: add changelog placeholder 2018-04-03 23:12:15 +02:00
Frédéric Pierret
601e88854c
Define backend_vmm macro 2018-04-03 23:12:15 +02:00
Frédéric Pierret
32fad062f1
Remove _builddir 2018-04-03 23:12:14 +02:00
Frédéric Pierret
fdea48aa1c
Create .spec.in and Source0 2018-04-03 23:12:14 +02:00
Marek Marczykowski-Górecki
bda9264e19
Add qubes.GetDate proxy service
This enable two things:
1. Follow global clockvm setting, without adjusting qrexec policy.
2. Avoid starting clockvm by arbitrary VM.

Fixes QubesOS/qubes-issues#3588
2018-03-03 03:50:58 +01:00
Marek Marczykowski-Górecki
5840dd76f9
Add default policy for qubes.StartApp service
Fixes QubesOS/qubes-issues#3044
2018-02-28 04:41:45 +01:00
Marek Marczykowski-Górecki
b1e83f899f
tests: convert pvgrub tests to core3 API
QubesOS/qubes-issues#3563
2018-02-28 04:41:30 +01:00
Marek Marczykowski-Górecki
1562defd42
Merge remote-tracking branch 'qubesos/pr/193'
* qubesos/pr/193:
  tests: add a test for removing expired firewall rules
  firewall: use asyncio's call_later instead of systemd to reload rules
2018-02-22 19:47:37 +01:00
Marek Marczykowski-Górecki
522a9594ee
rpm: adjust dependencies 2018-02-20 01:18:15 +01:00
Rusty Bird
1695a732b8
file-reflink, a storage driver optimized for CoW filesystems
This adds the file-reflink storage driver. It is never selected
automatically for pool creation, especially not the creation of
'varlibqubes' (though it can be used if set up manually).

The code is quite small:

               reflink.py  lvm.py      file.py + block-snapshot
    sloccount  334 lines   447 (134%)  570 (171%)

Background: btrfs and XFS (but not yet ZFS) support instant copies of
individual files through the 'FICLONE' ioctl behind 'cp --reflink'.
Which file-reflink uses to snapshot VM image files without an extra
device-mapper layer. All the snapshots are essentially freestanding;
there's no functional origin vs. snapshot distinction.

In contrast to 'file'-on-btrfs, file-reflink inherently avoids
CoW-on-CoW. Which is a bigger issue now on R4.0, where even AppVMs'
private volumes are CoW. (And turning off the lower, filesystem-level
CoW for 'file'-on-btrfs images would turn off data checksums too, i.e.
protection against bit rot.)

Also in contrast to 'file', all storage features are supported,
including

    - any number of revisions_to_keep
    - volume.revert()
    - volume.is_outdated
    - online fstrim/discard

Example tree of a file-reflink pool - *-dirty.img are connected to Xen:

    - /var/lib/testpool/appvms/foo/volatile-dirty.img
    - /var/lib/testpool/appvms/foo/root-dirty.img
    - /var/lib/testpool/appvms/foo/root.img
    - /var/lib/testpool/appvms/foo/private-dirty.img
    - /var/lib/testpool/appvms/foo/private.img
    - /var/lib/testpool/appvms/foo/private.img@2018-01-02T03:04:05Z
    - /var/lib/testpool/appvms/foo/private.img@2018-01-02T04:05:06Z
    - /var/lib/testpool/appvms/foo/private.img@2018-01-02T05:06:07Z
    - /var/lib/testpool/appvms/bar/...
    - /var/lib/testpool/appvms/...
    - /var/lib/testpool/template-vms/fedora-26/...
    - /var/lib/testpool/template-vms/...

It looks similar to a 'file' pool tree, and in fact file-reflink is
drop-in compatible:

    $ qvm-shutdown --all --wait
    $ systemctl stop qubesd
    $ sed 's/ driver="file"/ driver="file-reflink"/g' -i.bak /var/lib/qubes/qubes.xml
    $ systemctl start qubesd
    $ sudo rm -f /path/to/pool/*/*/*-cow.img*

If the user tries to create a fresh file-reflink pool on a filesystem
that doesn't support reflinks, qvm-pool will abort and mention the
'setup_check=no' option. Which can be passed to force a fallback on
regular sparse copies, with of course lots of time/space overhead. The
same fallback code is also used when initially cloning a VM from a
foreign pool, or from another file-reflink pool on a different
mountpoint.

'journalctl -fu qubesd' will show all file-reflink copy/rename/remove
operations on VM creation/startup/shutdown/etc.
2018-02-12 21:20:05 +00:00
Marek Marczykowski-Górecki
5e89b23288
firewall: use asyncio's call_later instead of systemd to reload rules
When some expiring rules are present, it is necessary to reload firewall
when those rules expire. Previously systemd timer was used to trigger
this action, but since we have own daemon now, it isn't necessary
anymore - use this daemon for that.
Additionally automatically removing expired rules was completely broken
in R4.0.

Fixes QubesOS/qubes-issues#1173
2018-02-07 02:48:11 +01:00
Marek Marczykowski-Górecki
76836d4ff6
rpm: require libvirt new enough for type-pvh 2018-01-27 01:21:05 +01:00
Marek Marczykowski-Górecki
2041e6f01c
rpm: require new enough libvirt for pci permissive setting 2018-01-22 15:31:12 +01:00
Marek Marczykowski-Górecki
f1a5ca64fd
Merge remote-tracking branch 'qubesos/pr/180'
* qubesos/pr/180:
  vm/qubesvm: default to PVH unless PCI devices are assigned
  vm/qubesvm: expose 'start_time' property over Admin API
  vm/qubesvm: revert backup_timestamp to '%s' format
  doc: link qvm-device man page for qvm-block, qvm-pci, qvm-usb
2018-01-15 04:22:28 +01:00
Marek Marczykowski-Górecki
be7b278032
doc: link qvm-device man page for qvm-block, qvm-pci, qvm-usb
Those three are special cases of qvm-device tool, so lets use its
documentation too.
2018-01-11 03:44:53 +01:00
Marek Marczykowski-Górecki
962742880f
tests: basic salt integration tests
Test base functions of dom0 module (creating VM, setting property) and
configuring system inside of VM (through DispVM). The later is done for
each available template (the process use salt installed in that
template, not copied from dom0).

QubesOS/qubes-issues#3316
2017-12-23 02:26:15 +01:00
Wojtek Porczyk
61c164e1c3 qubes-rpc: policy.RegisterArgument
This qrexec is meant for services, which require some kind of
"registering" before use. After registering, the backend should invoke
this call with frontend as the intended destination, with the actual
service in argument of this call and the argument as the payload.

By default this qrexec is disabled by policy.

Signed-off-by: Wojtek Porczyk <woju@invisiblethingslab.com>
2017-11-30 00:26:21 +01:00
Marek Marczykowski-Górecki
2164a8d7b8
Change license to LGPL v2.1+
See this thread for reasoning and acceptance from contributors:
https://groups.google.com/d/topic/qubes-devel/G7KzrfU0lWY/discussion
"Changing qubes-core-admin license to LGPL v2.1+"
2017-10-12 00:11:50 +02:00
Marek Marczykowski-Górecki
53903a3685
rpm: remove commented out rpm macro
Just commenting out macro isn't enough - % sequences are still
processed. Remove it completely, as it is already provided in build
environment.
2017-10-11 22:41:31 +02:00
Wojtek Porczyk
044e10a6ec Merge remote-tracking branches 'origin/pull/142/head', 'origin/pull/143/head' and 'origin/pull/144/head' 2017-08-14 16:32:56 +02:00
Marek Marczykowski-Górecki
7a9e5c3650
tests: qrexec-policy cli tool tests 2017-08-14 02:24:32 +02:00
Marek Marczykowski-Górecki
12661dccf8
qubespolicy: ask to create default policy if none is found
Fixes QubesOS/qubes-issues#3005
2017-08-14 02:24:31 +02:00
Marek Marczykowski-Górecki
8d8bbb99e9
Order qubesd.service after lvm2-pvscan@.service
qubesd needs to list LVM volumes, so start it when the list is
available.

Fixes QubesOS/qubes-issues#2960
2017-08-12 19:50:50 +02:00
Marek Marczykowski-Górecki
5d0a2fe463
Add default policy for qubes.VMRootShell service 2017-08-06 20:54:11 +02:00
Marek Marczykowski-Górecki
691a6f4d8c
vm/dispvm: add auto_cleanup property, unify creating new DispVM
Add auto_cleanup property, which remove DispVM after its shutdown
- this is to unify DispVM handling - less places needing special
handling after DispVM shutdown.

New DispVM inherit all settings from respective AppVM. Move this from
classmethod `DispVM.from_appvm()`, to DispVM constructor. This unify
creating new DispVM with any other VM class.
Notable exception are attached devices - because only one running VM can
have a device attached, this would prevent second DispVM started from
the same AppVM. If one need DispVM with some device attached, one can
create DispVM with auto_cleanup=False. Such DispVM will still not have
persistent storage (as any other DispVM).

Tests included.

QubesOS/qubes-issues#2974
2017-08-06 20:54:09 +02:00
Marek Marczykowski-Górecki
36f1a3abaf
Merge branch 'services'
* services:
  tests: check clockvm-related handlers
  doc: include list of extensions
  qubesvm: fix docstring
  ext/services: move exporting 'service.*' features to extensions
  app: update handling features/service os ClockVM
2017-07-29 05:09:32 +02:00
Marek Marczykowski-Górecki
1a6728cb12
ext/services: move exporting 'service.*' features to extensions
This means core code will not publish any features by default.
2017-07-28 16:32:47 +02:00
Marek Marczykowski-Górecki
cdbe1d14b3
rpm: include directory for backup profiles
QubesOS/qubes-issues#2931
2017-07-21 03:16:57 +02:00
Marek Marczykowski-Górecki
6dbce8259f
api/admin: plug backup into Admin API
Fixes QubesOS/qubes-issues#2931
2017-07-20 12:19:01 +02:00
Marek Marczykowski-Górecki
7f85d81197
Remove qvm-backup tool
The tool itself should live in qubes-core-admin-client and communicate
with actual backup code over Admin API.
2017-07-18 01:42:37 +02:00
Marek Marczykowski-Górecki
61519014cb
Remove qvm-backup-restore and its backend code
This functionality is moved to qubes-core-admin-client, which does it
over Admin API

QubesOS/qubes-issues#1214
2017-07-18 01:30:39 +02:00
Marek Marczykowski-Górecki
3c99aab75a
Merge branch 'drop-netvm-startup'
* drop-netvm-startup:
  Remove qubes-netvm.service
2017-07-14 01:13:42 +02:00
Marek Marczykowski-Górecki
a9ece6f706
Remove qubes-netvm.service
Since qubesd properly handle chained startup of sys-net->sys-firewall
etc, we don't need a separate service to start netvm explicitly earlier.

Fixes QubesOS/qubes-issues#2533
2017-07-07 20:35:13 +02:00
Marta Marczykowska-Górecka
6da06d424e
clock synchronization rewrite
clock synchronization mechanism rewritten to use systemd-timesync instead of NtpDate; at the moment, requires:
- modifying /etc/qubes-rpc/policy/qubes.GetDate to redirect GetDate to designated clockvm
- enabling clocksync service in clockvm ( qvm-features clockvm-name service/clocksync true )

Works as specified in issue listed below, except for:
- each VM synces with clockvm after boot and every 6h
- clockvm synces time with the Internet using systemd-timesync
- dom0 synces itself with clockvm every 1h (using cron)

fixes QubesOS/qubes-issues#1230
2017-07-06 23:37:26 +02:00
Marek Marczykowski-Górecki
a96a85bdc9
qubespolicy: add a tool to analyze policy in form of graph
Output possible connections between VMs in form of dot file.

Fixes QubesOS/qubes-issues#2873
2017-07-04 04:27:36 +02:00
Marek Marczykowski-Górecki
3d803acfde
Generate policy for Admin API calls based on annotations on actual methods
This ease Admin API administration, and also adds checking if qrexec
policy + scripts matches actual Admin API methods implementation.
The idea is to classify every Admin API method as either local
read-only, local read-write, global read-only or global read-write.
Where local/global means affecting a single VM, or the whole system.

See QubesOS/qubes-issues#2871 for details.

Fixes QubesOS/qubes-issues#2871
2017-07-04 04:27:34 +02:00
Marek Marczykowski-Górecki
26013122a0
Merge remote-tracking branch 'woju/devel-adminext' into core3-devel 2017-06-23 10:34:11 +02:00
Wojtek Porczyk
9f57db8749 rpm_spec: fix 2017-06-23 10:26:04 +02:00
Marek Marczykowski-Górecki
c1f4c219f9
tests: adjust TC_00_QubesDaemonProtocol for reorganized api module 2017-06-21 06:59:58 +02:00
Marek Marczykowski-Górecki
ee442c754f
api/misc: integrate qubes.NotifyTools logic with qubes.FeaturesRequest
Make qubes.NotifyTools reuse logic of qubes.FeaturesRequest, then move
actual request processing to 'features-request' event handler. At the
same time implement handling 'qrexec' and 'gui' features request -
allowing to set template features when wasn't already there.
Behavior change: template is no longer allowed to change feature value
(regardless of being True or False). This means the user will always be
able to override what template have set.
2017-06-12 12:22:39 +02:00
Marek Marczykowski-Górecki
67fcda2cde
tests: basic tests for AppVM
Specifically, check if root volume is updated after template switch.
2017-06-06 03:33:31 +02:00
Marek Marczykowski-Górecki
28737d16ce
Migrate qubes.NotifyTools, qubes.FeaturesRequest, qubes.NotifyUpdates
Make them call into qubesd. Create separate socket for "misc" calls - VM
accessible, but not part of Admin API.
2017-06-06 03:33:30 +02:00
Marek Marczykowski-Górecki
1f86c9253c
rpm: depend on linux-stubdom package
Install both stubdom implementations: mini-os one (xen-hvm) and linux
one (xen-hvm-stubdom-linux).

QubesOS/qubes-issues#2185
2017-06-05 23:33:58 +02:00