* origin/pr/354: (35 commits)
tests/lvm: re-introduce POOL_CONF
tests/lvm & callback: remove explicit class references
storage/callback: remove the "word of caution"
storage/callback: comment fixes
storage/callback: add the config ID as callback argument
storage/callback: some callbacks added & removed
tests/lvm & callback: Refactoring
Revert "storage/callback: do not run sync code async"
tests/callback: ensure missing conf causes errors
storage/callback: do not run sync code async
tests/callback: added callback-specific tests
storage/callback: async Volume.export() & added Volume.export_end()
storage/lvm: make the "hack" work with CallbackPool instances
storage/callback: add the backend_class property
tests/callback: add them to the rpm build
storage/callback: various fixes
tests/callback: add rudimentary tests for the callback driver
tests/lvm: make the tests re-usable for other drivers
storage/callback: fix issues detected by pylint
storage/callback: volume callbacks now also rceive the source volume as argument (if there's any)
...
Policy allows a VM with 'backup-restore-mgmt' tag to create VMs, and
then manage VMs with 'backup-restore-in-progress' tag (which is added by
AdminExtension, based on 'tag-created-vm-with' feature).
VM with 'backup-restore-mgmt' tag can also call qubes.RestoreById
service to a VM with 'backup-restore-storage' tag. This service allows
to retrieve backup archive.
QubesOS/qubes-issues#5310
Remove intermediate qubesd-query-fast proxy process.
This requires changing socket protocol to match what qrexec is sending
in the header.
FixesQubesOS/qubes-issues#3293
This should allow importing a volume and changing the size at the
same time, without performing the resize operation on original
volume first.
The internal API has been renamed to internal.vm.volume.ImportBegin
to avoid confusion, and for symmetry with ImportEnd.
See QubesOS/qubes-issues#5239.
Allow getting all the VM properties with one call. This greatly improve
performance of an applications retrieving many/all of them (qvm-ls,
qubes manager etc)
QubesOS/qubes-issues#5415FixesQubesOS/qubes-issues#3293
admin.pool.UsageDetails reports the usage data, unlike
admin.pool.Info, which should report the config/unchangeable data.
At the moment admin.Pool.Info still reports usage, to maintain
compatibility, but once all relevant tools are updated,
it should just return configuration data.
- Two new methods: .features.check_with_adminvm() and
.check_with_template_and_adminvm(). Common code refactored.
- Two new AdminAPI calls to take advantage of the methods:
- admin.vm.feature.CheckWithAdminVM
- admin.vm.feature.CheckWithTemplateAndAdminVM
- Features manager moved to separate module in anticipation of features
on app object in R5.0. The attribute Features.vm renamed to
Features.subject.
- Documentation, tests.
This enable two things:
1. Follow global clockvm setting, without adjusting qrexec policy.
2. Avoid starting clockvm by arbitrary VM.
FixesQubesOS/qubes-issues#3588
Allow using default feature value from netvm, not template. This makes
sense for network-related features like using tor, supporting ipv6 etc.
Similarly to check_with_template, expose it also on Admin API.
This qrexec is meant for services, which require some kind of
"registering" before use. After registering, the backend should invoke
this call with frontend as the intended destination, with the actual
service in argument of this call and the argument as the payload.
By default this qrexec is disabled by policy.
Signed-off-by: Wojtek Porczyk <woju@invisiblethingslab.com>
This one pool/volume property makes sense to change dynamically. There
may be more such properties, but lets be on the safe side and take
whitelist approach - allow only selected (just one for now), instead of
blacklisting any harmful ones.
QubesOS/qubes-issues#3256
Add public Admin API call to create Disposable VM that would be
automatically destroyed after shutdown. Do not keep this functionality
for qrexec-policy tool only.
Also, use admin.vm.Start there, instead of internal.vm.Start and
admin.vm.Kill instead of internal.vm.CleanupDispVM (this is enough,
because DispVM now have auto_cleanup property).
QubesOS/qubes-issues#2974
clock synchronization mechanism rewritten to use systemd-timesync instead of NtpDate; at the moment, requires:
- modifying /etc/qubes-rpc/policy/qubes.GetDate to redirect GetDate to designated clockvm
- enabling clocksync service in clockvm ( qvm-features clockvm-name service/clocksync true )
Works as specified in issue listed below, except for:
- each VM synces with clockvm after boot and every 6h
- clockvm synces time with the Internet using systemd-timesync
- dom0 synces itself with clockvm every 1h (using cron)
fixesQubesOS/qubes-issues#1230
This ease Admin API administration, and also adds checking if qrexec
policy + scripts matches actual Admin API methods implementation.
The idea is to classify every Admin API method as either local
read-only, local read-write, global read-only or global read-write.
Where local/global means affecting a single VM, or the whole system.
See QubesOS/qubes-issues#2871 for details.
FixesQubesOS/qubes-issues#2871
Implement this in two parts:
1. Permissions checks, getting a path from appropriate storage pool
2. Actual data import
The first part is done by qubesd in a standard way, but then, instead of
accepting all the data (which may be several GB), return a path to which
a shell script (in practice: `dd` command) will write the data.
Then the script call back to qubesd again to report success/failure and
qubesd response from that call is actually returned to the user.
This way we do not pass all the data through qubesd, but still can
control the process from there in a meaningful way. Note that the last
part (second call to qubesd) may perform all kind of verification (like
a signature check on the data, or so) and can also prevent VM from
starting (hooking also domain-pre-start event) from not verified image.
QubesOS/qubes-issues#2622
Install files in /etc/qubes-rpc for all methods defined in API
documentation, even if not yet implemented (qubesd will handle it
raising appropriate exception).
Use minimal program written in C (qubesd-query-fast), instead of
qubesd-query in python for performance reasons:
- a single qubesd-query run: ~300ms
- equivalent in shell (echo | nc -U): ~40ms
- qubesd-query-fast: ~20ms
Many tools makes multiple API calls, so performance here do matter. For
example qvm-ls (from VM) currently takes about 60s on a system with 24
VMs.
Also make use of `$include:` directive in policy file, to make it easier
defining a VM with full Admin API access.
QubesOS/qubes-issues#853
This all either have been migrated to core3, or is not needed anymore.
There is still qvm-tools directory with a few tools that needs to be
migrated, or installed as is.
