Commit Graph

5332 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
af2435c0d4
Make some properties default to template's value (if any)
Multiple properties are related to system installed inside the VM, so it
makes sense to have them the same for all the VMs based on the same
template. Modify default value getter to first try get the value from a
template (if any) and only if it fails, fallback to original default
value.
This change is made to those properties:
 - default_user (it was already this way)
 - kernel
 - kernelopts
 - maxmem
 - memory
 - qrexec_timeout
 - vcpus
 - virt_mode

This is especially useful for manually installed templates (like
Windows).

Related to QubesOS/qubes-issues#3585
2018-07-16 22:02:58 +02:00
Marek Marczykowski-Górecki
af7d54d388
Update windows-related feature requests
Handle 'os' feature - if it's Windows, then set rpc-clipboard feature.
Handle 'gui-emulated' feature - request for specifically stubdomain GUI.
With 'gui' feature it is only possible to enable gui-agent based on, or
disable GUI completely.
Handle 'default-user' - verify it for weird characters and set
'default_user' property (if wasn't already set).

QubesOS/qubes-issues#3585
2018-07-16 22:02:58 +02:00
Marek Marczykowski-Górecki
ce87451c73
Merge branch 'lvm-snapshots'
* lvm-snapshots:
  tests: fix handling app.pools iteration
  storage/lvm: add repr(ThinPool) for more meaningful test reports
  tests: adjust for variable volume path
  api/admin: expose volume path in admin.vm.volume.Info
  tests: LVM: import, list_volumes, volatile volume, snapshot volume
  tests: collect all SIGCHLD before cleaning event loop
  storage/lvm: use temporary volume for data import
  tests: ThinVolume.revert()
  tests: LVM volume naming migration, and new naming in general
  storage/lvm: improve handling interrupted commit
2018-07-16 16:40:35 +02:00
Marek Marczykowski-Górecki
be2465c1f9
Fix issues found by pylint 2.0
Resolve:
 - no-else-return
 - useless-object-inheritance
 - useless-return
 - consider-using-set-comprehension
 - consider-using-in
 - logging-not-lazy

Ignore:
 - not-an-iterable - false possitives for asyncio coroutines

Ignore all the above in qubespolicy/__init__.py, as the file will be
moved to separate repository (core-qrexec) - it already has a copy
there, don't desynchronize them.
2018-07-15 23:51:15 +02:00
Marek Marczykowski-Górecki
f8d17012c3
tests: fix loading tests for templates
Fix #209
2018-07-15 21:57:34 +02:00
Marek Marczykowski-Górecki
69e3018b94
tests: fix handling app.pools iteration 2018-07-15 21:31:48 +02:00
Marek Marczykowski-Górecki
2af1815ab7
storage/lvm: add repr(ThinPool) for more meaningful test reports 2018-07-15 21:30:04 +02:00
Marek Marczykowski-Górecki
e644378f18
tests: adjust for variable volume path
LVM volumes now have variable volume path. Compare strip path before
comparing content's hash in tests.
2018-07-12 16:52:48 +02:00
Marek Marczykowski-Górecki
d211a2771a
api/admin: expose volume path in admin.vm.volume.Info
Since (for LVM at least) path is dynamic now, add information about it
to volume info. This is not very useful outside of dom0, but in dom0 it
can be very useful for various scripts.
This will disclose current volume revision id, but it is already
possible to deduce it from snapshots list.
2018-07-12 16:52:48 +02:00
Marek Marczykowski-Górecki
4282a41fcb
tests: LVM: import, list_volumes, volatile volume, snapshot volume 2018-07-12 16:52:47 +02:00
Marek Marczykowski-Górecki
76c872a43a
tests: collect all SIGCHLD before cleaning event loop
On python 3.6.4 apparently it requires two callbacks runs to cleanup
stale SIGCHLD handlers.
2018-07-12 16:52:47 +02:00
Marek Marczykowski-Górecki
2b80f0c044
storage/lvm: use temporary volume for data import
Do not write directly to main volume, instead create temporary volume
and only commit it to the main one when operation is finished. This
solve multiple problems:
 - import operation can be aborted, without data loss
 - importing new data over existing volume will not leave traces of
previous content - especially when importing smaller volume to bigger
one
 - import operation can be reverted - it create separate revision,
similar to start/stop
 - easier to prevent qube from starting during import operation
 - template still can be used when importing new version

QubesOS/qubes-issues#2256
2018-07-12 16:52:46 +02:00
Marek Marczykowski-Górecki
aea0de35ad
tests: ThinVolume.revert() 2018-07-12 16:52:46 +02:00
Marek Marczykowski-Górecki
8cf9264283
tests: LVM volume naming migration, and new naming in general 2018-07-12 16:52:45 +02:00
Marek Marczykowski-Górecki
11c7b4bb51
storage/lvm: improve handling interrupted commit
First rename volume to backup revision, regardless of revisions_to_keep,
then rename -snap to current volume. And only then remove backup
revision (if exceed revisions_to_keep). This way even if commit
operation is interrupted, there is still a volume with the data.
This requires also adjusting few functions to actually fallback to most
recent backup revision if the current volume isn't found - create
_vid_current property for this purpose.
Also, use -snap volume for clone operation and commit it normally later.
This makes it safer to interrupt or even revert.

QubesOS/qubes-issues#2256
2018-07-12 16:10:44 +02:00
Marek Marczykowski-Górecki
39a9e4e422
Merge remote-tracking branch 'qubesos/pr/209'
* qubesos/pr/209:
  qubes/tests: change globals= to module= and fix syntax errors
  qubes/tests: use loadTestsFromNames for nose2 compat
2018-04-30 01:13:11 +02:00
Wojtek Porczyk
843bbdb2c5 qubes/tests: change globals= to module= and fix syntax errors 2018-04-24 18:30:55 +02:00
Marek Marczykowski-Górecki
2b2cdf40e7
version 4.0.27 2018-04-16 22:37:00 +02:00
Marek Marczykowski-Górecki
f9e34eff51
api/admin: fix volume info formatting
Newline was put in a wrong place.

Fixes QubesOS/qubes-issues#3831
2018-04-16 22:17:48 +02:00
Marek Marczykowski-Górecki
e145e85dba
version 4.0.26 2018-04-16 18:36:32 +02:00
Max Zinkus
6dd8a5e212
Match error output string format 2018-04-09 21:58:10 -07:00
Max Zinkus
65435dce78
Prevent qvm-clock-sync permissions exception
qvm-clock-sync fails with a python subprocess error on line 43 when the `date ... -s ...` fails due to insufficient privileges to set the date. This commit checks to see if the program has effective UID root before attempting to set the date.
2018-04-09 21:56:33 -07:00
Marek Marczykowski-Górecki
b168c4a6d7
api/admin: fix admin.vm.volume.Info when is_outdated() isn't supported
is_outdated() may be not supported by given volume pool driver. In that
case skip is_outdated information, instead of crashing the call.

Fixes QubesOS/qubes-issues#3767
2018-04-08 23:25:34 +02:00
Marek Marczykowski-Górecki
93b2424867
vm/qubesvm: fix missing icon handling in clone_disk_files()
Check for icon existence, not a directory for it.
2018-04-06 12:10:50 +02:00
Marek Marczykowski-Górecki
2706562a6f
Merge remote-tracking branch 'qubesos/pr/210'
* qubesos/pr/210:
  spec.in: remove useless condition on Fedora 17 which induces problem with Travis
  spec.in: add changelog placeholder
  Define backend_vmm macro
  doc: handle release definition when not being in a git repository
  Remove _builddir
  Create .spec.in and Source0
2018-04-06 02:49:27 +02:00
Marek Marczykowski-Górecki
8df90f1a86
tests: yet another fix for integration tests cleanup code
Before waiting for remaining tasks on event loop (including libvirt
events), make sure all destroyed objects are really destroyed. This is
especially important for libvirt connections, which gets cleaned up only
when appropriate destructor (__del__) register a cleanup callback and it
gets called by the loop.
2018-04-05 01:28:55 +02:00
Marek Marczykowski-Górecki
3726692530
tests: skip tests using netcat when it isn't installed 2018-04-05 01:28:54 +02:00
Frédéric Pierret
ecdaa157a9
spec.in: remove useless condition on Fedora 17 which induces problem with Travis 2018-04-04 16:24:21 -04:00
Frédéric Pierret
6fb14e8198
spec.in: add changelog placeholder 2018-04-03 23:12:15 +02:00
Frédéric Pierret
601e88854c
Define backend_vmm macro 2018-04-03 23:12:15 +02:00
Frédéric Pierret
ab131083af
doc: handle release definition when not being in a git repository 2018-04-03 23:12:14 +02:00
Frédéric Pierret
32fad062f1
Remove _builddir 2018-04-03 23:12:14 +02:00
Frédéric Pierret
fdea48aa1c
Create .spec.in and Source0 2018-04-03 23:12:14 +02:00
Marek Marczykowski-Górecki
2dee554ab7
vm/mix/net: make vm.gateway6 consistent with vm.gateway
Use VM's actual IP address as a gateway for other VMs, instead of
hardcoded link-local address. This is important for sys-net generated
ICMP diagnostics packets - those must _not_ have link-local source
address, otherwise wouldn't be properly forwarded back to the right VM.
2018-04-03 00:20:06 +02:00
Marek Marczykowski-Górecki
f4be284331
vm/qubesvm: handle libvirt reporting domain already dead when killing
If domain die when trying to kill it, qubesd may loose a race and try to
kill it anyway. Handle libvirt exception in that case and conver it to
QubesVMNotStartedError - as it would be if qubesd would win the race.

Fixes QubesOS/qubes-issues#3755
2018-04-02 23:56:03 +02:00
Marek Marczykowski-Górecki
1e9bf18bcf
Typo fix 2018-04-02 23:24:30 +02:00
Wojtek Porczyk
dfe7688158 qubes/tests: use loadTestsFromNames for nose2 compat 2018-03-30 03:05:01 +02:00
Marek Marczykowski-Górecki
3c2ac0fd41
version 4.0.25 2018-03-29 21:32:52 +02:00
Marek Marczykowski-Górecki
5e6c94bc35
Merge remote-tracking branch 'origin/tests-less-verbose'
* origin/tests-less-verbose:
  api/admin: do not log expected full tracebacks with ERROR level
2018-03-29 20:49:28 +02:00
Marek Marczykowski-Górecki
2c13ba9c48
storage/lvm: force default locale for lvm command cont.
Few more places where lvm commands are called.

Fixes QubesOS/qubes-issues#3753
2018-03-29 20:39:15 +02:00
Marek Marczykowski-Górecki
faca89875b
storage/lvm: force default locale for lvm command
Scripts do parse its output sometimes (especially `lvs`), so make sure
we always gets the same format, regardless of the environment. Including
decimal separator.

Fixes QubesOS/qubes-issues#3753
2018-03-29 00:53:17 +02:00
Marek Marczykowski-Górecki
bb7bb25d89
api/admin: do not log expected full tracebacks with ERROR level
Make tests output cleaner.
2018-03-23 02:26:05 +01:00
Marek Marczykowski-Górecki
d6b422cc36
Merge remote-tracking branch 'qubesos/pr/207'
* qubesos/pr/207:
  storage/reflink: strictly increasing revision ID
2018-03-22 01:54:38 +01:00
Marek Marczykowski-Górecki
74c1893ce3
tests: fix firewall tests after 6e8e48e3 "Avoid UTC datetime" 2018-03-22 01:47:41 +01:00
Rusty Bird
6a303760e9
storage/reflink: strictly increasing revision ID
Don't rely on timestamps to sort revisions - the clock can go backwards
due to time sync. Instead, use a monotonically increasing natural number
as the revision ID.

Old revision example: private.img@2018-01-02-03T04:05:06Z (ignored now)
New revision example: private.img.123@2018-01-02-03T04:05:06Z
2018-03-21 16:00:13 +00:00
Marek Marczykowski-Górecki
6469705196
Merge branch 'devel-storage-fixes'
* devel-storage-fixes:
  storage/file: use proper exception instead of assert
  storage/file: import data into temporary volume
  storage/lvm: check for LVM LV existence and type when creating ThinPool
  storage/lvm: fix size reporting just after creating LV
2018-03-21 02:06:25 +01:00
Marek Marczykowski-Górecki
5eceff84cb
storage/file: use proper exception instead of assert
Return readable message to the user.
2018-03-21 01:48:20 +01:00
Marek Marczykowski-Górecki
510fad9163
storage/file: import data into temporary volume
Similar to LVM changes, this fixes/improves multiple things:
 - no old data visible in the volume
 - failed import do not leave broken volume
 - parially imported data not visible to running VM

QubesOS/qubes-issues#3169
2018-03-21 01:48:07 +01:00
Marek Marczykowski-Górecki
e5413a3036
Merge branch 'storage-properties'
* storage-properties:
  storage: use None for size/usage properties if unknown
  tests: call search_pool_containing_dir with various dirs and pools
  storage: make DirectoryThinPool helper less verbose, add sudo
  api/admin: add 'included_in' to admin.pool.Info call
  storage: add Pool.included_in() method for checking nested pools
  storage: move and generalize RootThinPool helper class
  storage/kernels: refuse changes to 'rw' and 'revisions_to_keep'
  api/admin: implement admin.vm.volume.Set.rw method
  api/admin: include 'revisions_to_keep' and 'is_outdated' in volume info
2018-03-21 01:43:53 +01:00
Marek Marczykowski-Górecki
03dc3e315e
storage: use None for size/usage properties if unknown
Raising NotImplementedError in a _property_ is weird behaviour, better
suited for actions (methods). Use None instead.

QubesOS/qubes-issues#3241
2018-03-20 17:31:16 +01:00