Commit Graph

5884 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
e644378f18
tests: adjust for variable volume path
LVM volumes now have variable volume path. Compare strip path before
comparing content's hash in tests.
2018-07-12 16:52:48 +02:00
Marek Marczykowski-Górecki
d211a2771a
api/admin: expose volume path in admin.vm.volume.Info
Since (for LVM at least) path is dynamic now, add information about it
to volume info. This is not very useful outside of dom0, but in dom0 it
can be very useful for various scripts.
This will disclose current volume revision id, but it is already
possible to deduce it from snapshots list.
2018-07-12 16:52:48 +02:00
Marek Marczykowski-Górecki
4282a41fcb
tests: LVM: import, list_volumes, volatile volume, snapshot volume 2018-07-12 16:52:47 +02:00
Marek Marczykowski-Górecki
76c872a43a
tests: collect all SIGCHLD before cleaning event loop
On python 3.6.4 apparently it requires two callbacks runs to cleanup
stale SIGCHLD handlers.
2018-07-12 16:52:47 +02:00
Marek Marczykowski-Górecki
2b80f0c044
storage/lvm: use temporary volume for data import
Do not write directly to main volume, instead create temporary volume
and only commit it to the main one when operation is finished. This
solve multiple problems:
 - import operation can be aborted, without data loss
 - importing new data over existing volume will not leave traces of
previous content - especially when importing smaller volume to bigger
one
 - import operation can be reverted - it create separate revision,
similar to start/stop
 - easier to prevent qube from starting during import operation
 - template still can be used when importing new version

QubesOS/qubes-issues#2256
2018-07-12 16:52:46 +02:00
Marek Marczykowski-Górecki
aea0de35ad
tests: ThinVolume.revert() 2018-07-12 16:52:46 +02:00
Marek Marczykowski-Górecki
8cf9264283
tests: LVM volume naming migration, and new naming in general 2018-07-12 16:52:45 +02:00
Marek Marczykowski-Górecki
11c7b4bb51
storage/lvm: improve handling interrupted commit
First rename volume to backup revision, regardless of revisions_to_keep,
then rename -snap to current volume. And only then remove backup
revision (if exceed revisions_to_keep). This way even if commit
operation is interrupted, there is still a volume with the data.
This requires also adjusting few functions to actually fallback to most
recent backup revision if the current volume isn't found - create
_vid_current property for this purpose.
Also, use -snap volume for clone operation and commit it normally later.
This makes it safer to interrupt or even revert.

QubesOS/qubes-issues#2256
2018-07-12 16:10:44 +02:00
Wojtek Porczyk
0862ce8a1f doc: loading graph
fixes QubesOS/qubes-issues#1560
2018-06-12 02:28:58 +02:00
Wojtek Porczyk
4e49b951ce Do not use assert statement in security logic
This is because assert statement gets optimised out when Python is run
with -O flag. This was pointed out to me by audience at PyWaw 76.
2018-06-11 12:32:05 +02:00
Marek Marczykowski-Górecki
39a9e4e422
Merge remote-tracking branch 'qubesos/pr/209'
* qubesos/pr/209:
  qubes/tests: change globals= to module= and fix syntax errors
  qubes/tests: use loadTestsFromNames for nose2 compat
2018-04-30 01:13:11 +02:00
Wojtek Porczyk
843bbdb2c5 qubes/tests: change globals= to module= and fix syntax errors 2018-04-24 18:30:55 +02:00
Marek Marczykowski-Górecki
2b2cdf40e7
version 4.0.27 2018-04-16 22:37:00 +02:00
Marek Marczykowski-Górecki
f9e34eff51
api/admin: fix volume info formatting
Newline was put in a wrong place.

Fixes QubesOS/qubes-issues#3831
2018-04-16 22:17:48 +02:00
Marek Marczykowski-Górecki
e145e85dba
version 4.0.26 2018-04-16 18:36:32 +02:00
Marek Marczykowski-Górecki
bb40d61af9
storage/lvm: filter out warning about intended over-provisioning
Over-provisioning on LVM is intended. Since LVM do not have any option
to disable it (see [1] and discussion linked from there), filter the
warning in post-processing.

[1] https://bugzilla.redhat.com/1347008

Fixes QubesOS/qubes-issues#3744
2018-04-14 21:36:03 +02:00
Marek Marczykowski-Górecki
4794232745
tests: fix getting kernel package version inside VM
Use `sort -V` instead of `sort -n`.
2018-04-13 21:44:44 +02:00
Marek Marczykowski-Górecki
69f19bb7bb
tests/extra: add start_guid option to VMWrapper
Pass start_guid option to vm.start(), when using core2 compatibility
layer.
2018-04-13 21:43:15 +02:00
Marek Marczykowski-Górecki
6a191febc3
vm/qubesvm: fire 'domain-start-failed' event even if fail was early
Fire 'domain-start-failed' even even if failure occurred during
'domain-pre-start' event. This will make sure if _anyone_ have seen
'domain-pre-start' event, will also see 'domain-start-failed'. In some
cases it will look like spurious 'domain-start-failed', but it is safer
option than the alternative.
2018-04-13 16:07:32 +02:00
Marek Marczykowski-Górecki
ba82d9dc21
vm/qubesvm: check if all required devices are available before start
Fail the VM start early if some persistently-assigned device is missing.
This will both save time and provide clearer error message.

Fixes QubesOS/qubes-issues#3810
2018-04-13 16:03:42 +02:00
Marek Marczykowski-Górecki
2aa14623bf
storage/lvm: fix reporting lvm command error
Escape '%' in error message, as required by Admin API.

Fixes QubesOS/qubes-issues#3809
2018-04-13 15:57:09 +02:00
Marek Marczykowski-Górecki
94c8e25d3c
storage/lvm: save pool's revision_to_keep property
And also report it as part of admin.pool.Info Admin API.

QubesOS/qubes-issues#3256
2018-04-13 15:56:23 +02:00
Max Zinkus
6dd8a5e212
Match error output string format 2018-04-09 21:58:10 -07:00
Max Zinkus
65435dce78
Prevent qvm-clock-sync permissions exception
qvm-clock-sync fails with a python subprocess error on line 43 when the `date ... -s ...` fails due to insufficient privileges to set the date. This commit checks to see if the program has effective UID root before attempting to set the date.
2018-04-09 21:56:33 -07:00
Marek Marczykowski-Górecki
b168c4a6d7
api/admin: fix admin.vm.volume.Info when is_outdated() isn't supported
is_outdated() may be not supported by given volume pool driver. In that
case skip is_outdated information, instead of crashing the call.

Fixes QubesOS/qubes-issues#3767
2018-04-08 23:25:34 +02:00
Marek Marczykowski-Górecki
93b2424867
vm/qubesvm: fix missing icon handling in clone_disk_files()
Check for icon existence, not a directory for it.
2018-04-06 12:10:50 +02:00
Marek Marczykowski-Górecki
2706562a6f
Merge remote-tracking branch 'qubesos/pr/210'
* qubesos/pr/210:
  spec.in: remove useless condition on Fedora 17 which induces problem with Travis
  spec.in: add changelog placeholder
  Define backend_vmm macro
  doc: handle release definition when not being in a git repository
  Remove _builddir
  Create .spec.in and Source0
2018-04-06 02:49:27 +02:00
Marek Marczykowski-Górecki
8df90f1a86
tests: yet another fix for integration tests cleanup code
Before waiting for remaining tasks on event loop (including libvirt
events), make sure all destroyed objects are really destroyed. This is
especially important for libvirt connections, which gets cleaned up only
when appropriate destructor (__del__) register a cleanup callback and it
gets called by the loop.
2018-04-05 01:28:55 +02:00
Marek Marczykowski-Górecki
3726692530
tests: skip tests using netcat when it isn't installed 2018-04-05 01:28:54 +02:00
Frédéric Pierret
ecdaa157a9
spec.in: remove useless condition on Fedora 17 which induces problem with Travis 2018-04-04 16:24:21 -04:00
Frédéric Pierret
6fb14e8198
spec.in: add changelog placeholder 2018-04-03 23:12:15 +02:00
Frédéric Pierret
601e88854c
Define backend_vmm macro 2018-04-03 23:12:15 +02:00
Frédéric Pierret
ab131083af
doc: handle release definition when not being in a git repository 2018-04-03 23:12:14 +02:00
Frédéric Pierret
32fad062f1
Remove _builddir 2018-04-03 23:12:14 +02:00
Frédéric Pierret
fdea48aa1c
Create .spec.in and Source0 2018-04-03 23:12:14 +02:00
Marek Marczykowski-Górecki
2dee554ab7
vm/mix/net: make vm.gateway6 consistent with vm.gateway
Use VM's actual IP address as a gateway for other VMs, instead of
hardcoded link-local address. This is important for sys-net generated
ICMP diagnostics packets - those must _not_ have link-local source
address, otherwise wouldn't be properly forwarded back to the right VM.
2018-04-03 00:20:06 +02:00
Marek Marczykowski-Górecki
f4be284331
vm/qubesvm: handle libvirt reporting domain already dead when killing
If domain die when trying to kill it, qubesd may loose a race and try to
kill it anyway. Handle libvirt exception in that case and conver it to
QubesVMNotStartedError - as it would be if qubesd would win the race.

Fixes QubesOS/qubes-issues#3755
2018-04-02 23:56:03 +02:00
Marek Marczykowski-Górecki
1e9bf18bcf
Typo fix 2018-04-02 23:24:30 +02:00
Wojtek Porczyk
dfe7688158 qubes/tests: use loadTestsFromNames for nose2 compat 2018-03-30 03:05:01 +02:00
Marek Marczykowski-Górecki
3c2ac0fd41
version 4.0.25 2018-03-29 21:32:52 +02:00
Marek Marczykowski-Górecki
5e6c94bc35
Merge remote-tracking branch 'origin/tests-less-verbose'
* origin/tests-less-verbose:
  api/admin: do not log expected full tracebacks with ERROR level
2018-03-29 20:49:28 +02:00
Marek Marczykowski-Górecki
2c13ba9c48
storage/lvm: force default locale for lvm command cont.
Few more places where lvm commands are called.

Fixes QubesOS/qubes-issues#3753
2018-03-29 20:39:15 +02:00
Marek Marczykowski-Górecki
faca89875b
storage/lvm: force default locale for lvm command
Scripts do parse its output sometimes (especially `lvs`), so make sure
we always gets the same format, regardless of the environment. Including
decimal separator.

Fixes QubesOS/qubes-issues#3753
2018-03-29 00:53:17 +02:00
Marek Marczykowski-Górecki
bb7bb25d89
api/admin: do not log expected full tracebacks with ERROR level
Make tests output cleaner.
2018-03-23 02:26:05 +01:00
Marek Marczykowski-Górecki
d6b422cc36
Merge remote-tracking branch 'qubesos/pr/207'
* qubesos/pr/207:
  storage/reflink: strictly increasing revision ID
2018-03-22 01:54:38 +01:00
Marek Marczykowski-Górecki
74c1893ce3
tests: fix firewall tests after 6e8e48e3 "Avoid UTC datetime" 2018-03-22 01:47:41 +01:00
Rusty Bird
6a303760e9
storage/reflink: strictly increasing revision ID
Don't rely on timestamps to sort revisions - the clock can go backwards
due to time sync. Instead, use a monotonically increasing natural number
as the revision ID.

Old revision example: private.img@2018-01-02-03T04:05:06Z (ignored now)
New revision example: private.img.123@2018-01-02-03T04:05:06Z
2018-03-21 16:00:13 +00:00
Marek Marczykowski-Górecki
6469705196
Merge branch 'devel-storage-fixes'
* devel-storage-fixes:
  storage/file: use proper exception instead of assert
  storage/file: import data into temporary volume
  storage/lvm: check for LVM LV existence and type when creating ThinPool
  storage/lvm: fix size reporting just after creating LV
2018-03-21 02:06:25 +01:00
Marek Marczykowski-Górecki
5eceff84cb
storage/file: use proper exception instead of assert
Return readable message to the user.
2018-03-21 01:48:20 +01:00
Marek Marczykowski-Górecki
510fad9163
storage/file: import data into temporary volume
Similar to LVM changes, this fixes/improves multiple things:
 - no old data visible in the volume
 - failed import do not leave broken volume
 - parially imported data not visible to running VM

QubesOS/qubes-issues#3169
2018-03-21 01:48:07 +01:00