Commit Graph

5527 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
11c7b4bb51
storage/lvm: improve handling interrupted commit
First rename volume to backup revision, regardless of revisions_to_keep,
then rename -snap to current volume. And only then remove backup
revision (if exceed revisions_to_keep). This way even if commit
operation is interrupted, there is still a volume with the data.
This requires also adjusting few functions to actually fallback to most
recent backup revision if the current volume isn't found - create
_vid_current property for this purpose.
Also, use -snap volume for clone operation and commit it normally later.
This makes it safer to interrupt or even revert.

QubesOS/qubes-issues#2256
2018-07-12 16:10:44 +02:00
Wojtek Porczyk
0862ce8a1f doc: loading graph
fixes QubesOS/qubes-issues#1560
2018-06-12 02:28:58 +02:00
Wojtek Porczyk
4e49b951ce Do not use assert statement in security logic
This is because assert statement gets optimised out when Python is run
with -O flag. This was pointed out to me by audience at PyWaw 76.
2018-06-11 12:32:05 +02:00
Marek Marczykowski-Górecki
39a9e4e422
Merge remote-tracking branch 'qubesos/pr/209'
* qubesos/pr/209:
  qubes/tests: change globals= to module= and fix syntax errors
  qubes/tests: use loadTestsFromNames for nose2 compat
2018-04-30 01:13:11 +02:00
Wojtek Porczyk
843bbdb2c5 qubes/tests: change globals= to module= and fix syntax errors 2018-04-24 18:30:55 +02:00
Marek Marczykowski-Górecki
2b2cdf40e7
version 4.0.27 2018-04-16 22:37:00 +02:00
Marek Marczykowski-Górecki
f9e34eff51
api/admin: fix volume info formatting
Newline was put in a wrong place.

Fixes QubesOS/qubes-issues#3831
2018-04-16 22:17:48 +02:00
Marek Marczykowski-Górecki
e145e85dba
version 4.0.26 2018-04-16 18:36:32 +02:00
Marek Marczykowski-Górecki
bb40d61af9
storage/lvm: filter out warning about intended over-provisioning
Over-provisioning on LVM is intended. Since LVM do not have any option
to disable it (see [1] and discussion linked from there), filter the
warning in post-processing.

[1] https://bugzilla.redhat.com/1347008

Fixes QubesOS/qubes-issues#3744
2018-04-14 21:36:03 +02:00
Marek Marczykowski-Górecki
4794232745
tests: fix getting kernel package version inside VM
Use `sort -V` instead of `sort -n`.
2018-04-13 21:44:44 +02:00
Marek Marczykowski-Górecki
69f19bb7bb
tests/extra: add start_guid option to VMWrapper
Pass start_guid option to vm.start(), when using core2 compatibility
layer.
2018-04-13 21:43:15 +02:00
Marek Marczykowski-Górecki
6a191febc3
vm/qubesvm: fire 'domain-start-failed' event even if fail was early
Fire 'domain-start-failed' even even if failure occurred during
'domain-pre-start' event. This will make sure if _anyone_ have seen
'domain-pre-start' event, will also see 'domain-start-failed'. In some
cases it will look like spurious 'domain-start-failed', but it is safer
option than the alternative.
2018-04-13 16:07:32 +02:00
Marek Marczykowski-Górecki
ba82d9dc21
vm/qubesvm: check if all required devices are available before start
Fail the VM start early if some persistently-assigned device is missing.
This will both save time and provide clearer error message.

Fixes QubesOS/qubes-issues#3810
2018-04-13 16:03:42 +02:00
Marek Marczykowski-Górecki
2aa14623bf
storage/lvm: fix reporting lvm command error
Escape '%' in error message, as required by Admin API.

Fixes QubesOS/qubes-issues#3809
2018-04-13 15:57:09 +02:00
Marek Marczykowski-Górecki
94c8e25d3c
storage/lvm: save pool's revision_to_keep property
And also report it as part of admin.pool.Info Admin API.

QubesOS/qubes-issues#3256
2018-04-13 15:56:23 +02:00
Max Zinkus
6dd8a5e212
Match error output string format 2018-04-09 21:58:10 -07:00
Max Zinkus
65435dce78
Prevent qvm-clock-sync permissions exception
qvm-clock-sync fails with a python subprocess error on line 43 when the `date ... -s ...` fails due to insufficient privileges to set the date. This commit checks to see if the program has effective UID root before attempting to set the date.
2018-04-09 21:56:33 -07:00
Marek Marczykowski-Górecki
b168c4a6d7
api/admin: fix admin.vm.volume.Info when is_outdated() isn't supported
is_outdated() may be not supported by given volume pool driver. In that
case skip is_outdated information, instead of crashing the call.

Fixes QubesOS/qubes-issues#3767
2018-04-08 23:25:34 +02:00
Marek Marczykowski-Górecki
93b2424867
vm/qubesvm: fix missing icon handling in clone_disk_files()
Check for icon existence, not a directory for it.
2018-04-06 12:10:50 +02:00
Marek Marczykowski-Górecki
2706562a6f
Merge remote-tracking branch 'qubesos/pr/210'
* qubesos/pr/210:
  spec.in: remove useless condition on Fedora 17 which induces problem with Travis
  spec.in: add changelog placeholder
  Define backend_vmm macro
  doc: handle release definition when not being in a git repository
  Remove _builddir
  Create .spec.in and Source0
2018-04-06 02:49:27 +02:00
Marek Marczykowski-Górecki
8df90f1a86
tests: yet another fix for integration tests cleanup code
Before waiting for remaining tasks on event loop (including libvirt
events), make sure all destroyed objects are really destroyed. This is
especially important for libvirt connections, which gets cleaned up only
when appropriate destructor (__del__) register a cleanup callback and it
gets called by the loop.
2018-04-05 01:28:55 +02:00
Marek Marczykowski-Górecki
3726692530
tests: skip tests using netcat when it isn't installed 2018-04-05 01:28:54 +02:00
Frédéric Pierret
ecdaa157a9
spec.in: remove useless condition on Fedora 17 which induces problem with Travis 2018-04-04 16:24:21 -04:00
Frédéric Pierret
6fb14e8198
spec.in: add changelog placeholder 2018-04-03 23:12:15 +02:00
Frédéric Pierret
601e88854c
Define backend_vmm macro 2018-04-03 23:12:15 +02:00
Frédéric Pierret
ab131083af
doc: handle release definition when not being in a git repository 2018-04-03 23:12:14 +02:00
Frédéric Pierret
32fad062f1
Remove _builddir 2018-04-03 23:12:14 +02:00
Frédéric Pierret
fdea48aa1c
Create .spec.in and Source0 2018-04-03 23:12:14 +02:00
Marek Marczykowski-Górecki
2dee554ab7
vm/mix/net: make vm.gateway6 consistent with vm.gateway
Use VM's actual IP address as a gateway for other VMs, instead of
hardcoded link-local address. This is important for sys-net generated
ICMP diagnostics packets - those must _not_ have link-local source
address, otherwise wouldn't be properly forwarded back to the right VM.
2018-04-03 00:20:06 +02:00
Marek Marczykowski-Górecki
f4be284331
vm/qubesvm: handle libvirt reporting domain already dead when killing
If domain die when trying to kill it, qubesd may loose a race and try to
kill it anyway. Handle libvirt exception in that case and conver it to
QubesVMNotStartedError - as it would be if qubesd would win the race.

Fixes QubesOS/qubes-issues#3755
2018-04-02 23:56:03 +02:00
Marek Marczykowski-Górecki
1e9bf18bcf
Typo fix 2018-04-02 23:24:30 +02:00
Wojtek Porczyk
dfe7688158 qubes/tests: use loadTestsFromNames for nose2 compat 2018-03-30 03:05:01 +02:00
Marek Marczykowski-Górecki
3c2ac0fd41
version 4.0.25 2018-03-29 21:32:52 +02:00
Marek Marczykowski-Górecki
5e6c94bc35
Merge remote-tracking branch 'origin/tests-less-verbose'
* origin/tests-less-verbose:
  api/admin: do not log expected full tracebacks with ERROR level
2018-03-29 20:49:28 +02:00
Marek Marczykowski-Górecki
2c13ba9c48
storage/lvm: force default locale for lvm command cont.
Few more places where lvm commands are called.

Fixes QubesOS/qubes-issues#3753
2018-03-29 20:39:15 +02:00
Marek Marczykowski-Górecki
faca89875b
storage/lvm: force default locale for lvm command
Scripts do parse its output sometimes (especially `lvs`), so make sure
we always gets the same format, regardless of the environment. Including
decimal separator.

Fixes QubesOS/qubes-issues#3753
2018-03-29 00:53:17 +02:00
Marek Marczykowski-Górecki
bb7bb25d89
api/admin: do not log expected full tracebacks with ERROR level
Make tests output cleaner.
2018-03-23 02:26:05 +01:00
Marek Marczykowski-Górecki
d6b422cc36
Merge remote-tracking branch 'qubesos/pr/207'
* qubesos/pr/207:
  storage/reflink: strictly increasing revision ID
2018-03-22 01:54:38 +01:00
Marek Marczykowski-Górecki
74c1893ce3
tests: fix firewall tests after 6e8e48e3 "Avoid UTC datetime" 2018-03-22 01:47:41 +01:00
Rusty Bird
6a303760e9
storage/reflink: strictly increasing revision ID
Don't rely on timestamps to sort revisions - the clock can go backwards
due to time sync. Instead, use a monotonically increasing natural number
as the revision ID.

Old revision example: private.img@2018-01-02-03T04:05:06Z (ignored now)
New revision example: private.img.123@2018-01-02-03T04:05:06Z
2018-03-21 16:00:13 +00:00
Marek Marczykowski-Górecki
6469705196
Merge branch 'devel-storage-fixes'
* devel-storage-fixes:
  storage/file: use proper exception instead of assert
  storage/file: import data into temporary volume
  storage/lvm: check for LVM LV existence and type when creating ThinPool
  storage/lvm: fix size reporting just after creating LV
2018-03-21 02:06:25 +01:00
Marek Marczykowski-Górecki
5eceff84cb
storage/file: use proper exception instead of assert
Return readable message to the user.
2018-03-21 01:48:20 +01:00
Marek Marczykowski-Górecki
510fad9163
storage/file: import data into temporary volume
Similar to LVM changes, this fixes/improves multiple things:
 - no old data visible in the volume
 - failed import do not leave broken volume
 - parially imported data not visible to running VM

QubesOS/qubes-issues#3169
2018-03-21 01:48:07 +01:00
Marek Marczykowski-Górecki
e5413a3036
Merge branch 'storage-properties'
* storage-properties:
  storage: use None for size/usage properties if unknown
  tests: call search_pool_containing_dir with various dirs and pools
  storage: make DirectoryThinPool helper less verbose, add sudo
  api/admin: add 'included_in' to admin.pool.Info call
  storage: add Pool.included_in() method for checking nested pools
  storage: move and generalize RootThinPool helper class
  storage/kernels: refuse changes to 'rw' and 'revisions_to_keep'
  api/admin: implement admin.vm.volume.Set.rw method
  api/admin: include 'revisions_to_keep' and 'is_outdated' in volume info
2018-03-21 01:43:53 +01:00
Marek Marczykowski-Górecki
03dc3e315e
storage: use None for size/usage properties if unknown
Raising NotImplementedError in a _property_ is weird behaviour, better
suited for actions (methods). Use None instead.

QubesOS/qubes-issues#3241
2018-03-20 17:31:16 +01:00
Marek Marczykowski-Górecki
05c80c4531
tests: call search_pool_containing_dir with various dirs and pools
QubesOS/qubes-issues#3241
2018-03-20 16:53:41 +01:00
Marek Marczykowski-Górecki
825de49767
storage: make DirectoryThinPool helper less verbose, add sudo
Don't print scary messages when given pool cannot be found. Also, add
sudo to make it work from non-root user (tests)
2018-03-20 16:53:41 +01:00
Marek Marczykowski-Górecki
1bc640f3e0
api/admin: add 'included_in' to admin.pool.Info call
QubesOS/qubes-issues#3240
QubesOS/qubes-issues#3241
2018-03-20 16:53:40 +01:00
Marek Marczykowski-Górecki
d40fae9756
storage: add Pool.included_in() method for checking nested pools
It may happen that one pool is inside a volume of other pool. This is
the case for example for varlibqubes pool (file driver,
dir_path=/var/lib/qubes) and default lvm pool (lvm_thin driver). The
latter include whole root filesystem, so /var/lib/qubes too.
This is relevant for proper disk space calculation - to not count some
space twice.

QubesOS/qubes-issues#3240
QubesOS/qubes-issues#3241
2018-03-20 16:53:39 +01:00
Marek Marczykowski-Górecki
99f430511a
storage: move and generalize RootThinPool helper class
This is a class for finding thin pool containing root filesytem.
Generalize it to work for other filesystems too and rename to
DirectoryThinPool.
2018-03-20 16:52:48 +01:00