Commit Graph

1523 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
39a9e4e422
Merge remote-tracking branch 'qubesos/pr/209'
* qubesos/pr/209:
  qubes/tests: change globals= to module= and fix syntax errors
  qubes/tests: use loadTestsFromNames for nose2 compat
2018-04-30 01:13:11 +02:00
Wojtek Porczyk
843bbdb2c5 qubes/tests: change globals= to module= and fix syntax errors 2018-04-24 18:30:55 +02:00
Marek Marczykowski-Górecki
f9e34eff51
api/admin: fix volume info formatting
Newline was put in a wrong place.

Fixes QubesOS/qubes-issues#3831
2018-04-16 22:17:48 +02:00
Marek Marczykowski-Górecki
b168c4a6d7
api/admin: fix admin.vm.volume.Info when is_outdated() isn't supported
is_outdated() may be not supported by given volume pool driver. In that
case skip is_outdated information, instead of crashing the call.

Fixes QubesOS/qubes-issues#3767
2018-04-08 23:25:34 +02:00
Marek Marczykowski-Górecki
93b2424867
vm/qubesvm: fix missing icon handling in clone_disk_files()
Check for icon existence, not a directory for it.
2018-04-06 12:10:50 +02:00
Marek Marczykowski-Górecki
8df90f1a86
tests: yet another fix for integration tests cleanup code
Before waiting for remaining tasks on event loop (including libvirt
events), make sure all destroyed objects are really destroyed. This is
especially important for libvirt connections, which gets cleaned up only
when appropriate destructor (__del__) register a cleanup callback and it
gets called by the loop.
2018-04-05 01:28:55 +02:00
Marek Marczykowski-Górecki
3726692530
tests: skip tests using netcat when it isn't installed 2018-04-05 01:28:54 +02:00
Marek Marczykowski-Górecki
2dee554ab7
vm/mix/net: make vm.gateway6 consistent with vm.gateway
Use VM's actual IP address as a gateway for other VMs, instead of
hardcoded link-local address. This is important for sys-net generated
ICMP diagnostics packets - those must _not_ have link-local source
address, otherwise wouldn't be properly forwarded back to the right VM.
2018-04-03 00:20:06 +02:00
Marek Marczykowski-Górecki
f4be284331
vm/qubesvm: handle libvirt reporting domain already dead when killing
If domain die when trying to kill it, qubesd may loose a race and try to
kill it anyway. Handle libvirt exception in that case and conver it to
QubesVMNotStartedError - as it would be if qubesd would win the race.

Fixes QubesOS/qubes-issues#3755
2018-04-02 23:56:03 +02:00
Marek Marczykowski-Górecki
1e9bf18bcf
Typo fix 2018-04-02 23:24:30 +02:00
Wojtek Porczyk
dfe7688158 qubes/tests: use loadTestsFromNames for nose2 compat 2018-03-30 03:05:01 +02:00
Marek Marczykowski-Górecki
5e6c94bc35
Merge remote-tracking branch 'origin/tests-less-verbose'
* origin/tests-less-verbose:
  api/admin: do not log expected full tracebacks with ERROR level
2018-03-29 20:49:28 +02:00
Marek Marczykowski-Górecki
2c13ba9c48
storage/lvm: force default locale for lvm command cont.
Few more places where lvm commands are called.

Fixes QubesOS/qubes-issues#3753
2018-03-29 20:39:15 +02:00
Marek Marczykowski-Górecki
faca89875b
storage/lvm: force default locale for lvm command
Scripts do parse its output sometimes (especially `lvs`), so make sure
we always gets the same format, regardless of the environment. Including
decimal separator.

Fixes QubesOS/qubes-issues#3753
2018-03-29 00:53:17 +02:00
Marek Marczykowski-Górecki
bb7bb25d89
api/admin: do not log expected full tracebacks with ERROR level
Make tests output cleaner.
2018-03-23 02:26:05 +01:00
Marek Marczykowski-Górecki
d6b422cc36
Merge remote-tracking branch 'qubesos/pr/207'
* qubesos/pr/207:
  storage/reflink: strictly increasing revision ID
2018-03-22 01:54:38 +01:00
Marek Marczykowski-Górecki
74c1893ce3
tests: fix firewall tests after 6e8e48e3 "Avoid UTC datetime" 2018-03-22 01:47:41 +01:00
Rusty Bird
6a303760e9
storage/reflink: strictly increasing revision ID
Don't rely on timestamps to sort revisions - the clock can go backwards
due to time sync. Instead, use a monotonically increasing natural number
as the revision ID.

Old revision example: private.img@2018-01-02-03T04:05:06Z (ignored now)
New revision example: private.img.123@2018-01-02-03T04:05:06Z
2018-03-21 16:00:13 +00:00
Marek Marczykowski-Górecki
6469705196
Merge branch 'devel-storage-fixes'
* devel-storage-fixes:
  storage/file: use proper exception instead of assert
  storage/file: import data into temporary volume
  storage/lvm: check for LVM LV existence and type when creating ThinPool
  storage/lvm: fix size reporting just after creating LV
2018-03-21 02:06:25 +01:00
Marek Marczykowski-Górecki
5eceff84cb
storage/file: use proper exception instead of assert
Return readable message to the user.
2018-03-21 01:48:20 +01:00
Marek Marczykowski-Górecki
510fad9163
storage/file: import data into temporary volume
Similar to LVM changes, this fixes/improves multiple things:
 - no old data visible in the volume
 - failed import do not leave broken volume
 - parially imported data not visible to running VM

QubesOS/qubes-issues#3169
2018-03-21 01:48:07 +01:00
Marek Marczykowski-Górecki
e5413a3036
Merge branch 'storage-properties'
* storage-properties:
  storage: use None for size/usage properties if unknown
  tests: call search_pool_containing_dir with various dirs and pools
  storage: make DirectoryThinPool helper less verbose, add sudo
  api/admin: add 'included_in' to admin.pool.Info call
  storage: add Pool.included_in() method for checking nested pools
  storage: move and generalize RootThinPool helper class
  storage/kernels: refuse changes to 'rw' and 'revisions_to_keep'
  api/admin: implement admin.vm.volume.Set.rw method
  api/admin: include 'revisions_to_keep' and 'is_outdated' in volume info
2018-03-21 01:43:53 +01:00
Marek Marczykowski-Górecki
03dc3e315e
storage: use None for size/usage properties if unknown
Raising NotImplementedError in a _property_ is weird behaviour, better
suited for actions (methods). Use None instead.

QubesOS/qubes-issues#3241
2018-03-20 17:31:16 +01:00
Marek Marczykowski-Górecki
05c80c4531
tests: call search_pool_containing_dir with various dirs and pools
QubesOS/qubes-issues#3241
2018-03-20 16:53:41 +01:00
Marek Marczykowski-Górecki
825de49767
storage: make DirectoryThinPool helper less verbose, add sudo
Don't print scary messages when given pool cannot be found. Also, add
sudo to make it work from non-root user (tests)
2018-03-20 16:53:41 +01:00
Marek Marczykowski-Górecki
1bc640f3e0
api/admin: add 'included_in' to admin.pool.Info call
QubesOS/qubes-issues#3240
QubesOS/qubes-issues#3241
2018-03-20 16:53:40 +01:00
Marek Marczykowski-Górecki
d40fae9756
storage: add Pool.included_in() method for checking nested pools
It may happen that one pool is inside a volume of other pool. This is
the case for example for varlibqubes pool (file driver,
dir_path=/var/lib/qubes) and default lvm pool (lvm_thin driver). The
latter include whole root filesystem, so /var/lib/qubes too.
This is relevant for proper disk space calculation - to not count some
space twice.

QubesOS/qubes-issues#3240
QubesOS/qubes-issues#3241
2018-03-20 16:53:39 +01:00
Marek Marczykowski-Górecki
99f430511a
storage: move and generalize RootThinPool helper class
This is a class for finding thin pool containing root filesytem.
Generalize it to work for other filesystems too and rename to
DirectoryThinPool.
2018-03-20 16:52:48 +01:00
Marek Marczykowski-Górecki
a0723a9e32
storage/kernels: refuse changes to 'rw' and 'revisions_to_keep'
This pool driver support only rw=False and revisions_to_keep=0 volumes.
Since there is API for changing those properties dynamically, block it
at pool driver level, instead of silently ignoring them.
2018-03-20 16:52:46 +01:00
Marek Marczykowski-Górecki
376c8ec00d
api/admin: implement admin.vm.volume.Set.rw method
Allow setting 'rw' volume property.
2018-03-20 16:52:06 +01:00
Marek Marczykowski-Górecki
b66a2e9a51
Merge remote-tracking branch 'qubesos/pr/202'
* qubesos/pr/202:
  api/admin: fix backup exclude
2018-03-20 14:44:20 +01:00
Rusty Bird
a0c5014737
api/admin: fix backup exclude
Bugfix on 59abdeb
2018-03-20 02:41:37 +00:00
Marek Marczykowski-Górecki
93bccf583e
Merge remote-tracking branch 'qubesos/pr/200'
* qubesos/pr/200:
  Removed self.rules != old_rules
  Avoid UTC datetime
  Wrong init var to bool and missing call to total_seconds()

Fixes QubesOS/qubes-issues#3661
2018-03-20 01:19:58 +01:00
Marek Marczykowski-Górecki
a3a6a462f3
Merge remote-tracking branch 'qubesos/pr/201'
* qubesos/pr/201:
  storage/reflink: reorder start() to be more readable
  storage/reflink: simplify
  storage/reflink: let _remove_empty_dir() ignore ENOTEMPTY
  storage/reflink: show size in refused volume shrink message
  storage/reflink: fsync() after resizing existing file
2018-03-20 00:57:31 +01:00
Marek Marczykowski-Górecki
2a962c54db
api/admin: include 'revisions_to_keep' and 'is_outdated' in volume info
Since Volume.is_outdated() is a method, not a property, add a function
for handling serialization. And at the same time, fix None serialization
(applicable to 'source' property).

QubesOS/qubes-issues#3256
2018-03-19 04:00:21 +01:00
Rusty Bird
1743c76ca9
storage/reflink: reorder start() to be more readable
This also makes slightly more sense in the exotic (and currently unused)
case of restarting a crashed snap_on_start *and* save_on_stop volume.
2018-03-12 16:38:56 +00:00
Rusty Bird
31810db977
storage/reflink: simplify 2018-03-11 17:39:51 +00:00
Rusty Bird
c382eb3752
storage/reflink: let _remove_empty_dir() ignore ENOTEMPTY 2018-03-11 17:39:51 +00:00
Rusty Bird
023cb49293
storage/reflink: show size in refused volume shrink message
Like e6bb282 did for lvm.
2018-03-11 15:34:56 +00:00
Rusty Bird
c31d317c63
storage/reflink: fsync() after resizing existing file
Ensure that the updated metadata is written to disk.
2018-03-11 15:34:55 +00:00
donoban
39d904ea82
Removed self.rules != old_rules
After lot of testing it does not work properly. Could do something more
sophisticated but since calling save() is safe and probably lightweigth it is
not worth probably.
2018-03-08 05:25:42 -05:00
donoban
6e8e48e32d
Avoid UTC datetime
utcfromtimestamp() does not seems reliable and qubes-manager uses local time
2018-03-07 16:40:07 -05:00
donoban
b3b18f97f8
Wrong init var to bool and missing call to total_seconds()
fix https://github.com/QubesOS/qubes-issues/issues/3661
2018-03-07 16:37:44 -05:00
Marek Marczykowski-Górecki
e2b70306e5
Fix error message when using invalid VM as a template for DispVM
Don't crash (producing misleading error) when checking if
template_for_dispvms=True.

Fixes QubesOS/qubes-issues#3341
2018-03-05 23:47:33 +01:00
Marek Marczykowski-Górecki
7c4566ec14
vm/qubesvm: allow 'features-request' to have async handlers
Some handlers may want to call into other VMs (or even the one asking),
but vm.run() functions are coroutines, so needs to be called from
another coroutine. Allow for that.
Also fix typo in documentation.
2018-03-02 01:16:38 +01:00
Marek Marczykowski-Górecki
b1e83f899f
tests: convert pvgrub tests to core3 API
QubesOS/qubes-issues#3563
2018-02-28 04:41:30 +01:00
Marek Marczykowski-Górecki
1774be1c69
storage/kernel: skip modules.img block device if kernel dir doesn't have it
Some kernels (like pvgrub2) may not provide modules.img and it isn't an
error. Don't break VM startup in that case, skip that device instead.

Fixes QubesOS/qubes-issues#3563
2018-02-27 04:35:05 +01:00
Marek Marczykowski-Górecki
ba5d19e1b4
vm: provide better error message for VM startup timeout
"Cannot execute qrexec-daemon!" error is very misleading for a startup
timeout error, make it clearer. This rely on qrexec-daemon using
distinct exit code for timeout error, but even without that, include its
stderr in the error message.
2018-02-27 04:35:05 +01:00
Marek Marczykowski-Górecki
173e7e4250
vm: fix calling vm.detach_network() when really needed
*oldvalue* argument to property event handler is provided only when the
value was not default. Check vm.netvm directly to resolve also default.
2018-02-26 02:52:27 +01:00
Marek Marczykowski-Górecki
92a460526f
tests: fix extra test's VMWrapper.run(wait=False)
The call was ignored.
2018-02-26 02:45:26 +01:00