Commit Graph

5585 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
510fad9163
storage/file: import data into temporary volume
Similar to LVM changes, this fixes/improves multiple things:
 - no old data visible in the volume
 - failed import do not leave broken volume
 - parially imported data not visible to running VM

QubesOS/qubes-issues#3169
2018-03-21 01:48:07 +01:00
Marek Marczykowski-Górecki
e5413a3036
Merge branch 'storage-properties'
* storage-properties:
  storage: use None for size/usage properties if unknown
  tests: call search_pool_containing_dir with various dirs and pools
  storage: make DirectoryThinPool helper less verbose, add sudo
  api/admin: add 'included_in' to admin.pool.Info call
  storage: add Pool.included_in() method for checking nested pools
  storage: move and generalize RootThinPool helper class
  storage/kernels: refuse changes to 'rw' and 'revisions_to_keep'
  api/admin: implement admin.vm.volume.Set.rw method
  api/admin: include 'revisions_to_keep' and 'is_outdated' in volume info
2018-03-21 01:43:53 +01:00
Marek Marczykowski-Górecki
03dc3e315e
storage: use None for size/usage properties if unknown
Raising NotImplementedError in a _property_ is weird behaviour, better
suited for actions (methods). Use None instead.

QubesOS/qubes-issues#3241
2018-03-20 17:31:16 +01:00
Marek Marczykowski-Górecki
05c80c4531
tests: call search_pool_containing_dir with various dirs and pools
QubesOS/qubes-issues#3241
2018-03-20 16:53:41 +01:00
Marek Marczykowski-Górecki
825de49767
storage: make DirectoryThinPool helper less verbose, add sudo
Don't print scary messages when given pool cannot be found. Also, add
sudo to make it work from non-root user (tests)
2018-03-20 16:53:41 +01:00
Marek Marczykowski-Górecki
1bc640f3e0
api/admin: add 'included_in' to admin.pool.Info call
QubesOS/qubes-issues#3240
QubesOS/qubes-issues#3241
2018-03-20 16:53:40 +01:00
Marek Marczykowski-Górecki
d40fae9756
storage: add Pool.included_in() method for checking nested pools
It may happen that one pool is inside a volume of other pool. This is
the case for example for varlibqubes pool (file driver,
dir_path=/var/lib/qubes) and default lvm pool (lvm_thin driver). The
latter include whole root filesystem, so /var/lib/qubes too.
This is relevant for proper disk space calculation - to not count some
space twice.

QubesOS/qubes-issues#3240
QubesOS/qubes-issues#3241
2018-03-20 16:53:39 +01:00
Marek Marczykowski-Górecki
99f430511a
storage: move and generalize RootThinPool helper class
This is a class for finding thin pool containing root filesytem.
Generalize it to work for other filesystems too and rename to
DirectoryThinPool.
2018-03-20 16:52:48 +01:00
Marek Marczykowski-Górecki
a0723a9e32
storage/kernels: refuse changes to 'rw' and 'revisions_to_keep'
This pool driver support only rw=False and revisions_to_keep=0 volumes.
Since there is API for changing those properties dynamically, block it
at pool driver level, instead of silently ignoring them.
2018-03-20 16:52:46 +01:00
Marek Marczykowski-Górecki
376c8ec00d
api/admin: implement admin.vm.volume.Set.rw method
Allow setting 'rw' volume property.
2018-03-20 16:52:06 +01:00
Marek Marczykowski-Górecki
b66a2e9a51
Merge remote-tracking branch 'qubesos/pr/202'
* qubesos/pr/202:
  api/admin: fix backup exclude
2018-03-20 14:44:20 +01:00
Rusty Bird
a0c5014737
api/admin: fix backup exclude
Bugfix on 59abdeb
2018-03-20 02:41:37 +00:00
Marek Marczykowski-Górecki
93bccf583e
Merge remote-tracking branch 'qubesos/pr/200'
* qubesos/pr/200:
  Removed self.rules != old_rules
  Avoid UTC datetime
  Wrong init var to bool and missing call to total_seconds()

Fixes QubesOS/qubes-issues#3661
2018-03-20 01:19:58 +01:00
Marek Marczykowski-Górecki
a3a6a462f3
Merge remote-tracking branch 'qubesos/pr/201'
* qubesos/pr/201:
  storage/reflink: reorder start() to be more readable
  storage/reflink: simplify
  storage/reflink: let _remove_empty_dir() ignore ENOTEMPTY
  storage/reflink: show size in refused volume shrink message
  storage/reflink: fsync() after resizing existing file
2018-03-20 00:57:31 +01:00
Marek Marczykowski-Górecki
2a962c54db
api/admin: include 'revisions_to_keep' and 'is_outdated' in volume info
Since Volume.is_outdated() is a method, not a property, add a function
for handling serialization. And at the same time, fix None serialization
(applicable to 'source' property).

QubesOS/qubes-issues#3256
2018-03-19 04:00:21 +01:00
Rusty Bird
1743c76ca9
storage/reflink: reorder start() to be more readable
This also makes slightly more sense in the exotic (and currently unused)
case of restarting a crashed snap_on_start *and* save_on_stop volume.
2018-03-12 16:38:56 +00:00
Rusty Bird
31810db977
storage/reflink: simplify 2018-03-11 17:39:51 +00:00
Rusty Bird
c382eb3752
storage/reflink: let _remove_empty_dir() ignore ENOTEMPTY 2018-03-11 17:39:51 +00:00
Rusty Bird
023cb49293
storage/reflink: show size in refused volume shrink message
Like e6bb282 did for lvm.
2018-03-11 15:34:56 +00:00
Rusty Bird
c31d317c63
storage/reflink: fsync() after resizing existing file
Ensure that the updated metadata is written to disk.
2018-03-11 15:34:55 +00:00
donoban
39d904ea82
Removed self.rules != old_rules
After lot of testing it does not work properly. Could do something more
sophisticated but since calling save() is safe and probably lightweigth it is
not worth probably.
2018-03-08 05:25:42 -05:00
donoban
6e8e48e32d
Avoid UTC datetime
utcfromtimestamp() does not seems reliable and qubes-manager uses local time
2018-03-07 16:40:07 -05:00
donoban
b3b18f97f8
Wrong init var to bool and missing call to total_seconds()
fix https://github.com/QubesOS/qubes-issues/issues/3661
2018-03-07 16:37:44 -05:00
Marek Marczykowski-Górecki
e2b70306e5
Fix error message when using invalid VM as a template for DispVM
Don't crash (producing misleading error) when checking if
template_for_dispvms=True.

Fixes QubesOS/qubes-issues#3341
2018-03-05 23:47:33 +01:00
Marek Marczykowski-Górecki
a8784df349
version 4.0.24 2018-03-04 04:00:13 +01:00
Marek Marczykowski-Górecki
232a00a2b0
qubespolicy: translate '' target to '@default' earlier
Translate empty target to '@default' keyword earlier, so that
original_target will report '@default' instead of ''. The latter is
rejected by qubes-rpc-multiplexer when the call is directed to dom0,
because it expects to get non-empty arguments about original
target.
2018-03-03 03:50:59 +01:00
Marek Marczykowski-Górecki
bda9264e19
Add qubes.GetDate proxy service
This enable two things:
1. Follow global clockvm setting, without adjusting qrexec policy.
2. Avoid starting clockvm by arbitrary VM.

Fixes QubesOS/qubes-issues#3588
2018-03-03 03:50:58 +01:00
Marek Marczykowski-Górecki
7c4566ec14
vm/qubesvm: allow 'features-request' to have async handlers
Some handlers may want to call into other VMs (or even the one asking),
but vm.run() functions are coroutines, so needs to be called from
another coroutine. Allow for that.
Also fix typo in documentation.
2018-03-02 01:16:38 +01:00
Marek Marczykowski-Górecki
5840dd76f9
Add default policy for qubes.StartApp service
Fixes QubesOS/qubes-issues#3044
2018-02-28 04:41:45 +01:00
Marek Marczykowski-Górecki
b1e83f899f
tests: convert pvgrub tests to core3 API
QubesOS/qubes-issues#3563
2018-02-28 04:41:30 +01:00
Marek Marczykowski-Górecki
1774be1c69
storage/kernel: skip modules.img block device if kernel dir doesn't have it
Some kernels (like pvgrub2) may not provide modules.img and it isn't an
error. Don't break VM startup in that case, skip that device instead.

Fixes QubesOS/qubes-issues#3563
2018-02-27 04:35:05 +01:00
Marek Marczykowski-Górecki
ba5d19e1b4
vm: provide better error message for VM startup timeout
"Cannot execute qrexec-daemon!" error is very misleading for a startup
timeout error, make it clearer. This rely on qrexec-daemon using
distinct exit code for timeout error, but even without that, include its
stderr in the error message.
2018-02-27 04:35:05 +01:00
Marek Marczykowski-Górecki
173e7e4250
vm: fix calling vm.detach_network() when really needed
*oldvalue* argument to property event handler is provided only when the
value was not default. Check vm.netvm directly to resolve also default.
2018-02-26 02:52:27 +01:00
Marek Marczykowski-Górecki
92a460526f
tests: fix extra test's VMWrapper.run(wait=False)
The call was ignored.
2018-02-26 02:45:26 +01:00
Marek Marczykowski-Górecki
81aea0c2c7
Merge remote-tracking branch 'qubesos/pr/198'
* qubesos/pr/198:
  backup.py: add vmN/empty file if no other files to backup
  Allow include=None to be passed to admin.backup.Info
  Add include_in_backups property for AdminVM
  Use !auto_cleanup as DispVM include_in_backups default
2018-02-25 00:09:12 +01:00
Rusty Bird
6886e70aa9
backup.py: add vmN/empty file if no other files to backup 2018-02-24 22:43:53 +00:00
Rusty Bird
59abdeb30e
Allow include=None to be passed to admin.backup.Info
This is forwarded to qubes.backup.Backup(), which uses None to choose
VMs according to their include_in_backups property.

('git show -w' recommended)
2018-02-23 21:29:15 +00:00
Rusty Bird
dbaf60ca24
Add include_in_backups property for AdminVM 2018-02-23 21:29:15 +00:00
Rusty Bird
84ca0c6df5
Use !auto_cleanup as DispVM include_in_backups default 2018-02-23 21:29:15 +00:00
Marek Marczykowski-Górecki
716114f676
Merge remote-tracking branch 'qubesos/pr/197'
* qubesos/pr/197:
  Don't fire domain-stopped/-shutdown while VM is still Dying
2018-02-22 21:14:55 +01:00
Rusty Bird
f96fd70f76
Don't fire domain-stopped/-shutdown while VM is still Dying
Lots of code expects the VM to be Halted after receiving one of these
events, but it could also be Dying or Crashed. Get rid of the Dying case
at least, by waiting until the VM has transitioned out of it.

Fixes e.g. the following DispVM cleanup bug:

    $ qvm-create -C DispVM --prop auto_cleanup=True -l red dispvm
    $ qvm-start dispvm
    $ qvm-shutdown --wait dispvm  # this won't remove dispvm
    $ qvm-start dispvm
    $ qvm-kill dispvm  # but this will
2018-02-22 19:53:29 +00:00
Marek Marczykowski-Górecki
1562defd42
Merge remote-tracking branch 'qubesos/pr/193'
* qubesos/pr/193:
  tests: add a test for removing expired firewall rules
  firewall: use asyncio's call_later instead of systemd to reload rules
2018-02-22 19:47:37 +01:00
Marek Marczykowski-Górecki
7731a56daa
Merge remote-tracking branch 'qubesos/pr/192'
* qubesos/pr/192:
  storage/lvm: add more details to refused volume shrink message
  tests: storage/lvm resize
  storage/lvm: fix resizing not persistent volumes
2018-02-22 19:46:11 +01:00
Marek Marczykowski-Górecki
b283ab2b52
Merge remote-tracking branch 'qubesos/pr/195'
* qubesos/pr/195:
  Fix is_running non-boolean
2018-02-22 17:52:09 +01:00
Marek Marczykowski-Górecki
033100fa7e
Merge remote-tracking branch 'qubesos/pr/194'
* qubesos/pr/194:
  reflink: style fix
  storage: typo fix
  lvm_thin: _remove_revisions() on revisions_to_keep==0
  lvm_thin: don't purge one revision too few
  lvm_thin: really remove revision
  lvm_thin: fill in volume's revisions_to_keep from pool
2018-02-22 17:51:37 +01:00
Christopher Laprise
75d8c553f9
Fix is_running non-boolean 2018-02-20 22:30:47 -05:00
Marek Marczykowski-Górecki
4f52a3b62e
version 4.0.23 2018-02-20 01:18:31 +01:00
Marek Marczykowski-Górecki
522a9594ee
rpm: adjust dependencies 2018-02-20 01:18:15 +01:00
Marek Marczykowski-Górecki
68b6f1ec76
qubespolicy: use '@' instead of '$' for policy keywords
Using '$' is easy to misuse in shell scripts, shell commands etc. After
all this years, lets abandon this dangerous character and move to
something safer: '@'. The choice was made after reviewing specifications
of various shells on different operating systems and this is the
character that have no special meaning in none of them.

To preserve compatibility, automatically translate '$' to '@' when
loading policy files.
2018-02-19 03:33:40 +01:00
Marek Marczykowski-Górecki
c87fcd7e2e
qubespolicy: use separate arguments for original target type and value
Provide original target as two arguments: type, value
This will ease handling special keywords without risking hitting shell
special characters or other problems.
2018-02-19 03:32:44 +01:00