Commit Graph

95 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
e5413a3036
Merge branch 'storage-properties'
* storage-properties:
  storage: use None for size/usage properties if unknown
  tests: call search_pool_containing_dir with various dirs and pools
  storage: make DirectoryThinPool helper less verbose, add sudo
  api/admin: add 'included_in' to admin.pool.Info call
  storage: add Pool.included_in() method for checking nested pools
  storage: move and generalize RootThinPool helper class
  storage/kernels: refuse changes to 'rw' and 'revisions_to_keep'
  api/admin: implement admin.vm.volume.Set.rw method
  api/admin: include 'revisions_to_keep' and 'is_outdated' in volume info
2018-03-21 01:43:53 +01:00
Marek Marczykowski-Górecki
03dc3e315e
storage: use None for size/usage properties if unknown
Raising NotImplementedError in a _property_ is weird behaviour, better
suited for actions (methods). Use None instead.

QubesOS/qubes-issues#3241
2018-03-20 17:31:16 +01:00
Marek Marczykowski-Górecki
1bc640f3e0
api/admin: add 'included_in' to admin.pool.Info call
QubesOS/qubes-issues#3240
QubesOS/qubes-issues#3241
2018-03-20 16:53:40 +01:00
Marek Marczykowski-Górecki
376c8ec00d
api/admin: implement admin.vm.volume.Set.rw method
Allow setting 'rw' volume property.
2018-03-20 16:52:06 +01:00
Rusty Bird
a0c5014737
api/admin: fix backup exclude
Bugfix on 59abdeb
2018-03-20 02:41:37 +00:00
Marek Marczykowski-Górecki
2a962c54db
api/admin: include 'revisions_to_keep' and 'is_outdated' in volume info
Since Volume.is_outdated() is a method, not a property, add a function
for handling serialization. And at the same time, fix None serialization
(applicable to 'source' property).

QubesOS/qubes-issues#3256
2018-03-19 04:00:21 +01:00
Marek Marczykowski-Górecki
7c4566ec14
vm/qubesvm: allow 'features-request' to have async handlers
Some handlers may want to call into other VMs (or even the one asking),
but vm.run() functions are coroutines, so needs to be called from
another coroutine. Allow for that.
Also fix typo in documentation.
2018-03-02 01:16:38 +01:00
Rusty Bird
59abdeb30e
Allow include=None to be passed to admin.backup.Info
This is forwarded to qubes.backup.Backup(), which uses None to choose
VMs according to their include_in_backups property.

('git show -w' recommended)
2018-02-23 21:29:15 +00:00
Marek Marczykowski-Górecki
68b6f1ec76
qubespolicy: use '@' instead of '$' for policy keywords
Using '$' is easy to misuse in shell scripts, shell commands etc. After
all this years, lets abandon this dangerous character and move to
something safer: '@'. The choice was made after reviewing specifications
of various shells on different operating systems and this is the
character that have no special meaning in none of them.

To preserve compatibility, automatically translate '$' to '@' when
loading policy files.
2018-02-19 03:33:40 +01:00
Rusty Bird
fe77b0ec85
Make 'qvm-volume revert' really use the latest revision
admin.vm.volume.ListSnapshots returned volume revisions in undefined
order, but 'qvm-volume revert' assumes the list to be in chronological
order. Make that assumption true.
2018-01-20 23:20:23 +00:00
Marek Marczykowski-Górecki
d83a07177b
Merge remote-tracking branch 'qubesos/pr/176'
* qubesos/pr/176:
  Do not remove VMs installed via rpm
2018-01-05 16:27:12 +01:00
Christopher Laprise
47b49c4755
Do not remove VMs installed via rpm 2017-12-29 23:24:41 -05:00
Marek Marczykowski-Górecki
32c6083e1c
Make pylint happy
Fix thing detected by updated pylint in Travis-CI
2017-12-21 18:19:10 +01:00
Marek Marczykowski-Górecki
fd45378041
api/admin: make libvirt start error more informative
Point where to look for details.
2017-12-14 23:26:52 +01:00
Marek Marczykowski-Górecki
bf59b00f1d
features: add check_with_netvm, similar to check_with_template
Allow using default feature value from netvm, not template. This makes
sense for network-related features like using tor, supporting ipv6 etc.

Similarly to check_with_template, expose it also on Admin API.
2017-12-07 01:40:30 +01:00
Marek Marczykowski-Górecki
749e8497e3
api/admin: exclude regex attribute from DeviceInfo structure
DeviceInfo may contain 'regex' attribute - it isn't intended to be
reported through Admin API. Also, mark 'libvirt_regex' attribute as
private.
2017-12-05 17:39:32 +01:00
Jean-Philippe Ouellet
b41c118166
Fix typo in volume import end handler 2017-11-21 01:05:06 -05:00
Marek Marczykowski-Górecki
a92dd99fbb
Merge branch '20171107-storage'
* 20171107-storage:
  api/admin: add API for changing revisions_to_keep dynamically
  storage/file: move revisions_to_keep restrictions to property setter
  api/admin: hide dd statistics in admin.vm.volume.Import call
  storage/lvm: fix importing different-sized volume from another pool
  storage/file: fix preserving spareness on volume clone
  api/admin: add pool size and usage to admin.pool.Info response
  storage: add size and usage properties to pool object
2017-11-20 22:52:50 +01:00
Marek Marczykowski-Górecki
73c33525a0
api: do not log handled errors sent to a client
Those "errors" are already properly handled, and if necessary logged
independently by appropriate function. In some cases, such logs are
misleading (for example QubesNoSuchPropertyError is a normal thing
happening during qvm-ls).

Fixes QubesOS/qubes-issues#3238
2017-11-07 03:10:40 +01:00
Marek Marczykowski-Górecki
c3afdde3ef
api/admin: add API for changing revisions_to_keep dynamically
This one pool/volume property makes sense to change dynamically. There
may be more such properties, but lets be on the safe side and take
whitelist approach - allow only selected (just one for now), instead of
blacklisting any harmful ones.

QubesOS/qubes-issues#3256
2017-11-07 02:57:59 +01:00
Marek Marczykowski-Górecki
f18f4c9bff
api/admin: add pool size and usage to admin.pool.Info response
QubesOS/qubes-issues#3240
2017-11-07 02:57:58 +01:00
Marek Marczykowski-Górecki
4809c16e08
admin-api: add admin.*.property.GetDefault
Fixes QubesOS/qubes-issues#3197
2017-10-21 04:05:15 +02:00
Marek Marczykowski-Górecki
27b96cce4c
firewall: skip expired rules
Expired rules are skipped while loading the firewall. Do that also when
such rules expired after loading the firewall. This applies to both
Admin API and actually applying the rules (sending them to appropriate
VM).

Related QubesOS/qubes-issues#3020
2017-10-21 03:35:00 +02:00
Marek Marczykowski-Górecki
2164a8d7b8
Change license to LGPL v2.1+
See this thread for reasoning and acceptance from contributors:
https://groups.google.com/d/topic/qubes-devel/G7KzrfU0lWY/discussion
"Changing qubes-core-admin license to LGPL v2.1+"
2017-10-12 00:11:50 +02:00
Marek Marczykowski-Górecki
55208a0c7c
Merge branch 'dispvm-policy' 2017-09-14 00:15:29 +02:00
Marek Marczykowski-Górecki
99bd193688
Rename 'dispvm_allowed' to 'template_for_dispvms'
'dispvm_allowed' name was confusing, because it suggested being able to
spawn new DispVMs, not being a template for DispVM.

Fixes QubesOS/qubes-issues#3047
2017-09-05 02:07:26 +02:00
Marek Marczykowski-Górecki
cea70748a6
api/admin: implement admin.vm.device....Set.persistent
This will allow converting persistent device (used to boot VM for
example) to non-persistent.

QubesOS/qubes-issues#3055
2017-09-05 01:33:47 +02:00
Marek Marczykowski-Górecki
8c1ba905a0
api: do not fail events when listener is cancelled in the meantime
Raising any error from event handler will abort further handlers of this
event.
2017-09-05 01:33:46 +02:00
Marek Marczykowski-Górecki
971c7d4ac9
api/admin: add admin.vm.CreateDisposable in place of internal.vm.Create.DispVM
Add public Admin API call to create Disposable VM that would be
automatically destroyed after shutdown. Do not keep this functionality
for qrexec-policy tool only.
Also, use admin.vm.Start there, instead of internal.vm.Start and
admin.vm.Kill instead of internal.vm.CleanupDispVM (this is enough,
because DispVM now have auto_cleanup property).

QubesOS/qubes-issues#2974
2017-08-06 20:54:10 +02:00
Marek Marczykowski-Górecki
c8c32d4956
Merge branch 'stats'
* stats:
  tests: admin.vm.Stats
  api: fix handling interrupted calls
  api/admin: implement admin.vm.Stats
  app: refresh getting VM statistics, rename to QubesHost.get_vm_stats
  api: use str(subject) instead of explicit subject.name
2017-07-29 05:05:32 +02:00
Marek Marczykowski-Górecki
76640df091
Merge branch 'tests-storage'
* tests-storage:
  tests: register libvirt events
  tests: even more agressive cleanup in tearDown
  app: do not wrap libvirt_conn.close() in auto-reconnect wrapper
  api: keep track of established connections
  tests: drop VM cleanup from tearDownClass, fix asyncio usage in tearDown
  storage: fix Storage.clone and Storage.clone_volume
  tests: more tests fixes
  firewall: raise ValueError on invalid hostname in dsthost=
  qmemman: don't load qubes.xml
  tests: fix AdminVM test
  tests: create temporary files in /tmp
  tests: remove renaming test - it isn't supported anymore
  tests: various fixes for storage tests
  tests: fix removing LVM volumes
  tests: fix asyncio usage in some tests
  tests: minor fixes to api/admin tests
  storage/file: create -cow.img only when needed
  storage: move volume_config['source'] filling to one place
  app: do not create 'default' storage pool
  app: add missing setters for default_pool* global properties
2017-07-29 05:01:42 +02:00
Marek Marczykowski-Górecki
639fa26079
Merge branch 'qdb-watch'
* qdb-watch:
  tests: add qdb_watch test
  ext/block: make use of QubesDB watch
  vm: add API for watching changes in QubesDB
  vm: optimize imports
  api/admin: don't send internal events in admin.Events
  Add explanation why admin.vm.volume.Import is a custom script
  Follow change of qubesdb path return type
  Rename vm.qdb to vm.untrusted_qdb
2017-07-29 05:01:13 +02:00
Marek Marczykowski-Górecki
27ce27b6e9
backup: improve error reporting about passphrase retrieval
QubesOS/qubes-issues#2931
2017-07-29 04:47:44 +02:00
Marek Marczykowski-Górecki
1556814f59
backup: fix handling already encoded passphrase
When passphrase is retrieved from VM, it is already encoded. Do not try
to encode it again.

QubesOS/qubes-issues#2931
2017-07-29 04:47:43 +02:00
Marek Marczykowski-Górecki
2f4b4d97e7
api: fix handling interrupted calls
When an API call is interrupted, the relevant coroutine is cancelled -
which means it may throw CancelledError. At the same time, cancelled
call have related socket already closed (and transport set to None). But
QubesDaemonProtocol.respond try to close the transport again, which
fails. Fix handling this case.
2017-07-28 03:02:33 +02:00
Marek Marczykowski-Górecki
147bca1648
api/admin: implement admin.vm.Stats
QubesOS/qubes-issues#853
2017-07-28 03:02:32 +02:00
Marek Marczykowski-Górecki
0d9574d9fc
api: use str(subject) instead of explicit subject.name
This allows better flexibility, when subject is not necessary a VM
object.
2017-07-27 22:15:02 +02:00
Marek Marczykowski-Górecki
074c705f77
api: keep track of established connections
This will be needed to gracefuly cleanup them between individual tests.
2017-07-26 03:26:59 +02:00
Marek Marczykowski-Górecki
354aa14c53
api/admin: don't send internal events in admin.Events
Some events are internal for a sole purpose of getting some data from
extension. Since listeners of admin.Events cannot return anything, there
is no sense in sending those events there.
2017-07-25 05:20:38 +02:00
Marek Marczykowski-Górecki
0fb7c1fbed
Follow change of qubesdb path return type
It's str/unicode, not bytes now.

QubesOS/qubes-issues#2937
2017-07-25 05:20:38 +02:00
Marek Marczykowski-Górecki
1759bca00f
Rename vm.qdb to vm.untrusted_qdb
QubesDB can be freely modified by a VM, so one should take care when
reading any data retrieved from it.

Fixes QubesOS/qubes-issues#2934
2017-07-24 13:01:55 +02:00
Marek Marczykowski-Górecki
82918ee704
api/admin: gracefuly report cancelled backup operation
QubesOS/qubes-issues#2931
2017-07-21 03:10:26 +02:00
Marek Marczykowski-Górecki
6dbce8259f
api/admin: plug backup into Admin API
Fixes QubesOS/qubes-issues#2931
2017-07-20 12:19:01 +02:00
Marek Marczykowski-Górecki
c6e09b5a99
storage: fix resize and revert handling
- fix coroutine calling from Admin API
 - fix off-by-one error in resize
 - adjust tests
2017-07-18 01:42:37 +02:00
Marek Marczykowski-Górecki
cbc7241a93
api/admin: rename mgmt-permission:* event to admin-permission:*
Be more consistent with Admin API name and method names.
2017-07-17 02:41:42 +02:00
Marek Marczykowski-Górecki
ab7ed776f9
Merge branch 'dispvm-fix'
* dispvm-fix:
  storage: add comment about source volume lookup
  vm/dispvm: convert DispVM related function to coroutines
  vm/dispvm: fix DispVM storage definition
2017-07-14 01:13:20 +02:00
Marek Marczykowski-Górecki
66b88cc943
api: extract function to make pylint happy 2017-07-12 21:33:14 +02:00
Marek Marczykowski-Górecki
e6d5337fa7
api: cleanup already started servers when some later failed 2017-07-12 21:33:14 +02:00
Marek Marczykowski-Górecki
c32f0db582
vm/dispvm: convert DispVM related function to coroutines
Some functions used there (create_on_disk, remove_from_disk, kill) are
coroutines, so callers needs to be too.

Fixes QubesOS/qubes-issues#2896
2017-07-08 03:04:55 +02:00
Marek Marczykowski-Górecki
dddd94b339
Merge branch 'core3-storage3' 2017-07-04 16:44:53 +02:00