Commit Graph

1333 Commits

Author SHA1 Message Date
qubesuser
92c452a655 change default pool code to be fast
currently it takes 100ms+ to determine the default pool every time,
including subprocess spawning (!), which is unacceptable since
finding out the default value of properties should be instantaneous

instead of checking every thin pool to see if the root device is in
it, find and cache the name of the root device thin pool and see if
there is a configured pool with that name
2017-11-11 02:38:16 +01:00
qubesuser
d183ab1c18 cache PropertyHolder.property_list and use O(1) property name lookups 2017-11-11 02:38:16 +01:00
qubesuser
f2b8ad7d38 remove unused netid code
it's unused and has a netid property with name different than key
that would cause issues in the next commit
2017-11-11 02:37:37 +01:00
qubesuser
b297ecbcf1 cache isinstance(default, collections.Callable) 2017-11-10 18:14:40 +01:00
qubesuser
9cc86b3be2 don't access netvm if it's None in visible_gateway/netmask
Causes an unnecessary exception
2017-11-10 18:14:31 +01:00
Marek Marczykowski-Górecki
e2eaa57f65
Merge branch 'tests-20171020'
* tests-20171020:
  tests: fix asyncio usage some more places
  tests: fix reporting in network tests
  tests: domain-shutdown event race condition
  tests: improve events tests
  tests: fix logic error in wait_for_window()
  tests: cleanup QubesDB connection on domain remove
2017-10-21 04:33:35 +02:00
Marek Marczykowski-Górecki
42f8619559
tests: fix asyncio usage some more places
gui_memory_pinning tests finally works. Also, fix DispVM and private
volume resize tests.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
511502ae7e
tests: fix reporting in network tests
Do not trash stdout with VM's command output. Log it as part of
(potential) failure message.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
c3430fe5f6
tests: domain-shutdown event race condition
Second tests, for actual events, not storage handling done by its
handler.

cc @HW42

QubesOS/qubes-issues#3164
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
96f2e503db
tests: improve events tests
Check property-related events, including their arguments.
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
71fc5217f7
tests: fix logic error in wait_for_window()
Revert change introduced in cbc5dbb9b1
2017-10-21 04:06:15 +02:00
Marek Marczykowski-Górecki
43c09467b1
tests: cleanup QubesDB connection on domain remove
If domain got removed during the tests (for example DispVM), vm.close()
wouldn't be called in cleanup and some file descriptors will be
leaked. Add event handler for cleaning this up. Do not use close()
method here, because it is destructive, but the object may still be used
by the test.
2017-10-21 04:06:14 +02:00
Marek Marczykowski-Górecki
0b1a0b028b
app: fix setting default_*netvm
1. Fire both property-pre-del:netvm and property-del:netvm - those
events should be fired in pairs - especially one may assume the other
will be called too. This is the case here - one disconnect old netvm,
the other connect the new one.

2. Remove spurious 'newvalue' argument for property-del:netvm event.

3. Fix logic for default_fw_netvm/default_netvm usage. The former is
used if vm.provides_network=True.
2017-10-21 04:05:17 +02:00
Marek Marczykowski-Górecki
4500c4dcab
vm/adminvm: add is_halted() method
It is used in some places, for example attaching devices.

Fixes QubesOS/qubes-issues#3054
2017-10-21 04:05:17 +02:00
Marek Marczykowski-Górecki
433430884d
devices: clarify error on attaching non-persistent device to stopped VM
Fixes QubesOS/qubes-issues#3195
2017-10-21 04:05:16 +02:00
Marek Marczykowski-Górecki
9a9452ae3a
vm/net: fix handling netvm reset to default
If there was some netvm set, unset it first (same as with ordinary set).
Otherwise it will try to attach new netvm without detaching the old one
first.
2017-10-21 04:05:16 +02:00
Marek Marczykowski-Górecki
98818b0ad9
base: fix property-(pre-)del events arguments
oldvalue should contain the old value, even if it was default one. Same
as in property-(pre-)set events. If event want to check if that is
default value, it is always possible (in pre- event), but in practice
actual value is most useful.

This bug prevented netvm changing events from working when reseting
netvm.
2017-10-21 04:05:16 +02:00
Marek Marczykowski-Górecki
8f1bfa65f8
tests: fix typo 2017-10-21 04:05:15 +02:00
Marek Marczykowski-Górecki
4809c16e08
admin-api: add admin.*.property.GetDefault
Fixes QubesOS/qubes-issues#3197
2017-10-21 04:05:15 +02:00
Marek Marczykowski-Górecki
038ca6d5d7
base: add method for getting default property value
Allow to get default value even it isn't set currently. This will allow
(G)UI to present better view, without duplicating logic for default
value.

Fixes QubesOS/qubes-issues#3197
2017-10-21 04:05:15 +02:00
Marek Marczykowski-Górecki
85c3368a0d
comments fixes 2017-10-21 04:05:15 +02:00
Marek Marczykowski-Górecki
c2c3225622
tests: root fs resize tests - for all templates
Check resize on each template separately, because it involves VM's
scripts (either qubes.ResizeDisk service, or some startup script).

QubesOS/qubes-issues#3173
2017-10-21 04:05:14 +02:00
Marek Marczykowski-Górecki
77ce956c2d
tests: improve root volume resize tests
- clone all features, not just qrexec (especially include 'gui')
- do not leak VM reference on failed test
- add test for online root volume resize

QubesOS/qubes-issues#3173
2017-10-21 03:35:01 +02:00
Marek Marczykowski-Górecki
49e35e66aa
storage: improve error reporting
- make it clear that failed qubes.ResizeDisk service call means the need
to resize filesystem manually (but not necessarily the volume itself)
- propagate exceptions raised by async storage pool implementations

Related QubesOS/qubes-issues#3173
2017-10-21 03:35:01 +02:00
Marek Marczykowski-Górecki
27b96cce4c
firewall: skip expired rules
Expired rules are skipped while loading the firewall. Do that also when
such rules expired after loading the firewall. This applies to both
Admin API and actually applying the rules (sending them to appropriate
VM).

Related QubesOS/qubes-issues#3020
2017-10-21 03:35:00 +02:00
Marek Marczykowski-Górecki
836d9f902a
Merge branch 'bug3164'
* bug3164:
  tests: add regression test for #3164
  storage/lvm: make sure volume cache is refreshed after changes
  storage/lvm: fix Volume.verify()
  storage/lvm: remove old volume only after successfully cloning new one
2017-10-16 04:28:26 +02:00
Marek Marczykowski-Górecki
c8519a700f
tests: add regression test for #3164
This is a race condition, so to make it more likely to fail (if it's
broken), make some things manually. In normal circumstances this order
of actions is also possible, just less likely to happen. But as seen in
the bug report, happens from time to time.

QubesOS/qubes-issues#3164
2017-10-16 00:48:25 +02:00
Marek Marczykowski-Górecki
08583af1ff
storage/lvm: make sure volume cache is refreshed after changes
Even if start/stop fails, still refresh the cache. The failed operation
might still modified something.
2017-10-16 00:44:52 +02:00
Marek Marczykowski-Górecki
145ccfb34e
storage/lvm: fix Volume.verify()
This function is supposed to raise an exception if something is wrong,
not just return False. Document this.
2017-10-16 00:43:10 +02:00
Marek Marczykowski-Górecki
021047f950
storage/lvm: remove old volume only after successfully cloning new one
In some cases, it may happen that new volume (`self._vid_snap`) does not
exists. This is normally an error, but even in such a case, do not
remove the only remaining instance of volume (`self.vid`). Instead,
rename it temporarily and remove only after new volume is successfully
cloned.

Fixes QubesOS/qubes-issues#3164
2017-10-13 01:11:55 +02:00
Marek Marczykowski-Górecki
2164a8d7b8
Change license to LGPL v2.1+
See this thread for reasoning and acceptance from contributors:
https://groups.google.com/d/topic/qubes-devel/G7KzrfU0lWY/discussion
"Changing qubes-core-admin license to LGPL v2.1+"
2017-10-12 00:11:50 +02:00
Marek Marczykowski-Górecki
08640f4779
Merge remote-tracking branch 'qubesos/pr/154'
* qubesos/pr/154:
  Add basic PVHv2 support
2017-10-09 00:08:35 +02:00
Marek Marczykowski-Górecki
b8d45c214d
storage/lvm: don't crash when resizing to the same size
This applies to qvm-template-postprocess, which at the beginning try to
resize root volume to appropriate size. It makes more sense to silently
succeed here, instead of forcing every client-side utility to check if
the volume have already desired size.
2017-10-08 18:37:25 +02:00
Marek Marczykowski-Górecki
bce7b64d4a
Fix calling qubes.SuspendPre/qubes.SuspendPost services
Those should be called as root, not default user.

Thanks @jpouellet for debugging this.
Fixes QubesOS/qubes-issues#3151
Fixes QubesOS/qubes-issues#3142
2017-10-07 02:36:02 +02:00
Marek Marczykowski-Górecki
451cc339c7
ext/block: accept any boolean value for 'read-only' option
Be consistent with other parts of the Admin API. Especially ...Attach
accept "standard" boolean values for persistent= option.
2017-10-04 15:24:53 +02:00
Marek Marczykowski-Górecki
af60fbb25d
tests: dispvm tests fixes 2017-10-04 15:24:53 +02:00
Marek Marczykowski-Górecki
df32acfad9
Allow setting global default_dispvm to None
There is no technical need for having this set to anything. If set to
None, DispVM startup requests will be refused.
2017-10-04 15:24:53 +02:00
Marek Marczykowski-Górecki
6990550884
tests: more tests fixes, drop expectedFailure where not needed anymore 2017-10-04 15:24:52 +02:00
Marek Marczykowski-Górecki
e0be7861f9
tests: extend qrexec_policy context manager - custom action
Allow to use custom action - for example with 'target=' argument.
2017-10-04 15:24:52 +02:00
Marek Marczykowski-Górecki
cbc5dbb9b1
tests: various fixes
- Use proper features/services names (updates proxy test).
- Fix logic error in wait_for_window.
- Fix test for qvm-sync-clock (first sync clockvm, then dom0), also fix
  cleanup (unset clockvm before removing it)
- More fixes for asyncio usage
2017-10-04 15:24:52 +02:00
Marek Marczykowski-Górecki
64c5b2c00f
tests: cleanup DispVMs created during tests
Since those do not have specific prefix (test-) compare list of them
with original qubes.xml
2017-10-04 15:24:51 +02:00
Marek Marczykowski-Górecki
6091bbbe88
storage/lvm: fix size units on resize
lvextend uses MiB, not MB.
2017-10-03 18:46:39 +02:00
Marek Marczykowski-Górecki
3a5e8482c0
Merge branch 'fixes-20170929'
* fixes-20170929:
  vm: do not start QubesDB watch instance multiple times
  vm: report storage.stop() errors to log
  vm: move comment
  storage: fix method name in LinuxModules volume
  Prevent removing domain that is referenced from anywhere
  vm: add vm.klass property
  Move QubesVM.{name,qid,uuid,label} to BaseVM
  vm: do not allow deleting template property from AppVM and DispVM
  vm/qubesvm: emit event on failed startup
  vm/qubesvm: remove duplicated qmemman_client.close()
  vm/dispvm: cleanup DispVM also on failed startup
  vm/dispvm: fix error message
  ext/block: properly list devtype=cdrom option
  block: fix handling non-existing devices
  block: improve handling device name and description
2017-10-03 12:04:00 +02:00
HW42
c583cf5190 Add basic PVHv2 support 2017-10-02 22:23:27 +02:00
Marek Marczykowski-Górecki
12b7e22d27
vm: do not start QubesDB watch instance multiple times
vm.create_qdb_entries can be called multiple times - for example when
changing VM IP. Move starting qdb watcher to start(). And just in case,
cleanup old watcher (if still exists) before starting new one.
This fixes one FD leak.
2017-09-29 12:05:23 +02:00
Marek Marczykowski-Górecki
9172a7708d
tests: fix various object leaks cont
Do not use self.fail when handling exception - this will keep exception
object referenced, which in turn have reference to domain object (via
traceback).
2017-09-29 11:53:59 +02:00
Marek Marczykowski-Górecki
f9da1f2dd7
tests: various fixes for networking tests
- Update feature name for enabling network-manager.
- Fix error reporting
2017-09-29 11:53:59 +02:00
Marek Marczykowski-Górecki
fdf780aad0
tests/network: match vm names with variable names
Having testvm1 variable for "vm2" is really confusing.
2017-09-29 11:53:59 +02:00
Marek Marczykowski-Górecki
8b34b9a5e4
tests: hide unneeded messages 2017-09-29 11:53:59 +02:00
Marek Marczykowski-Górecki
3c54244fdc
tests: fix various object and FD leaks
- Prefer instance attributes over local variables - the former ones do
not leak into traceback object and are cleaned up by tests framework.
- Use 'with' syntax for handling files.
- Use subprocess.DEVNULL instead of open('/dev/null') where applicable
- Delete local variables when not needed anymore.
2017-09-29 11:53:59 +02:00