Commit Graph

5728 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
dd037f4663
storage/file: get volume size from the actual image file size
Don't realy on a volume configuration only, it's easy to miss updating
it. Specifically, import_volume() function didn't updated the size based
on the source volume.
The size that the actual VM sees is based on the
file size, and so is the filesystem inside. Outdated size property can
lead to a data loss if the user perform an action based on a incorrect
assumption - like extending size, which actually will shrink the volume.

Fixes QubesOS/qubes-issues#4821
2019-10-31 21:53:35 +01:00
Marek Marczykowski-Górecki
598d059c57
tests: check if storage driver adjust the size on import_volume/clone
Regression test for QubesOS/qubes-issues#4821
2019-10-31 01:17:26 +01:00
Marek Marczykowski-Górecki
5908ab1568
app: fix get_free_xen_memory function
It's app.vmm.xc, not app.xc. Since nobody have noticed it, this function
might be unused...
2019-10-30 15:46:11 +01:00
Marek Marczykowski-Górecki
361550c621
vm: improve error message about missing IOMMU
Handle this case specifically, as way too many users ignore the message
during installation and complain it doesn't work later.

Name the problem explicitly, instead of pointing at libvirt error log.

Fixes QubesOS/qubes-issues#4689
2019-10-30 15:45:52 +01:00
Marek Marczykowski-Górecki
89e3ae3945
version 4.1.4 2019-10-28 04:18:43 +01:00
Marek Marczykowski-Górecki
8603571cbc
tests: skip networking tests for minimal templates
Minimal templates don't have networking packages installed by default.
2019-10-28 03:30:49 +01:00
Marek Marczykowski-Górecki
4dac995089
Merge branch 'start-logging'
* start-logging:
  vm: log startup errors in every case
2019-10-23 13:50:25 +02:00
Marek Marczykowski-Górecki
de3c7edd4f
travis: switch to python 3.6
For system site-packages to work, it needs to be the same as in the base
distribution. Switch to the version in Ubuntu bionic

QubesOS/qubes-issues#4613
2019-10-23 04:43:33 +02:00
Marek Marczykowski-Górecki
5c3bb1cbd0
Merge remote-tracking branch 'origin/pr/288'
* origin/pr/288:
  travis: switch to bionic
2019-10-23 04:40:07 +02:00
Marek Marczykowski-Górecki
cc56c6f96a
Merge remote-tracking branch 'origin/pr/278'
* origin/pr/278:
  Added admin.pool.UsageDetails API method
  Add metadata info to LVM AdminAPI
2019-10-23 04:36:49 +02:00
Marta Marczykowska-Górecka
2f6497e48d
Added admin.pool.UsageDetails API method
admin.pool.UsageDetails reports the usage data, unlike
admin.pool.Info, which should report the config/unchangeable data.
At the moment admin.Pool.Info still reports usage, to maintain
compatibility, but once all relevant tools are updated,
it should just return configuration data.
2019-10-23 03:04:30 +02:00
Marta Marczykowska-Górecka
04a215fb83
Add metadata info to LVM AdminAPI
Added usage_details method to Pool class
(returns a dictionary with detailed information
on pool usage) and LVM implementation that returns
metadata info.
Needed for QubesOS/qubes-issues#5053
2019-10-22 17:29:01 +02:00
Marek Marczykowski-Górecki
5cc453f097
vm: remove dead (commented out) code
default_user property was re-implemented, remove dead version.
2019-10-22 16:44:02 +02:00
Marek Marczykowski-Górecki
656e36f1ee
Merge remote-tracking branch 'origin/pr/287'
* origin/pr/287:
  app: fix docstrings PEP8 refactor
  tests: remove iptables_header content in test_622_qdb_keyboard_layout
  tests: add test for guivm and keyboard_layout
  gui: simplify setting guivm xid and keyboard layout
  Make pylint happier
  gui: set keyboard layout from feature
  Handle GuiVM properties
  Make PEP8 happier
2019-10-22 14:10:56 +02:00
Marek Marczykowski-Górecki
dd8de570c8
doc: update reference to Features class, add NetVMMixin 2019-10-22 13:09:13 +02:00
Frédéric Pierret (fepitre)
7c8556891c
app: fix docstrings PEP8 refactor 2019-10-22 09:26:25 +02:00
Frédéric Pierret (fepitre)
5ee97f4eeb
tests: remove iptables_header content in test_622_qdb_keyboard_layout 2019-10-22 09:26:03 +02:00
Marek Marczykowski-Górecki
04dcc0da71
Merge remote-tracking branch 'origin/pr/286'
* origin/pr/286:
  tests: fix network re-attach tests
  tests: fix a fix for too short delay in 201_shutdown_event_race test
2019-10-21 20:48:27 +02:00
Frédéric Pierret (fepitre)
e667639914
tests: add test for guivm and keyboard_layout 2019-10-20 17:36:06 +02:00
Frédéric Pierret (fepitre)
5ee2f5d889
gui: simplify setting guivm xid and keyboard layout 2019-10-20 17:35:43 +02:00
Frédéric Pierret (fepitre)
d2d1ffb806
Make pylint happier 2019-10-20 16:40:40 +02:00
Frédéric Pierret (fepitre)
9734921d9c
gui: set keyboard layout from feature 2019-10-20 13:22:31 +02:00
Frédéric Pierret (fepitre)
27aad9bd38
Handle GuiVM properties 2019-10-20 13:22:31 +02:00
Frédéric Pierret (fepitre)
a52cb6bb91
Make PEP8 happier 2019-10-20 13:22:29 +02:00
Frédéric Pierret (fepitre)
ad470b1391
travis: switch to bionic
QubesOS/qubes-issues#4613
2019-10-19 21:44:22 +02:00
Marek Marczykowski-Górecki
d2f4b5d694
Make qubesd-query-fast support keyword targets
Specifically, allow to use `@default` and `@adminvm` as a target.
2019-10-16 14:23:48 +02:00
Marek Marczykowski-Górecki
a1dabfefa0
tests: fix network re-attach tests
On slow systems (OpenQA), 5s isn't enough. Instead of hardcoding some
timeout, simply wait for the full VM startup.
2019-10-13 05:58:50 +02:00
Marek Marczykowski-Górecki
f0ae8c0454
tests: fix a fix for too short delay in 201_shutdown_event_race test
It's _domain_stopped_lock, not startup_lock.
2019-10-13 05:58:19 +02:00
Marek Marczykowski-Górecki
d5b0a6e5b6
vm: log startup errors in every case 2019-10-10 19:41:02 +02:00
Marek Marczykowski-Górecki
45a6252c06
version 4.1.3 2019-10-09 05:41:43 +02:00
Marek Marczykowski-Górecki
56759bd1ed
Merge branch 'bug5362'
* bug5362:
  Add missing source admin.vm.Console call in qvm-console-dispvm
2019-10-09 05:08:34 +02:00
Marek Marczykowski-Górecki
ad8f1b06b4
Add missing source admin.vm.Console call in qvm-console-dispvm
Qrexec service called manually needs both source and destination

Fixes QubesOS/qubes-issues#5362
2019-10-09 04:41:48 +02:00
Marek Marczykowski-Górecki
51af2ed27c
vm: add domain-shutdown-failed event
Similar to domain-start-failed, add an event fired when
domain-pre-shutdown was fired but the actual operation failed.
Note it might not catch all the cases, as shutdown() may be called with
wait=False, which means it won't wait fot the actual shutdown. In that
case, timeout won't result in domain-shutdown-failed event.

QubesOS/qubes-issues#5380
2019-10-08 23:35:24 +02:00
Marek Marczykowski-Górecki
0300e895f2
Fix domain-start-failed event documentation. 2019-10-08 23:35:18 +02:00
Marek Marczykowski-Górecki
cc1ac0b859
tests: log stderr of paplay command 2019-09-29 06:43:34 +02:00
Marek Marczykowski-Górecki
7def96c248
tests: register syslog logger, log test start
Move this functionality from our custom runner (qubes.tests.run),
into base test class. This is very useful for correlating logs, so lets
have it with nose2 runner too.
2019-09-29 06:43:34 +02:00
Marek Marczykowski-Górecki
e0e0c7eaf9
tests: remove VM under startup_lock
Prevent starting a VM while it's being removed. Something could try to
start a VM just after it's being killer but before removing it (Whonix
example from previous commit is real-life case). The window specifically
is between kill() call and removing it from collection
(`del app.domains[vm.qid]`). Grab a startup_lock for the whole operation
to prevent it.
2019-09-29 06:14:21 +02:00
Marek Marczykowski-Górecki
732231efb0
vm: refuse to start a VM not present in a collection
Check early (but after grabbing a startup_lock) if VM isn't just
removed. This could happen if someone grabs its reference from other
places (netvm of something else?) or just before removing it.
This commit makes the simple removal from the collection (done as the
first step in admin.vm.Remove implementation) efficient way to block
further VM startups, without introducing extra properties.

For this to be effective, removing from the collection, needs to happen
with the startup_lock held. Modify admin.vm.Remove accordingly.
2019-09-29 06:14:21 +02:00
Marek Marczykowski-Górecki
6cfda328bf
tests: add workaround for Whonix re-starting VMs
Workaround for https://phabricator.whonix.org/T930
For now, unregister all the VMs to be killed manually.
2019-09-29 06:14:21 +02:00
Marek Marczykowski-Górecki
34e2f3a322
tests: fix sorting kernel version
Debian now has 4.9 and 4.19 kernels installed, so `sort -n` sorts them
wrong.
2019-09-27 16:29:21 +02:00
Marek Marczykowski-Górecki
5fa75d73be
Make pylint happy 2019-09-27 16:29:20 +02:00
Marek Marczykowski-Górecki
3dd92faa0e
pylint: disable import-outside-toplevel
All the cases highlighted by this checker are intentional.
2019-09-27 16:29:20 +02:00
Marek Marczykowski-Górecki
a7c6897ba6
Move pylint configuration into standard location
Make it easier to run pylint with correct configuration. This was
especially bad as .pylintrc did existed, but was plain default only.
2019-09-27 16:29:20 +02:00
Marek Marczykowski-Górecki
eb39f69882
api: improve handling destination removed just before the call
There are cases when destination domain doesn't exist when the call gets
to qubesd. Namely:
 1. The call comes from dom0, which bypasses qrexec policy
 2. Domain was removed between checking the policy and here
Handle the the same way as if the domain wouldn't exist at policy
evaluation stage either - i.e. refuse the call.

On the client side it doesn't change much, but on the server call it
avoids ugly, useless tracebacks in system journal.

Fixes QubesOS/qubes-issues#5105
2019-09-27 16:29:20 +02:00
Marek Marczykowski-Górecki
8ecf00bd0e
tests: add helpful decorator to wait before test cleanup
Allow to manual inspect test environment after test fails. This is
similar to --do-not-clean option we had in R3.2.

The decorator should be used only while debugging and should never be
applied to the code committed into repository.
2019-09-27 16:29:19 +02:00
Marek Marczykowski-Górecki
4fa45dbc91
tests: fix (hopefully) too short delay in 201_shutdown_event_race test
Domain shutdown handling may take extended amount of time, especially on
slow machine (all the LVM teardown etc). Take care of it by
synchronizing using vm.startup_lock, instead of increasing constant
delay. This way, the shutdown event handler needs to be started within
3s, not finish in this time.
2019-09-27 16:29:19 +02:00
Marek Marczykowski-Górecki
dfa0626cea
tests: check event handler re-registration after libvirt restart
QubesOS/qubes-issues#5303
2019-09-27 16:29:19 +02:00
Marek Marczykowski-Górecki
8e36a2ac61
app: re-register event handler after libvirt daemon restart
When libvirt daemon is restarted, qubesd attempt to re-connect to the
new instance transparently (through virConnect object wrapper). But the
code lacked re-registering event handlers.
Fix this by adding reconnect callback argument to virConnectWrappper, to
be called after new connection is established. This callback will
additionally get old connection as an argument, if any cleanup is
needed. The old connection is closed just after callback returns.

Use this to re-register event handler, but also unregister old handler
first. While full unregister wont work on since old libvirt daemon
instance is dead already, it will still cleanup client structures.

Since the old libvirt connection is closed now, adjust also domain
reconnection logic, to handle stale connection object. In that case
isAlive() call throws an exception.

Fixes QubesOS/qubes-issues#5303
2019-09-26 01:57:59 +02:00
Marek Marczykowski-Górecki
273238bd2a
tests: fix qrexec abort test
Exit code depends on when exactly the other end was terminated.
2019-09-22 23:30:38 +02:00
Marek Marczykowski-Górecki
784fd966a2
Merge branch 'devel20190819'
* devel20190819:
  tests: make libvirt mockup more robust
  tests: update for not needing custom kernel modules anymore
2019-09-21 02:20:00 +02:00