Commit Graph

6121 Commits

Author SHA1 Message Date
Rusty Bird
9b6d082673
app: use suppress() in simple cases 2021-02-10 12:58:01 +00:00
Rusty Bird
7159f206a5
firewall: save firewall.xml with utils.replace_file()
Don't rewrite the file in-place.

Also change the error message from 'save error' to 'firewall save
error'.
2021-02-10 12:58:00 +00:00
Rusty Bird
c988a2218b
utils: take tweaked helper functions from storage/reflink
replace_file(), rename_file(), and remove_file() now have optional
'logger' and 'log_level' (defaulting to DEBUG) arguments.

replace_file() now has a required 'permissions' and an optional
'close_on_success' (defaulting to True) argument. Also, it doesn't
create any directories; and in case of an exception, the tempfile is
removed even when closing it raises another exception.

remove_file() now returns a value: True if the file was removed, or
False if it already didn't exist.

(fsync_path() is unchanged.)

!!! After cherry-picking for release4.0, consider a fixup !!!
!!! adding 'import qubes.utils' to storage/reflink there  !!!
2021-02-10 12:47:09 +00:00
Rusty Bird
6c8fb4180b
storage/reflink: quote logged filenames 2021-02-10 12:47:07 +00:00
Marek Marczykowski-Górecki
d3d6b9de2b
version 4.1.18 2021-01-03 22:28:22 +01:00
Marek Marczykowski-Górecki
2bc3393fce
tests: skip pvgrub tests on Fedora
Fedora compresses the kernel with zstd now, which isn't suppported in
grub (as of 2.04). Since we don't care about PV that much anymore,
simply skip the test there.
2020-12-31 06:12:02 +01:00
Marek Marczykowski-Górecki
e160fc61b3
version 4.1.17 2020-12-03 15:52:30 +01:00
Marek Marczykowski-Górecki
39d9a619b4
Merge remote-tracking branch 'origin/pr/379'
* origin/pr/379:
  qubes-hcl-report: run lspci as root
2020-12-01 07:19:21 +01:00
Marek Marczykowski-Górecki
4417f252c0
Merge remote-tracking branch 'origin/pr/378'
* origin/pr/378:
  Enhance qvm-console-dispvm and admin.vm.Console
2020-12-01 06:29:02 +01:00
Matt McCutchen
e015165cf4
qubes-hcl-report: run lspci as root
This allows the "Capabilities" field to be shown.
2020-11-29 16:21:03 -05:00
Demi Marie Obenour
cb1cdd8fda
Only install needed packages and ignore failure
If the missing package was needed, it will fail later anyway.
2020-11-27 23:26:39 -05:00
Demi Marie Obenour
afbba5e552
Check for RPM and DNF before using them
This allows dependencies to be automatically installed on Fedora,
without causing problems on Debian.
2020-11-27 23:06:26 -05:00
Demi Marie Obenour
5daa964b9c
Only install dependencies if needed
If all of the dependencies are already installed, there is no need to
install them with DNF.
2020-11-27 22:20:39 -05:00
Demi Marie Obenour
27c70bff05
make pylint happy 2020-11-27 20:29:15 -05:00
Demi Marie Obenour
8b8c17f8b8
Avoid permission problems 2020-11-27 20:05:18 -05:00
Demi Marie Obenour
88cb57ca7a
Fix tests for DispVMs 2020-11-27 19:32:53 -05:00
Demi Marie Obenour
13cd47ecb4
Try to get the tests to work
They fail because the root volume is not actually updated.
2020-11-27 17:16:20 -05:00
Demi Marie Obenour
76cd08af00
Fix test suite
It failed to consider that the tests run outside of dom0.
2020-11-27 14:29:35 -05:00
Demi Marie Obenour
f4d93fd68e
Test that a DispVM’s template can be changed
Previously this raised a QubesValueError.
2020-11-27 14:29:34 -05:00
Demi Marie Obenour
9b3a598eff
DispVM has no “default_volume_config” property
Use “volume_config” (an instance property) instead
2020-11-27 14:29:34 -05:00
Demi Marie Obenour
d46657a244
Changing AppVM template breaks running DispVMs
So prevent changing a VM’s template if there are running DispVMs based
on it.
2020-11-27 14:29:30 -05:00
Demi Marie Obenour
e11f94b496
Implement changing the template of a DispVM 2020-11-27 14:28:02 -05:00
Demi Marie Obenour
ca9a496e42
Enhance qvm-console-dispvm and admin.vm.Console
- use `/run` instead of deprecated symlink `/var/run`
- add `--autostart` option to automatically start the qube
- check for invalid qube name before querying qubesd
- use `--fail` option of qubesd-query instead of parsing its output file
- add `--` and double quotes in various places
- allow `--` to be passed to `qvm-console-dispvm` to mark end of options
2020-11-26 21:15:34 -05:00
Marek Marczykowski-Górecki
39ef189a93
Merge remote-tracking branch 'origin/pr/354'
* origin/pr/354: (35 commits)
  tests/lvm: re-introduce POOL_CONF
  tests/lvm & callback: remove explicit class references
  storage/callback: remove the "word of caution"
  storage/callback: comment fixes
  storage/callback: add the config ID as callback argument
  storage/callback: some callbacks added & removed
  tests/lvm & callback: Refactoring
  Revert "storage/callback: do not run sync code async"
  tests/callback: ensure missing conf causes errors
  storage/callback: do not run sync code async
  tests/callback: added callback-specific tests
  storage/callback: async Volume.export() & added Volume.export_end()
  storage/lvm: make the "hack" work with CallbackPool instances
  storage/callback: add the backend_class property
  tests/callback: add them to the rpm build
  storage/callback: various fixes
  tests/callback: add rudimentary tests for the callback driver
  tests/lvm: make the tests re-usable for other drivers
  storage/callback: fix issues detected by pylint
  storage/callback: volume callbacks now also rceive the source volume as argument (if there's any)
  ...
2020-11-27 00:26:18 +01:00
Marek Marczykowski-Górecki
7ab1703a06
Merge remote-tracking branch 'origin/pr/375'
* origin/pr/375:
  Fix line lengths
  Return better error messages from file pool
  Fix bugs found by Rusty Bird
  Fix export locking
  Re-add dirty check in case qubesd is restarted
  File volumes are started NAND exported
  file pool: snapshotting dirty volume not supported
  Always snapshot in the FILE pool
2020-11-27 00:19:51 +01:00
Marek Marczykowski-Górecki
b3bb65d3b0
Merge branch 'tests20201030'
* tests20201030:
  tests: fix test_550_ipv6_spoof_ip
  tests: wait for full VM startup before testing network and few other
  tests: fix logging network info on failed test
  tests: use dup-ed stdin in wait_on_fail
  tests: add PVH grub2 tests
  tests: adjust for applications list stored in features dict
  tests: remove test_030_clone
  tests: make sure dnsmasq is stopped before starting it again
  tests: fail the test early if VM fails to start
2020-11-26 22:44:39 +01:00
Demi Marie Obenour
542fee173d
Fix line lengths 2020-11-26 13:40:07 -05:00
Demi Marie Obenour
09785449ed
Return better error messages from file pool
A `qubes.storage.StoragePoolException` will be returned as a useful
error from `qvm-backup`.
2020-11-26 11:17:33 -05:00
Marek Marczykowski-Górecki
86ccbcc206
tests: fix test_550_ipv6_spoof_ip
Use 'ip route replace' instead of 'ip route add' to not fail on already
existing route entry. This worked before only because of a race
condition with a network setup.
2020-11-26 05:20:49 +01:00
Marek Marczykowski-Górecki
7d8d0c10d1
tests: wait for full VM startup before testing network and few other
Network may be configured (by qubes-misc-post service in some cases)
after qrexec is started. Delay any test after VM is fully started (as
the user would do).
Adjust wait_for_session function, to re-use VM startup timeout, which
make it adjustable for slower machines (like nested virt on openQA).
2020-11-26 05:19:53 +01:00
Marek Marczykowski-Górecki
12bc523b98
Make run-tests script try to find qrexec
Parts of core-admin (specifically some extensions) require qrexec python
module to work, and to be tested. Try to find it in a standard place (as
qubes-builder place it).
2020-11-26 05:13:56 +01:00
Marek Marczykowski-Górecki
77323cd30d
Start using 'async def' syntax
The Python version (and pylint) are new enough finally.
Adjust QubesVM.run* functions for now.
Python 3.8 brings also AsyncMock() which makes tests slightly less ugly.
2020-11-26 05:13:48 +01:00
Marek Marczykowski-Górecki
8b321e8fb0
gitlab-ci: install dependencies via dnf
Make them available system-wide, not only in user home - needed to call
some of the code via sudo in tests.
2020-11-26 05:13:38 +01:00
Demi Marie Obenour
7275939000
Fix bugs found by Rusty Bird 2020-11-25 16:42:35 -05:00
Demi Marie Obenour
ec51673f21
Fix export locking
If qubesd has restarted then _export_lock will be None
2020-11-25 15:08:13 -05:00
Demi Marie Obenour
e53d040051
Re-add dirty check in case qubesd is restarted 2020-11-25 13:33:32 -05:00
Demi Marie Obenour
e4854df42f
File volumes are started NAND exported
So add a lock to ensure this.
2020-11-25 13:25:57 -05:00
Demi Marie Obenour
14e9154e4e
file pool: snapshotting dirty volume not supported
Raise a NotImplementedError rather than risking corruption.
2020-11-24 19:15:25 -05:00
Marek Marczykowski-Górecki
5e4b495682
gitlab-ci: install pip before using it, adjust pylint call 2020-11-22 00:54:44 +01:00
Demi Marie Obenour
cee8201989
Always snapshot in the FILE pool
We must snapshot a VM’s disk before exporting it.  Otherwise, we will
likely corrupt the VM’s filesystem.

Fixes https://github.com/QubesOS/qubes-issues/issues/4324
2020-11-20 16:50:56 -05:00
Marek Marczykowski-Górecki
2c8057c43a
gitlab-ci: add tests job
- run tests in a vm runner to have LVM and loop devices access
- upload coverage data via wrapper to extract PR number
2020-11-18 05:47:50 +01:00
Marek Marczykowski-Górecki
a40344252a
rpm: fix %if syntax
rpm in fedora-33 complains about not quoted strings:

    error: bare words are no longer supported, please use "...":  xxen == xxen
2020-11-17 23:06:46 +01:00
3hhh
c717dacd8d
tests/lvm: re-introduce POOL_CONF
It's still needed by the lvm integration tests. If possible, it should
be avoided inside lvm unit tests though as using it will break
compatibility with callback unit tests.
2020-11-17 17:43:37 +01:00
Frédéric Pierret (fepitre)
a21783e5f6
Add .gitlab-ci.yml 2020-11-15 23:31:36 +01:00
Marek Marczykowski-Górecki
8233009e2a
tests: fix logging network info on failed test
- log from requested VM
- use 'critical' log level to reach journalctl
- log /var/log/xen/xen-hotplug.log too
2020-11-13 04:41:03 +01:00
Marek Marczykowski-Górecki
79fb0cade1
tests: use dup-ed stdin in wait_on_fail
transport.close() (necessary to detach reader from the loop) will close
the FD. In case of stdin, it will prevent another call to this waiting
function. Use dup(2) to register cloned FD, which is safe to close
without side effects.
2020-11-13 03:21:37 +01:00
Marek Marczykowski-Górecki
6db24d3eaf
tests: add PVH grub2 tests 2020-11-12 06:00:21 +01:00
Marek Marczykowski-Górecki
8f38753bdb
Re-enable SMAP for VMs
Buggy Linux version is no longer present in any supported template (the
last one was Debian jessie).

QubesOS/qubes-issues#2881
2020-11-05 05:28:39 +01:00
Marek Marczykowski-Górecki
73e55eb99a
tests: adjust for applications list stored in features dict 2020-11-02 01:44:32 +01:00
Marek Marczykowski-Górecki
2b49979c2a
tests: remove test_030_clone
Since qubesd-side clone_vm function is gone, it doesn't make sense to
test it. For some time already this tested only if manual step-by-step
clone implemented _in the test itself_ was done correctly.

Actual cloning is part of qubes-core-admin-client and is tested there.
2020-11-02 01:44:17 +01:00