Commit Graph

6176 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
9e39f1d207
Merge remote-tracking branch 'origin/pr/262'
* origin/pr/262:
  tests/api_admin: list serialized into a str property, using 'dns' property
  vm/net: make 'dns' property visible in Admin API
2021-05-30 03:45:57 +02:00
Marek Marczykowski-Górecki
e258dd0864
Merge remote-tracking branch 'origin/pr/398'
* origin/pr/398:
  api: improve handling of removed VMs just before the call (try 2)
2021-05-30 03:44:46 +02:00
Demi Marie Obenour
d669068b43
Do not throw out the whole environment
It is too risky from a compat perspective.
2021-05-14 17:05:04 -04:00
Demi Marie Obenour
93b8c34053
Better error for read-only properties
Previously, the Admin API caller would bet a misleading error message
indicating that the property did not exist.
2021-05-14 17:05:04 -04:00
Demi Marie Obenour
b7b6d1907c
Use a bare-bones environment for LVM and blkdiscard
They don’t need much!
2021-05-14 17:05:03 -04:00
Demi Marie Obenour
0c943dfcd6
Remove obsolete code no longer needed
- We now use a newer LVM version in CI
- Properties default to read-only in Python if no setter is provided
2021-05-14 17:04:44 -04:00
Marek Marczykowski-Górecki
7b1b2672d0
Merge remote-tracking branch 'origin/pr/395'
* origin/pr/395:
  tests: set whonix-gw-based VMs to provide network
2021-05-13 20:08:34 +02:00
Marek Marczykowski-Górecki
7c62e9703c
Merge remote-tracking branch 'origin/pr/405'
* origin/pr/405:
  Test that quotes and ampersand are XML escaped
2021-05-13 20:04:29 +02:00
Marek Marczykowski-Górecki
3678ba077f
Merge remote-tracking branch 'origin/pr/402'
* origin/pr/402:
  Do not require all the autostart VMs to run before user login
2021-05-13 20:04:04 +02:00
Demi Marie Obenour
5db7af0ce4
Test that quotes and ampersand are XML escaped
This ensures that we are safe if we interpolate into an XML attribute.
2021-05-13 12:52:48 -04:00
Brendan Hoar
16d069289c
Update lvm.py
Break the ioctl calls from blkdiscard into max 1GB worth of blocks per call. This ensures that the amount of pending discards (both for the virtual lvm thin volume as well as any lower level device recipients) do not dominate the IO streams to the devices. Before this change, IO blocking was leading to both UI and processing freezes in Qubes while very large LVM volumes were being removed. The change will allow other device IOs to be interleaved between the internal ioctl calls from blkdiscard.
2021-05-12 18:18:11 -04:00
Marek Marczykowski-Górecki
3766413dea
Do not require all the autostart VMs to run before user login
Allow user login while some VMs are still starting. This improves UX of
the system startup, since user can start using the system earlier.

Fixes QubesOS/qubes-issues#3149
2021-05-04 04:00:16 +02:00
3hhh
d2ab40de54
api: improve handling of removed VMs just before the call (try 2)
If the destination domain doesn't exist anymore when the call gets
to qubesd, inform the client accordingly rather than giving it
a generic PermissionDenied error.
This enables client applications to handle such edge cases. Some
may want to inform the user about an incorrect request, others
(e.g. qvm-ls) may want to remove the specific domain from their
result list or try again later.

Fixes QubesOS/qubes-issues#5105
2021-04-09 16:46:07 +02:00
Marek Marczykowski-Górecki
f9bfcd8732
tests: fix starting VMs in mime test
Fixes 79d4b7162a
2021-03-18 04:09:42 +01:00
Marek Marczykowski-Górecki
df6407f508
version 4.1.20 2021-03-03 19:28:02 +01:00
Marek Marczykowski-Górecki
852b44e984
tools/qubesd-query: limit maximum payload size
Qubesd limits max payload to 64kb. Do the same in qubesd-query, to avoid
loading to memory potentially unbounded amount of data that would be
refused later anyway.

Reported by @DemiMarie
2021-03-03 18:31:35 +01:00
Marek Marczykowski-Górecki
a0f82a2a93
Limit maximum length of kernelopts property
Theoretically libvirt/libxl should reject too long values anyway, but
lets provide early feedback and reject value that would cause VM start
fail later.

Reported by @DemiMarie
2021-03-03 18:31:35 +01:00
Marek Marczykowski-Górecki
d95b42df16
Validate network related features values
Validate the syntax of net.fake-* features values. While improper values
currently would lead "only" to lack of network connectivity for a VM,
the scripts handling those may assume they are properly formatted
already. Furthermore, this gives better error reporting to the user.
2021-03-03 18:31:35 +01:00
Marek Marczykowski-Górecki
e1a2f8dcb2
Enable autoescape in libvirt xml template
This avoids XML-injection by a malformed property value. If a property
value is controlled by a less privileged entity (like Management VM), it
could lead to a privilege escalation.

Reported by @DemiMarie
2021-03-03 18:31:34 +01:00
Marek Marczykowski-Górecki
2c83c7b16c
qmemman: fix setting logger for foreground mode 2021-03-03 18:30:38 +01:00
Marek Marczykowski-Górecki
740a56781e
qmemman: Load default log-level value if not present in the config
Don't crash if the config wasn't updated.

Fixes QubesOS/qubes-issues#6427
2021-03-03 18:30:37 +01:00
Marek Marczykowski-Górecki
3b1e57dc73
Enable pylint for qubes.tools.qmemmand
Adjust the code to make pylint pass. It's just one part of qmemman, the
smalest one.
2021-03-03 18:30:37 +01:00
Marek Marczykowski-Górecki
18e8edfe65
tests: set whonix-gw-based VMs to provide network
Avoid whonixcheck complaining about wrong VM type, which interferes with
GUI tests.
2021-02-26 13:55:25 +01:00
Marek Marczykowski-Górecki
e7888ffc9b
version 4.1.19 2021-02-25 17:43:50 +01:00
Marek Marczykowski-Górecki
41e559d999
tests/api_admin: list serialized into a str property, using 'dns' property
QubesOS/qubes-issues#5050
2021-02-24 14:02:23 +01:00
Marek Marczykowski-Górecki
2a42b7c7c7
vm/net: make 'dns' property visible in Admin API
Serialize the list with a space as a separator.

QubesOS/qubes-issues#5050
2021-02-24 14:01:13 +01:00
Rusty Bird
d4b1794c15
storage: avoid concurrent umask()
umask() modifies the whole process, which could lead to odd permission
problems in concurrent code. Use explicit fchmod() calls instead.

Preserve the legacy storage/file behavior of creating files 0o664 in
FileVolume.create() and 0o644 (from the normal qubesd umask) otherwise.
2021-02-15 15:14:03 +00:00
Rusty Bird
5d2fe7643c
storage/file: delete -cow.img.old on volume remove() 2021-02-15 14:10:47 +00:00
Rusty Bird
c7d74f2a63
storage/file: add is_outdated() that always returns False
Fixes QubesOS/qubes-issues#6398
2021-02-15 14:10:46 +00:00
Rusty Bird
87b9ed2a5f
storage/reflink: consistently use _io suffix 2021-02-15 14:10:45 +00:00
Rusty Bird
c56f06166b
storage/reflink: tweak FICLONE definition
Get rid of an otherwise unused module-level variable (HOST_MACHINE), and
show the relevant file (reflink.py) in the error message:

Traceback (most recent call last):
  File "/usr/bin/qubesd", line 2, in <module>
    from qubes.tools.qubesd import main
  File "/usr/lib/python3.8/site-packages/qubes/__init__.py", line 801, in <module>
    from qubes.app import Qubes
  File "/usr/lib/python3.8/site-packages/qubes/app.py", line 64, in <module>
    import qubes.storage.reflink
  File "/usr/lib/python3.8/site-packages/qubes/storage/reflink.py", line 46, in <module>
    FICLONE = {
KeyError: 'armv7l'
qubesd.service: Main process exited, code=exited, status=1/FAILURE
2021-02-15 14:10:44 +00:00
Rusty Bird
d1800f4960
vm/qubesvm: omit stray 'import qubes.storage.file' 2021-02-15 14:10:43 +00:00
Rusty Bird
7a43ee6ae2
utils: assume Fedora 18+ in dom0 2021-02-15 14:10:42 +00:00
Rusty Bird
7860c3b365
utils: "parallel" -> "concurrent" 2021-02-15 14:10:41 +00:00
Rusty Bird
e0fcbd3498
Drop obsolete core2 dvmdata/ and servicevms/ directories 2021-02-15 14:10:39 +00:00
Marek Marczykowski-Górecki
a1931057b6
Merge remote-tracking branch 'origin/pr/381'
* origin/pr/381:
  api/admin: add 'wait' parameter to admin.vm.Shutdown
2021-02-15 08:54:55 +01:00
donoban
d5e77066e9
Removed unused method get_loglevel_from_verbosity() 2021-02-13 00:47:44 +01:00
donoban
c9a0f4ea66
Parse logLevel from /etc/qmemman.conf 2021-02-13 00:38:52 +01:00
donoban
9571765b4c
Remove '/var/log/qmemman.log' backward compatiblity 2021-02-11 23:57:00 +01:00
Marek Marczykowski-Górecki
e1991d5c33
Merge remote-tracking branch 'origin/pr/389'
* origin/pr/389:
  app: save qubes.xml with utils.replace_file()
  app: use suppress() in simple cases
  firewall: save firewall.xml with utils.replace_file()
  utils: take tweaked helper functions from storage/reflink
  storage/reflink: quote logged filenames
2021-02-11 13:48:12 +01:00
Marek Marczykowski-Górecki
6abaa7b619
Merge remote-tracking branch 'origin/pr/390'
* origin/pr/390:
  Fix pylint invalid-overridden-method warning
  Fix asyncio.Lock usage for Python 3.9+
  Fix formatting glitches after conversion to await
2021-02-11 13:15:45 +01:00
Rusty Bird
0ca5e87129
Fix pylint invalid-overridden-method warning
"Method 'start' was expected to be 'async', found it instead as
'non-async'"

QubesOS/qubes-issues#2738
2021-02-11 11:17:43 +00:00
Rusty Bird
a7fe59449a
Fix asyncio.Lock usage for Python 3.9+
'with (yield from alock):' is incompatible with Python 3.9+.

Change it to 'async with alock:', and then change the affected functions
to 'async def'.

This makes the test suite pass again in a Fedora 33 VM.

QubesOS/qubes-issues#2738
2021-02-11 11:17:41 +00:00
Rusty Bird
7a91af9b2c
Fix formatting glitches after conversion to await
QubesOS/qubes-issues#2738
2021-02-11 11:17:40 +00:00
Marek Marczykowski-Górecki
0605a1197a
Merge remote-tracking branch 'origin/pr/386'
* origin/pr/386:
  log: don't write to qubes.log or vm-*.log, only stderr
  log: remove unused DBusHandler
  log: avoid qubesd restart on logrotate
  log: remove orphaned LOGFILE variable
2021-02-11 11:32:07 +01:00
Marek Marczykowski-Górecki
df7cd63bd1
Merge remote-tracking branch 'origin/pr/384'
* origin/pr/384:
  storage/file: fix is_dirty() false positive
  storage/file: refactor is_dirty()
2021-02-11 11:26:53 +01:00
Rusty Bird
05eb051193
storage/file: fix is_dirty() false positive
is_dirty() returned a false positive if the volume was merely the source
of a currently running volume. For example, if fedora-33:root was the
source volume for myappvm:root and myappvm was running - then is_dirty()
returned True for fedora-33:root, because fedora-33/root-cow.img
contains some allocated blocks (one 256 KiB chunk containing only the
header) in this scenario, even though fedora-33 is shut down.

Fixes QubesOS/qubes-issues#6371
2021-02-11 09:34:48 +00:00
Shawn Anastasio
801a8f7154
storage/reflink: Add IOCTL definitions for ppc64le
Update the hardcoded x86_64-only IOCTL constants to match the correct
values for the host architecture.
2021-02-10 15:42:20 -06:00
Rusty Bird
12d117b20a
app: save qubes.xml with utils.replace_file()
That takes care of the missing fsync() calls.

Fixes QubesOS/qubes-issues#3376
2021-02-10 12:58:02 +00:00
Rusty Bird
9b6d082673
app: use suppress() in simple cases 2021-02-10 12:58:01 +00:00