Commit Graph

248 Commits

Author SHA1 Message Date
Marek Marczykowski-Górecki
6980e7ba14
Store template repo configuration in /etc
This way it's easier to permanently enable repos.
2021-02-06 16:03:40 +01:00
Marek Marczykowski-Górecki
f1424812b0
qvm-template: improve install lock
Use fcntl.flock() instead of just file existence check, so it won't fail
on a stale lock file. While at it, move locking to a function decorator,
to de-clutter the install function a bit. This will allow reducing
indentation level, but don't do it yet, to make the patch readable.

Move lock testing into a separate test, and remove it from install
tests.

QubesOS/qubes-issues#2534
2021-02-06 16:03:39 +01:00
Marek Marczykowski-Górecki
f3954fb225
qvm-template: download templates to a temporary directory
Avoid risk of conflicting downloads to the same directory, reusing
partial downloads, leaving broken files etc. Move template package out
of temporary directory only after its verified.

QubesOS/qubes-issues#2534
2021-02-06 16:03:38 +01:00
Marek Marczykowski-Górecki
f3f6750a3f
qvm-template: call rpmkeys --checksig for signature verification
RPM API is confusing and it's easy to get it wrong when verifying
package signatures.
Call 'rpmkeys --checksig' which is more rebust here - RPM authors should
know how to use their API.

QubesOS/qubes-issues#2534
2021-02-06 16:03:38 +01:00
WillyPillow
1671b4216f
qvm-template: Add tests for download function and fix minor bugs 2020-10-04 03:05:14 +08:00
WillyPillow
63f488f64c
qvm-template: Mock print_table in tests for consistent output 2020-09-15 01:09:45 +08:00
WillyPillow
20443d5c6f
qvm-template: Add tests for functions list_templates and search 2020-09-15 00:25:49 +08:00
WillyPillow
dc26ba0ebf
qvm-template: Add tests for qrexec_repoquery and get_dl_list 2020-09-09 03:04:37 +08:00
WillyPillow
161ff01d7d
qvm-template: Fix compatibility with Python 3.6 in tests 2020-09-07 02:12:03 +08:00
WillyPillow
3fac2097eb
qvm-template: Add partial tests for qrexec_repoquery 2020-09-07 01:52:56 +08:00
WillyPillow
5e1e0daa5c
Make TestProcess.communicate return str instead of IO object 2020-09-07 01:18:59 +08:00
WillyPillow
3f75e6e49e
qvm-template: Add tests for qrexec_payload 2020-09-07 00:41:03 +08:00
WillyPillow
89895038b5
qvm-template: Fix date formats to "%Y-%m-%d %H:%M:%S" 2020-09-06 23:57:42 +08:00
WillyPillow
205eee4d80
qvm-template: Fix timezone issues by storing timezone explictly in features 2020-09-05 01:51:45 +08:00
WillyPillow
757bb33329
Add stubs for rpm module and initial tests for qvm-template install 2020-09-04 01:56:15 +08:00
WillyPillow
fbf6c4e3c3
Merge remote-tracking branch 'origin/master' into qvm-template 2020-08-31 01:53:15 +08:00
WillyPillow
4199a9a222
qvm-template: Fix qvm_template_postprocess tests 2020-08-31 01:48:31 +08:00
Marta Marczykowska-Górecka
fe583cd88e
Make incorrect label name error more descriptive
Instead of unintuitive Value Error now we have dedicated QubesLabelNotFoundError.
2020-08-13 20:21:46 +02:00
Marek Marczykowski-Górecki
6e91fba942
Merge remote-tracking branch 'origin/pr/157'
* origin/pr/157:
  qvm-run will unpause paused VMs by defaults
2020-08-11 18:42:35 +02:00
Marek Marczykowski-Górecki
b04a14685c
Merge branch 'paranoid-restore'
* paranoid-restore:
  backup/restore: make error reporting work also for StandaloneVM based DispVM
  backup/restore: better error detection for --paranoid-mode
  doc: document 'tag-created-vm-with' feature
  tests: add paranoid backup restore unit tests
  tests: remove extra empty lines
  tests: extend run_service mockup for pre-recorded output
  rpm: add BR: python3-lxml and python3-xcffib
  backup/restore: add option for unattended restore and extracting log
  tools: remove obsolete _want_app argument
  Add "paranoid restore" mode
  rpm/deb: add dependency on scrypt
  utils: add simple locking primitive
  backup/restore: use qfile-unpacker in a VM, request disk space monitoring
  backup/restore: option for alternative qrexec service
  backup/restore: improve error message about restoring tags
  backup/restore: distinguish dom0 by name
2020-08-07 02:55:27 +02:00
Marta Marczykowska-Górecka
4a6b5dbae2
qvm-run will unpause paused VMs by defaults
If qvm-run is run with the autostart option (true by default), it will
also unpause paused VMs.

fixes QubesOS/qubes-issues#5967
2020-08-05 20:49:38 +02:00
Marek Marczykowski-Górecki
1660a1cbf6
backup/restore: better error detection for --paranoid-mode
Xterm doesn't preserve exit code of the process running inside. This
means, the whole xterm always exits with 0, even if qvm-backup-restore
failed.
Fix this by printing the exit code at the end to the log and then extract
that last line from the log on the calling side. This way we can also
distinguish whether qvm-backup-restore or xterm failed.
2020-08-05 05:06:54 +02:00
Marek Marczykowski-Górecki
852b689984
tests: add paranoid backup restore unit tests 2020-08-05 05:06:53 +02:00
Marek Marczykowski-Górecki
bf7b6225e3
tests: remove extra empty lines 2020-08-05 04:37:45 +02:00
Marek Marczykowski-Górecki
7fb90e0233
tests: extend run_service mockup for pre-recorded output
And also handle input written if just stdin.flush() is called but not
stdin.close().
2020-08-05 04:37:45 +02:00
Marek Marczykowski-Górecki
f16f14b020
utils: fix encoding '+' for qubes.VMExec
'+' in an argument needs to be encoded too, otherwise it is interpreted
as arguments separator.
2020-08-04 22:17:04 +02:00
Marek Marczykowski-Górecki
cc71dd5876
Add "paranoid restore" mode
Having Admin API, it is possible to do this properly now:
 - create DisposableVM
 - assign it proper permissions to create VMs and control those created
   VMs
 - run restore process inside
 - cleanup DisposableVM afterwards

Since the RestoreInDisposableVM class contains de facto reverse parser
for qvm-backup-restore command line, add a test that will spot when it
gets out of sync.

This feature depends on modifications in various other components,
including:
 - linux-utils and core-agent-linux for update qfile-unpacker
 - core-admin for qrexec policy modification

QubesOS/qubes-issues#5310
2020-08-04 04:06:53 +02:00
Marek Marczykowski-Górecki
db1d4b5d48
backup/restore: option for alternative qrexec service
Allow setting alternative qrexec service to retrieve backup content. The
service API is slightly different than the default one: it will get only
list of files/directories to extract on its stdin, but not backup
location. The latter could be provided as a service argument, or using
other out-of-band mechanism.
This will be useful for paranoid backup restore mode, to take away
control over location/command from sandboxed qvm-backup-restore process.

QubesOS/qubes-issues#5310
2020-08-04 04:06:30 +02:00
Marek Marczykowski-Górecki
114f6fb250
backup/restore: improve error message about restoring tags
Before reporting a tag as not restored, verify if it really wasn't
restored. Generally created-by-* tags cannot be created manually. But
when restoring a backup in dom0, created-by-dom0 tag is added, which in
many cases will match what want to be restored.

Adjust tests to check this too.
2020-08-04 04:06:29 +02:00
Marek Marczykowski-Górecki
b7eca976b5
tests: use asynctest some more
Instead of hoping normal Mock will work (which does not on Python 3.8).
2020-08-03 04:55:46 +02:00
Paweł Marczewski
cb7f191bd2
qvm-start-daemon: convert to async/await syntax 2020-07-29 11:27:19 +02:00
Marek Marczykowski-Górecki
af1c1e5234
tests: update for admin.vm.volume.Clear usage in qvm-template-postprocess 2020-07-16 04:07:07 +02:00
Marek Marczykowski-Górecki
470514d0dc
Merge remote-tracking branch 'origin/pr/144'
* origin/pr/144:
  Clean up the guid-conf file on domain stop
  Generate qubes-guid options based on features
2020-07-15 14:00:31 +02:00
Marek Marczykowski-Górecki
2bdeb7684a
Merge remote-tracking branch 'origin/pr/151'
* origin/pr/151:
  Fixed inconsistent firewall address checking
2020-07-15 13:50:12 +02:00
Marek Marczykowski-Górecki
f9c5b61961
Merge remote-tracking branch 'origin/pr/150'
* origin/pr/150:
  Fix VM validity check for cached VM objects
2020-07-15 13:49:39 +02:00
Marta Marczykowska-Górecka
9377addd1e
Fixed inconsistent firewall address checking
core-admin-client side had less strict rules for hostname than core-admin,
leading to unexpected empty qubesd response. Fixed by copying the logic from core-admin.
2020-07-14 20:10:34 +02:00
Marek Marczykowski-Górecki
45a28c29ae
Fix VM validity check for cached VM objects
Qubes().domains.refresh_cache() tries to preserve cached VM objects if
the class matches - this way if an application keeps reference to any,
it will still be the same as freshly obtained from the collection, and
also it will receive cache updates/invalidates based on events.

The check for class change was invalid - on core-admin-client side we
have just one QubesVM class with 'klass' attribute. This leads to VM
objects being disconnected from VMCollection and stale properties cache
there (because they no longer receive events).

Fix the check.

And also add a test if indeed the same object is returned.
2020-07-14 16:10:49 +02:00
Paweł Marczewski
3540f04a42
Generate qubes-guid options based on features
Allow configuring options per VM or globally per GuiVM. The
qvm-start-daemon program reads the options from VM features, and
generates a configuration file for qubes-guid.

Requires QubesOS/qubes-gui-daemon#47 (customizing the configuration
file).
2020-06-29 12:01:09 +02:00
Frédéric Pierret (fepitre)
6295b0ba2f
tests: kde_args are passed with property of launcher 2020-06-26 12:05:18 +02:00
Marek Marczykowski-Górecki
b1453953f9
Merge remote-tracking branch 'origin/pr/139'
* origin/pr/139:
  Added a safeguard for invalid firewall rules
2020-05-27 04:18:20 +02:00
Marek Marczykowski-Górecki
c081ed8c82
Enable caching in qvm-ls and qvm-prefs
Both tools issue a large number of Admin API calls and greatly benefit
from a cache filled with a single per-vm Admin API call
(admin.vm.property.GetAll). In case of qvm-ls, this also saves multiple
admin.vm.CurrentState calls (power state is given in the admin.vm.List
response too).

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
79c7392424
Cache power state when caching is enabled
Power state changes are signaled with events too, so it is possible to
cache it and update/invalidate cache with events.
Additionally, admin.vm.List returns a power state, so the cache can be
populated early. This in particular greatly improves qvm-ls performance -
eliminate admin.vm.CurrentState call at all.

QubesOS/qubes-issues#3293
2020-05-22 19:28:51 +02:00
Marek Marczykowski-Górecki
bfe1a3d541
Adjust for changed qubesd socket protocol
The socket protocol is adjusted to match qrexec socket service protocol.

QubesOS/qubes-issues#3293
2020-05-22 19:01:24 +02:00
Marek Marczykowski-Górecki
02cfab8257
Try to use new property.GetAll method to pre-fill the cache
When caching is enabled, reduce number of calls by getting all the
properties at once. If the call is not available (for example because of
the policy), fallback to getting individual values.

QubesOS/qubes-issues#5415
2020-05-22 19:01:24 +02:00
Marta Marczykowska-Górecka
d2f4a4533a
Added a safeguard for invalid firewall rules
Firewall rule cannot be missing value in declaration
(e.g. 'dsthost=' is not a valid rule).

fixes QubesOS/qubes-issues#5772
2020-05-16 12:18:13 +02:00
Marek Marczykowski-Górecki
83b1fc6c58
tools/qvm-volume import: do not retrieve old size anymore
Since admin.vm.volume.ImportWithSize method is used now, there is no
need to retrieve old size anymore (to decide whether to resize or not).
2020-05-10 03:57:28 +02:00
Marek Marczykowski-Górecki
4971faa462
Merge remote-tracking branch 'origin/pr/118'
* origin/pr/118:
  qvm-start-daemon: adjust pacat pid file path
  qvm-start-daemon: check if layout is parsed
  qvm-start-daemon: allow multiple options in keyboard layout
  qvm-start-daemon: improve parsing args for setting keyboard layout
  qvm-start-daemon: set keyboard-layout only for the first set layout
  gui: set keyboard layout when starting daemon
  daemon: start it for dom0 unconditionnaly
  qvm-start-daemon: ensure separate task between GUI/AUDIO
  qvm-start-daemon: allow starting only if service enabled
  Fix and improvements from Marek's comments
  Change qvm-start-gui to qvm-start-daemon for handling audio too
  Support for AudioVM
2020-04-09 05:24:26 +02:00
Marta Marczykowska-Górecka
f08d41eec5
Added tests for cloning VMs with persistent devices 2020-03-30 16:15:49 +02:00
Marta Marczykowska-Górecka
8e7db06918
Remove Python 2 workarounds
Python 2 support was discontinued and workaround
for Python2 mock is no longer needed.
2020-03-30 15:48:53 +02:00
Frédéric Pierret (fepitre)
0e049e682a
Change qvm-start-gui to qvm-start-daemon for handling audio too 2020-03-28 17:33:02 +01:00