Qubes/core-admin
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
4,167 Commits 1 Branch 0 Tags 15 MiB
1cf701ff60
Commit Graph

613 Commits

Author SHA1 Message Date
Wojtek Porczyk
27d0e11872 qubes/tests/vm: fix tests 2016-06-02 15:46:01 +02:00
Wojtek Porczyk
a615a45ecd Merge remote-tracking branch 'woju/pull/13/head' into core3-devel 2016-06-02 13:24:15 +02:00
Marek Marczykowski-Górecki
7e0af81ecc qubes/vm: fix PCI device detach 2016-06-02 13:22:48 +02:00
Marek Marczykowski-Górecki
485e75091b tests: fix checking rename 2016-06-02 13:22:43 +02:00
Marek Marczykowski-Górecki
077dec8d2e tests: fix rename test 2016-06-02 13:22:40 +02:00
Marek Marczykowski-Górecki
bb2e1f2870 qubes/vm: handle VM toplevel directory in QubesVM object not Storage 
This directory is not only for disk images (in fact disk images may be
elsewhere depending on choosen volume pool), so it would be cleaner to
handle (create/remove) it directly in QubesVM class.
 2016-06-02 13:22:07 +02:00
Marek Marczykowski-Górecki
ed6e69b77e qubes/vm: minor 2016-06-02 13:14:19 +02:00
Marek Marczykowski-Górecki
2d8d78bebe qubes/vm/standalonevm: set volume_config 
Otherwise VM wouldn't have any hard disk...
 2016-06-02 13:14:02 +02:00
Marek Marczykowski-Górecki
5b5f290c23 qubes/vm: fix setting autostart property 
This is actually workaround for systemd bug reported here:
https://bugzilla.redhat.com/show_bug.cgi?id=1181922

qubesos/qubes-issues#925

This is migration of 9bfcb72722 commit to
core3.
 2016-06-02 13:13:21 +02:00
Marek Marczykowski-Górecki
b37bf55f5e qubes/vm: fix handling rename 
Libvirt VM config is no longer named after VM.
 2016-06-02 13:10:56 +02:00
Marek Marczykowski-Górecki
5e2b617c6f qubes/vm: fix parameters for some even handlers 2016-06-02 13:10:43 +02:00
Marek Marczykowski-Górecki
2bb73ab0a1 qubes/vm: add validators for many properties 2016-06-02 13:10:28 +02:00
Marek Marczykowski-Górecki
36644f3710 qubes/vm: initialize vm.volumes in one place 
Move it to QubesVM, instead of each class separately.
 2016-06-02 13:10:02 +02:00
Marek Marczykowski-Górecki
28591fa6be tests: remove/skip some tests 
1. It is unclear yet whether dispvm_netvm will be implemented in core3, but
probably not.
2. Remove tests for setting memory/cpu above host resouces - rejecting
those values at property set time would break backup restore on some
machines (when migrating from bigger to smaller system).
 2016-06-02 13:09:39 +02:00
Marek Marczykowski-Górecki
bf0966bfcf tests: fix get_label mockup to return meaningful values 
Reject invalid argument.
 2016-06-02 13:08:48 +02:00
Marek Marczykowski-Górecki
98115eb541 tests/backup: fix to core3 API - there is no verbose= argument 2016-06-02 13:08:03 +02:00
Marek Marczykowski-Górecki
0a21300872 qubes/storage: unify _remove_if_exists argument type 
In some places full volume object was called, in others - just file
path. Since this function is also used in some volume init/teardown, use
path everywhere.
 2016-06-02 13:07:35 +02:00
Marek Marczykowski-Górecki
574834ac68 qubes/core2migration: rework load order 
To successfully load all the data, proceed in order:
- set app.default_kernel
- load all templates
- set app.default_template
- load other VMs
- update network dependencies between VMs
- set other global properties
 2016-06-02 12:44:31 +02:00
Marek Marczykowski-Górecki
554a99610f qubes/core2migration: use app.load_initial_values 
Reduce code duplication.
 2016-06-02 12:43:25 +02:00
Marek Marczykowski-Górecki
692f75353e qubes/app: split initialising empty Qubes object from saving it 
It may be useful to create fresh Qubes() object but not save it yet
before setting other things. One such case is migration from core2.
 2016-06-02 12:42:37 +02:00
Marek Marczykowski-Górecki
2909d252d6 backup: get_fisk_usage is now in storage.file 2016-06-02 12:41:59 +02:00
Marek Marczykowski-Górecki
ff78b26f66 qubes: implement offline mode 
Apparently the most important (the only?) property required in offline
mode is "is_running". So let's patch it to return False and make sure
any other libvirt usage would result in failure.

Or maybe better simply returh False in vm.is_running, when libvirt
connection fails? But then it would not be possible to use offline mode
and have (some, probably unrelated) libvirtd running at the same time.

Fixes QubesOS/qubes-issues#2008
 2016-06-02 12:41:26 +02:00
Marek Marczykowski-Górecki
b119b2c36b storage/file: simplify search for origin volume of snapshot/read-only 
Check directly vm.template, throwing AttributeError when not found.
There may be some value in converting it to more descriptive error, but
since that's mostly for internal users (not user facing actions) don't
bother for now.

QubesOS/qubes-issues#1842
 2016-06-02 12:38:07 +02:00
Marek Marczykowski-Górecki
07b72ef3fd tests: fix after moving VMCollection to qubes.app 2016-06-02 12:37:30 +02:00
Marek Marczykowski-Górecki
c965024287 qubes/vm: Implement Disposable VM 
Implement DispVM as a VM based on AppVM.

QubesOS/qubes-issues#866
 2016-06-02 12:37:19 +02:00
Marek Marczykowski-Górecki
54c70766a4 qubes/storage: allow snapshots of snapshots 
It may make sense to create 'snapshot' volume out of already 'snapshot',
not only 'origin'. In pracice it will exactly the same as 'snapshot
connected directly to 'origin'.

QubesOS/qubes-issues#866
 2016-06-02 12:37:00 +02:00
Marek Marczykowski-Górecki
48176d51f1 qubes/storage: minor error message fix 2016-06-02 12:29:39 +02:00
Marek Marczykowski-Górecki
d80f34888d qubes/storage: fix getting origin pool 
volume_config parameter is about target volume, not origin.

QubesOS/qubes-issues#1842
 2016-06-02 12:29:30 +02:00
Marek Marczykowski-Górecki
b24ab45d00 qubes/vm: fix network attach/detach 2016-06-02 12:28:53 +02:00
Marek Marczykowski-Górecki
3b407eb79e qubes/app: do not enable domain events when loading XML 
Event should be enabled only after all the domains are loaded (and
domain-load event if fired).

QubesOS/qubes-issues#1816
 2016-06-02 12:28:14 +02:00
Bahtiar `kalkin-` Gadimov
17790c32bb
Fix DomainPool missing a continue 2016-06-01 17:28:55 +02:00
Bahtiar `kalkin-` Gadimov
90928dc4a0
qubes-pool skip DomainPools without volumes 2016-05-30 13:31:58 +02:00
Bahtiar `kalkin-` Gadimov
c8363cfc95
qvm-pool simplify options parsing 2016-05-30 13:31:57 +02:00
Bahtiar `kalkin-` Gadimov
27305dd85d
qvm-pool actions remove access to app 2016-05-30 13:31:56 +02:00
Bahtiar `kalkin-` Gadimov
ed1a6977c0
Qubes.add_pool() add name parameter 2016-05-30 13:31:55 +02:00
Bahtiar `kalkin-` Gadimov
d703f2f44b
Add qvm-pool and manpage for it 
- Use full import paths in qvm-pool
- Add, Remove, Info and List options set `Namespace.command`. This fixes a crash
  when `-o dir_path=/mnt/foo` is specified after `-a foo xen`.
- Remove `_List`
- Remove 'added pool' and 'removed pool' messages. Unix tools are quiet
- qvm-pool call app.save()
- Rename create_parser in get_parser
- Rename local_parser variables to just parser
- qvm-pool uses print_table
 2016-05-30 13:31:54 +02:00
Bahtiar `kalkin-` Gadimov
9ef9575d4e
Fix typo in "No driver FOO for pool BAR" message 2016-05-30 13:31:54 +02:00
Bahtiar `kalkin-` Gadimov
357e8125eb
Add qubes.tools.PoolsAction 2016-05-30 13:31:53 +02:00
Bahtiar `kalkin-` Gadimov
0319df25e5
Add print_table function to qubes.tools 
- print_table uses the `column` tool with the ASCII Unit Separator to print a
pretty table
 2016-05-30 13:31:51 +02:00
Bahtiar `kalkin-` Gadimov
35974a5dbf
DomainPool check the untrusted data from qubes-db 2016-05-25 17:39:34 +02:00
Wojtek Porczyk
0484be518c Merge remote-tracking branch 'woju/pull/12/head' into core3-devel 
Conflicts:
    doc/manpages/qvm-kill.rst
 2016-05-25 11:01:19 +02:00
Bahtiar `kalkin-` Gadimov
3f5a92772a
A QubesVM always has an empty DomainPool 
- A DomainPool is initialized by QubesVM after Storage initialization on a
  `domain-load` event
 2016-05-22 22:09:56 +02:00
Bahtiar `kalkin-` Gadimov
ddf040ae64
Do not serialize the domain pool config 2016-05-22 22:09:55 +02:00
Bahtiar `kalkin-` Gadimov
baaac858bc
Add DomainPool 
- All domain pool volumes are removable volumes
- DomainVolume uses device name as vid
 2016-05-22 22:09:54 +02:00
Bahtiar `kalkin-` Gadimov
e30f894df9
Add Volume.removable field 2016-05-22 21:42:27 +02:00
Bahtiar `kalkin-` Gadimov
116ba64e51
Storage.remove() catch IO/OSError and log it 2016-05-21 01:35:32 +02:00
Bahtiar `kalkin-` Gadimov
c5810758c5
FilePool fix origin volume removale 2016-05-21 01:35:31 +02:00
Bahtiar `kalkin-` Gadimov
8959e5a77e
Implement qvm-remove 
- Remove old qvm-remove
- Remove a log line from Storage, because it prints confusing lines, like:
    Removing volume kernel: /var/lib/qubes/vm-kernels/4.1.13-6/modules.img
 2016-05-21 01:35:30 +02:00
Bahtiar `kalkin-` Gadimov
91f72dc56c
Rework argument checking when generating manpages 
Add the ability to handle commands having subcommands, like `qvm-block`

Split the ArgumentCheckVisitor in an OptionsCheckVisitor &
SubCommandCheckVisitor. The OptionsCheckVisitor checks options given
in a section named 'Options' (case insensitive), while the
SubCommandCheckVisitor dispatches on a section named 'Commands' (case
insensitive).

This also fixes finding the undocumented command arguments. The previous
solution with depart_document did not work. NodeVisitor does not dispatch to
depart_document() even if it's mentioned in the documentation.
 2016-05-21 01:25:14 +02:00
Bahtiar `kalkin-` Gadimov
e580131465
Add AliasedSubParsersAction 2016-05-21 01:25:13 +02:00
Bahtiar `kalkin-` Gadimov
910276e898
Rename want_vm_* in vmname_nargs 2016-05-21 01:24:17 +02:00
Bahtiar `kalkin-` Gadimov
d4c74d210f
Implement vmname parsing 2016-05-21 01:24:16 +02:00
Bahtiar `kalkin-` Gadimov
3549a9d4ec
Add VmNameGroup 2016-05-21 01:24:14 +02:00
Bahtiar `kalkin-` Gadimov
7fe827d858
Add VmNameAction 2016-05-21 01:24:13 +02:00
Bahtiar `kalkin-` Gadimov
c22d9e88c9
Add QubesAction 2016-05-21 01:24:12 +02:00
Wojtek Porczyk
e757444c35 qubes/tools/qvm-features: add tool for managing qvm-features 
QubesOS/qubes-issues#1637
 2016-05-19 03:02:24 +02:00
Bahtiar `kalkin-` Gadimov
a65b0edcd4 Add QubesArgumentParser.print_error() 2016-05-19 03:02:24 +02:00
Wojtek Porczyk
786884ad7a qubes: fix netvm properties and tests 
fixes QubesOS/qubes-issues#1816
 2016-05-19 03:02:23 +02:00
Wojtek Porczyk
d728f4b9ff qubes/app: reconnect to libvirtd after crash 
Sometimes libvirt crashes. After that the connection (and all
vm.libvirt_domain-s) were unusable.

fixes QubesOS/qubes-issues#990
 2016-05-05 17:33:00 +02:00
Wojtek Porczyk
e3aae7bc17 qubes: Minor fixes 
Fix mock TestApp object and argument parsing in qvm-run.
 2016-05-05 16:58:43 +02:00
Wojtek Porczyk
c76790fbde qubes/tests/vm/qubesvm: fix test for vm.internal 
On flipping .internal vm (re)creates appmenus. They need label.
 2016-05-05 14:38:04 +02:00
Wojtek Porczyk
63c09a090c qubes: Combat import cycles 
This commit eliminates import statements happening in the middle of the
file (between two classes definition). The cycles are still there. The
only magic module is qubes itself.
 2016-05-05 14:33:09 +02:00
Wojtek Porczyk
f8270a07bb qubes: add filename= argument to Qubes.save() 
fixes QubesOS/qubes-issues#1846
 2016-05-05 00:40:45 +02:00
Bahtiar `kalkin-` Gadimov
1d5b89f0d5 LinuxKernel keep track of own volumes 
This is squashed woju/qubes-core-admin#8 by @kalkin

- LinuxKernel.volumes() lists all available kernels
- LinuxKernel use kernel version as vid
- LinuxKernel add docstrings
- Linux.kernel use os.listdir instead of os.walk
- LinuxKernel dynamically list available kernels
 2016-05-04 17:21:05 +02:00
Bahtiar `kalkin-` Gadimov
7200e6153b
Rename default storage driver from xen to file 
- Rename XenPool   ⇒ FilePool
- Rename XenVolume ⇒ FileVolume
 2016-04-30 20:42:46 +02:00
Bahtiar `kalkin-` Gadimov
b2c1017488 Fix missing parameter in Pool.__init__() 2016-04-30 20:42:00 +02:00
Wojtek Porczyk
7f2f4a4e75 Fix GUI extension 2016-04-27 15:27:01 +02:00
Bahtiar `kalkin-` Gadimov
04a3e80311 SizeMixIn first assert than call super() 2016-04-25 07:17:21 +02:00
Bahtiar `kalkin-` Gadimov
29f4be0f10 If vm doesnt support volume_config raise TypeError 2016-04-25 07:17:21 +02:00
Bahtiar `kalkin-` Gadimov
37ca33b0d1 Add docstring to xen volumes implementations 2016-04-25 07:17:21 +02:00
Bahtiar `kalkin-` Gadimov
8f060a8746 Fix Pool and Volume __init__ 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
591134833b Replace Volume.__str__ with enhanced __repr__ 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
04536c5950 Don't exec app.save() after add_pool & remove_pool 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
d7ff4b9057 Move volume xml config from QubesVM to Volume 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
2e28849c90 Move pool xml config from Qubes to Pool 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
49b4951389 Storage move rename() logic to XenPool 
- Fix config renaming
 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
e3ae6cdc1b BackupTestsMixin.create_backup_vms uses volumes 
Instead of using root_img to access the path it uses now the proper volumes
 2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov
d7fd66070a Fix revert template changes test 2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov
8cc31e86a7 qvm-create handle --pool argument 2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov
2c2a778a1d Serialize volume_config from qubes.xml 2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov
fe6a35155e Move kernel file checks to LinuxKernel pool 2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov
ef485ca32a Add linux-kernel to defaults['pool_config'] 2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov
62c81044c5 Add XenPool.setup/destroy 2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov
97d04791b7 After add/remove_pool execute Pool.setup/destroy 2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov
9674d03088 Add pool LinuxKernel 2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov
d1a0542c85 Add XenPool.remove() 2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov
a37fc2464a Add XenPool.config() 2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov
971c4ae91d Add XenPool.driver field 2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov
5f7cb41a21 Move Storage.clone_disk_files logic to XenPool 
- Add XenVolume to identify volumes which can be cloned even if they are not in
the same pool
 2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov
973c83cedd Move most resize logic to XenPool 2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov
bdfb85ac19 Refactor Storage, Pool and XenPool 
- Remove all *_dev_config methods
 - Checks if a storage image exists moved to XenPool
 - Storage.remove wraps Pool.remove()
 - Stop volumes on domain sutdown/kill
 - Warn when using deprecated methods
 2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov
3c66d4b54c Fix storage_xen test 2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov
79ac3d3770 Fix storage test and simplify TestVM 2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov
88238c80f3 Add XenPool._reset_volume 2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov
3dab5193c6 XenPool add snapshot handling 2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov
792d94959f Add implementations of xen volumes 2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov
f02f9e3a41 Add XenPool init_volume 2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov
4d4b846ce8 Replace XenStorage with XenPool 2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov
32255a7916 Reverted Storage ←→ Pool dependency 
- Storage() operates on a pool and in future on multiple pools
 2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov
9d646aabd3 Add volume_config to AppVM and TemplateVM 2016-04-25 07:17:13 +02:00
Bahtiar `kalkin-` Gadimov
24193c4308 Add Volume class 2016-04-25 07:16:38 +02:00
Bahtiar `kalkin-` Gadimov
428dd5bc1b QubesVM.dir_path is set independent of storage 2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov
c3d8c899cc Add TemplateVM test for storage_xen 2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov
3c798bc825 Pool configuration include the pool name 2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov
bd4674b658 Remove obsolete tests 2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov
22d73e8fa9 Fix Pool dir_path normalization 2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov
36470310a2 Replace pool config parsing logic 
- Move add_pool/remove_pool to Qubes class
- Add Qubes.get_pool
- Remove storage.conf
 2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov
c791cb1935 Serialize pool configuration to XML 2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov
47e89d84b6 XenStorage.format_disk_dev returns now BlockDevice 2016-04-25 07:16:36 +02:00
Bahtiar `kalkin-` Gadimov
cc7dd625d9 Loop over QubesVM.block_devices in libvirt xml 2016-04-25 07:16:36 +02:00
Bahtiar `kalkin-` Gadimov
2f99efa4b8 Add BlockDevice 2016-04-25 07:16:36 +02:00
Wojtek Porczyk
62da967f78 Fix generating documentation 2016-04-22 15:23:39 +02:00
Bahtiar `kalkin-` Gadimov
ebb79e9c4f Fix TC_01_Properties/test_030_rename_conflict_app 2016-04-20 14:02:04 +02:00
Bahtiar `kalkin-` Gadimov
d1d72466d6 Remove unneeded imports from tests/int.basic 2016-04-20 13:59:55 +02:00
Bahtiar `kalkin-` Gadimov
f74646fb1b Assert that a TemplateVM can't have a template 2016-04-20 13:59:32 +02:00
Bahtiar `kalkin-` Gadimov
6aac0a5732 On startup-failure only force_shutdown if running 
This avoids losing the exception if an exception is raised in
self.force_shutdown(), because the vm is not running or paused
 2016-04-20 13:59:17 +02:00
Wojtek Porczyk
2a9752716c Merge remote-tracking branch 'marmarek/core3-devel-mm' into core3-devel 2016-04-20 13:55:46 +02:00
Wojtek Porczyk
fe8fdb264b qubes/vm/qubesvm: move is_guid_running to GUI extension 2016-04-20 13:54:56 +02:00
Wojtek Porczyk
adbca5c0f6 qubes/tools/qvm_run: fix --all/--passio exclusion 2016-04-20 13:54:56 +02:00
Wojtek Porczyk
e720e1634b qubes/tools: rewrite qvm-shutdown 
fixes QubesOS/qubes-issues#1228
 2016-04-20 13:52:58 +02:00
Wojtek Porczyk
94589915b0 Fix documentation 2016-04-20 13:50:42 +02:00
Marek Marczykowski-Górecki
2a46abbefd
qubes.vm: implement TemplateVM.appvms property 2016-04-20 02:28:11 +02:00
Marek Marczykowski-Górecki
7117847816
tests: fix int.dom0_update 
Environment must be preserved for QUBES_XML_PATH variable used to point
to a secondary (test) qubes.xml.
 2016-04-20 02:26:56 +02:00
Marek Marczykowski-Górecki
a707840596
tests: fix and enable external tests loader 
QubesOS/qubes-issues#1800
 2016-04-20 02:26:16 +02:00
Marek Marczykowski-Górecki
a1575b98d4
tests: do not drop 'qubes.tests.' prefix 
This cause troubles with "external" tests (loaded from outside of
qubes.tests module).
 2016-04-20 02:25:56 +02:00
Wojtek Porczyk
540942de47
qubes/ext: convert extensions to singletons 
From now the extensions are instantiated once. They no longer have .app
attribute, but can access it from event handlers via vm.app.
 2016-04-11 15:31:41 +02:00
Marek Marczykowski-Górecki
eaf5efd814
tests: convert 'extra' test loader to core3 API 
QubesOS/qubes-issues#1800
 2016-04-11 13:50:19 +02:00
Marek Marczykowski-Górecki
8c6fe7ed90
Merge remote-tracking branch 'origin/master' into core3-devel-mm 2016-04-11 13:03:12 +02:00
Marek Marczykowski-Górecki
4308fa7946
Fix events names in docstrings 2016-04-11 00:13:14 +02:00
Marek Marczykowski-Górecki
d9a8cd5d68
backup: minor code style fixes 2016-04-11 00:13:13 +02:00
Wojtek Porczyk
c5962910d0 qubes/events: Rework firing events for effect 
From now, the handlers should yield their values, not return.
 2016-04-08 12:35:11 +02:00
Marek Marczykowski-Górecki
c28f50f6fe backup: minor fixes 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
2261e6a4c7 backup: simplify auto-renaming handling 
Don't try to track names in restore_info_verify, instead resolve them at
actual restore time.
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
71a7730168 backup: simplify ordering of restoring VMs 
No other change than reducing two nested loops into one.
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
20d53fbf69 tests: more VMs combinations in backup 
VM with custom template, non-default NetVM

This requires to VMs being removed in reversed order (to first remove
VM, then its template).
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
98c8b7cd22 backup: use simple classes for data storage on restore too 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
424d3054f3 backup: use vm.features to store backup-specific metadata 
Do not keep them in system qubes.xml.
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
ed88b5a9a3 backup: use vm.run_service instead of vm.run 
This should be the only way used to call services.
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
e8f21929ad backup: by default include all the VMs with vm.include_in_backups 
If there is a need for more robust default, it should be handled at that
property.
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
a3051b956b backup: docstrings, don't expose internal attributes 
processes_to_kill_on_cancel shouldn't be initialized by user, so move
below applying user settings.
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
b4227f7581 backup: use simple classes instead of hashes, use events for appmenus 
This makes typo errors much easier to find (also using pylint or so).
While at it, also removed explicit appmenus backup, as it should be
provided by appmenus extension.
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
c01f43a5d3 backup: unify handling of template and non-template VMs 
Handle them the same way - individual files, not the whole directory for
templates.
Also don't backup obsolete 'kernels' subdir - it isn't supported in
core3.
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
d9cf64a41d backup: minor code structure changes 
- initialize internal objects object in constructor
- use 'with'
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
78dbadab57 backup: minor: compile regex once 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
e9b8b6428c backup: remove duplicated get_disk_usage function 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
c08f5986a9 backup: use constants for queue control strings 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
ad1f1738fa qubes: make 'store' property public (R/O) 
It makes sense to see from what Qubes object was loaded.
 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
c55ca8004a tests: port backupcompatibility to core3 API 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
8c5d42a095 backup: add support for restoring pre-core3 backups 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
304d2b10bc backup: fix handline "None" kernel 2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki
412bcfa88b backup: fix hadling netvm property 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
d72d8f8445 backup: code inspection annotations 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
3fb0754398 backup: formatting 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
697a26c8da backup: update license header 
Use link instead of address, as recommended by FSF.
 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
e04ea7512c backup: minor code style 
- Don't use catch-all except statement.
- Use str.format instead of "%" operator.
- Use static methods where applicable.
- Remove unused local variables.
- Don't shadow variables from outer scope
 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
44eed25511 tests/backup: report failure when error was expected but none detected 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
e5af64f2bf backup: fix restore summary table 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
595dfdc0a9 backup: add additional verification of backup header 
Ensure only alphanumeric characters are used.
 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
3342f637f4 backup: convert logging to proper python logging API 
QubesOS/qubes-issues#1213
QubesOS/qubes-issues#1214
 2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki
019678bf4d tests: fix backup test to not break default template... 
By default AppVM is template based. This means vm.root_img points at
default template's root image. Change this to StandaloneVM to have
independent root.img.
 2016-04-07 13:06:25 +02:00
Marek Marczykowski-Górecki
e557fe9989 backup: restructude the code into classes 
Introduce two main classes Backup and BackupRestore for storing the
state of the desired operation. Then a simple interface to adjust
parameters.

(Almost) no functional change.

QubesOS/qubes-issues#1213
QubesOS/qubes-issues#1214
 2016-04-07 13:06:14 +02:00
Marek Marczykowski-Górecki
fd1f8def6a tests: do not store dom0 home backup inside of dom0 home itself 2016-04-07 13:05:30 +02:00
Marek Marczykowski-Górecki
e0686e1e02 backup: initial conversion to core3 API 2016-04-07 13:05:25 +02:00
Marek Marczykowski-Górecki
45d6ab3862 qubes/vm: store libvirt config in libvirt.xml, fix rename handling 2016-04-07 13:03:46 +02:00
Marek Marczykowski-Górecki
12df07edd0 tests: invalid property should not be saved to qubes.xml 2016-04-07 13:03:42 +02:00
Marek Marczykowski-Górecki
de86a3c1a4 tools/qvm_prefs: reject invalid properties instead of ignoring silently 2016-04-07 13:03:39 +02:00
Marek Marczykowski-Górecki
c07981da88 tests: allow real prefix notation for selecting tests 2016-04-07 13:03:35 +02:00
Marek Marczykowski-Górecki
7f5feac98b qubes/vm: fixes for features handling 
Properly handle 'None' value, there is no difference between empty
string (<x></x>) and no content at all (<x/>).
 2016-04-07 13:03:30 +02:00
Marek Marczykowski-Górecki
06c315c94b qubes: fix/update event names 
Fix few typos, then rename:
 - recreate-appmenus -> domain-restore: there may be more to do than
 only appmenus
 2016-04-07 12:59:39 +02:00
Marek Marczykowski-Górecki
7cbe2dbd38 qubes/vm: add passio_stderr argument to vm.run_service 
This allows getting error messages from service call, to provide more
meaningful error messages.
 2016-04-07 12:58:05 +02:00
Marek Marczykowski-Górecki
1cd6acd875 core/events: allow handlers return some value 2016-04-07 12:58:01 +02:00
Marek Marczykowski-Górecki
6052143735 tests: qvm-prefs 
QubesOS/qubes-issues#1248
 2016-04-07 12:57:57 +02:00
Marek Marczykowski-Górecki
e319639146 qubes/vm: add StandaloneVM class 2016-04-07 12:57:52 +02:00
Marek Marczykowski-Górecki
43435405e7 tests: update checking 'None' VM representation 2016-04-07 12:57:47 +02:00
Marek Marczykowski-Górecki
c17f86e659 qubes: fix handling DEFAULT special value for VMProperty 2016-04-07 12:03:40 +02:00
Wojtek Porczyk
267a78e4e0 Fix resetting attribute to default when it's already default 
Based on marmarek's
  WIP fix resetting attribute to default when it's already default
 2016-04-06 18:09:57 +02:00
Bahtiar `kalkin-` Gadimov
b1978abce5 Use entry_points for pool driver discovery 
- Add qubes.storage entry point to setup.py
- Removed the old pool driver class loading logic
- Reworked pool tests
 2016-04-06 10:11:19 +02:00
Wojtek Porczyk
42666e0ec5 Throw meaningful exception when qubes.xml is missing 
Based on kalkin-'s Rais QubesException if qubes xml config is missing
 2016-04-05 16:33:49 +02:00
Bahtiar `kalkin-` Gadimov
156ecf4163 Add missing import in qubes/__init__.py 
This was: Fix qubes-create command
 2016-04-05 16:25:51 +02:00
Bahtiar `kalkin-` Gadimov
49ec29a235 Fix bug in Qubes.get_vm_class() 2016-04-05 16:24:30 +02:00
Wojtek Porczyk
c375334c54 qubes/devices: calculate event names using + 
This make static analysis of the events easier.
 2016-03-23 09:08:13 +01:00
Wojtek Porczyk
349e218638 qubes/devices: devices class framework 
This fixes loading PCI devices.
 2016-03-23 09:08:13 +01:00
Wojtek Porczyk
a4fa1adb82 qubes/ext/gui: fix sending monitor layout at VM startup 
Based on marmarek's commit 4edb42c1 under the same name.
 2016-03-21 11:44:59 +01:00
Marek Marczykowski-Górecki
15e032cddb qubes/vm: fix handling "None" kernel 
The "None" value still makes sense for HVM domains.
 2016-03-21 11:44:59 +01:00
Marek Marczykowski-Górecki
a02f7a30fc qmemman: fix core3 API usage for sending events 2016-03-21 11:44:59 +01:00
Wojtek Porczyk
29d0baa944 qubes/vm/mix/net: By default MAC is constant 
MAC address was one place, where qid was leaked to the VM. Different MAC
addresses are not needed, since we do routing, not switching.
 2016-03-21 11:44:59 +01:00
Wojtek Porczyk
04cc2099f7 HVM part 2 2016-03-21 11:44:54 +01:00
Marek Marczykowski-Górecki
8da7416f30 vm: Remove reference to old libvirt config template 
This file isn't installed anymore
 2016-03-21 11:44:54 +01:00
Wojtek Porczyk
5eaf03c4a2 HVM part 1 2016-03-21 11:44:46 +01:00
Wojtek Porczyk
e0210130da qubes: Fix error in get_vms_based_on() 
Not every vm has .template attribute.
 2016-03-21 11:43:33 +01:00
Wojtek Porczyk
d766b8e110 qubes: Fix "unify event names" 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
9567f7b40b vm: rename 'yum-proxy-setup' feature to 'updates-proxy-setup' 
The 'yum-proxy-setup' is deprecated since R3.0, so finally remove old
name.
But add it to R3.x compatibility layer.
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
b6d8c7fb81 ext/r3compatibility: features/services 
QubesOS/qubes-issues#1812
 2016-03-21 11:43:33 +01:00
Wojtek Porczyk
0f03b257ce qubes/vm: Fix features coercion 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
c9fd57e7e4 ext/r3compatibility: move old firewall code to R3.x compat layer 
QubesOS/qubes-issues#1812
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
afd4573a02 ext/r3compatibility: create R3.x QubesDB entries 
This allows the user to start VM based on "old" system (from R3.x) in
R4.0. For example after restoring from backup, or migration. This also
makes upgrade instruction much easier - no need complex recovery
instruction if one upgrade dom0 before upgrading all the templates.

QubesOS/qubes-issues#1812
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
54784405bd tests: adjust networking tests 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
3b1238f7cf vm: stub for new firewall API 
- introduce 'firewall-changed' event
 - add reload_firewall_for_vm stub function

Should that function be private, called only from appropriate event
handlers?

QubesOS/qubes-issues#1815
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
34fe4a2d71 tests: add function to provide real network to test case 
Very few tests should use it, as it makes the test result somehow
dependent on outside world state. But still, some of them need it.
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
c99a47d192 vm: fix handling firewall_conf path 
It may be (and by default is) path relative to VM directory.
This code will be gone in the final version, after merging firewall
configuration into qubes.xml. But for now have something testable.
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
c9cad71d03 vm/mix/net: create vm.connected_vms dynamically 
It is much less error-prone way. Previous approach didn't worked because
VMs weren't added here at 'domain-init'/'domain-loaded' event. And even
after adding such handlers it wasn't working because of
QubesOS/qubes-issues#1816.

It may be a little slower, but since it isn't used so often
(starting/stopping VM and reloading firewall), shouldn't be a problem.
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
3e3213e7c7 Redo "core: do not reset firewal when setting netvm=none" 
Apparently it got lost in merge.

QubesOS/qubes-issues#862
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
736773dbec vm: remove obsolete firewall handling code 
There is no vm.write_iptables_xenstore_entry().

QubesOS/qubes-issues#1815
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
a0fa8fcc8e ext/qubesmanager: fix events signature 
There is event itself passed just after VM object.
 2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki
12f778b539 qubes/vm: do not recursively use repr in BaseVM.__repr__ 
This will easily end up in infinite recursion. For example
'sys-net'.template points at 'fedora-23', which itself has
'fedora-23'.netvm set to 'sys-net'.
 2016-03-21 11:43:32 +01:00
Marek Marczykowski-Górecki
422f30b969 tests: port network tests to core3 API 2016-03-21 11:43:32 +01:00
Marek Marczykowski-Górecki
df5562d2c7 tests: fix rename test to report FAIL instead of ERROR when really failed 2016-03-21 11:43:32 +01:00
Wojtek Porczyk
394fa1e5d3 qubes/tests: tests for qubes.devices 2016-03-21 11:43:32 +01:00
Wojtek Porczyk
0f9ca47d90 qubes/ext/guid: Move gui-related code to extension 2016-03-21 11:43:32 +01:00
Wojtek Porczyk
75dd882b83 qubes: Unify event names 
Events will be named <object>-[pre-]-<verb>, where verb is in infinitive
form.
 2016-03-21 11:43:32 +01:00
Wojtek Porczyk
d09bd5ab6a qubes: Convert QubesVM and Extension discovery to pkg_resources 
QubesOS/qubes-issues#1238
 2016-03-21 11:43:32 +01:00
Wojtek Porczyk
93686eae06 qubes/vm: change services to features 2016-03-21 11:43:32 +01:00
Wojtek Porczyk
bf78e662f6 Import qubesdb from the new module 
fixes QubesOS/qubes-issues#1252
 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
5dca051b32 tests: enable storage tests 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
8262f2ef4b storage: fix handling standalone VMs 
Currently Standalone have template=None.
 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
a73c694f54 storage: access *_img directly from vm.storage object 
Not through vm.something_img proxy.
 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
c0c0e0022e tests: convert storage tests to core3 API 
QubesOS/qubes-issues#
 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
14f31134c0 tests: move make_vm_name to QubesTestCase 
This function is often useful also in unit tests, not only integration
tests.
 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
5375dce90d Prefer human readable output in qvm-prefs and qubes-prefs tools 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
68ed50c97d tests: enable dom0 update tests 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
fb74126e56 tests: convert dom0 update test to core3 API 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
0307ec22f2 tests: resolve template name when copying from host collection 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
81dbe3628f tests: skip system tests if not in dom0 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
50819c726c tests: set QUBES_XML_PATH env variable to force usage of qubes-test.xml 
This way even separate processes (even those started not directly - like
qrexec service calls) will use correct qubes.xml file.

Fixes QubesOS/qubes-issues#1730
 2016-03-03 01:18:17 +01:00
Wojtek Porczyk
c80e89d47e qubes: support QUBES_XML_PATH environment variable 
This is reworked
    core: load default qubes.xml location from QUBES_XML_PATH env variable

QubesOS/qubes-issues#1730
 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
e3c94f3949 core: fix handling numeric label value set 
Not sure if it should be supported this way, but surely it shouldn't
throw AttributeError.
 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
79a6da1163 tests: split off invalid values tests for better reporting 
Most of them fails for now...
 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
c45c91c5f0 tests: improve verbosity of assertNotRaises even further 
Include traceback in error message.
 2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki
5a409ad417 tests: few more (disabled for now) tests for hvm related properties 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
1b69f5c213 tests: make test cleanup more defensive - qubes.xml load failure 
If qubes.xml doesn't even load, remove it anyway and clean up the VMs
the hard way.
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
9633573408 core: adjust units comment for host.memory_total, and fix related places 
QubesOS/qubes-issues#1737
 2016-03-03 01:18:16 +01:00
Wojtek Porczyk
88a63cbe3a qubes/vm/qubesvm: do not clone VM-specific properties 
This is reworked
    core: do not clone VM-unique identifiers in clone_properties
by Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
2cbbe0bdb6 core: abandon 'kernels' in VM directory 
Since we have PV Grub now (which is much more convenient), it is useless
now.
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
9428c70d31 core: fix vm.create_on_disk 
QubesOS/qubes-issues#
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
62759fb1f6 core: move maxmem default initialization to property itself 
Also fix (for now?) units - memory_total is int bytes while maxmem in
megabytes.

QubesOS/qubes-issues#1737
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
2f3762ee68 core: minor fixes in handling properties 
Conflicts:
	qubes/vm/qubesvm.py
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
12941d67bf tests: do not unnecessary mangle test name passed as runner parameter 
Otherwise `qubes.tests.run -l` output would not be useful as input for
test list.
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
05d006b873 tests: minor fixes in int.tools tests 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
163ab25d4b tests: enable qubes.tests.int.basic 
It is fully converted to core3 API (or tests marked as 'skip' if not),
so can be reenabled.
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
9b1b82c592 tests: move properties tests to QubesVM unit tests 
Since qvm-prefs is now a generic tool without any property-specific
stuff, we can tests properties getters/setters in QubesVM class unit
tests.
Now most of properties are tested. Some of those tests still fails
because missing/incomplete underlying implementation.
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
81bc615bce tests: convert tests/int/basic to core3 API 
Not all tests converted - some of them will be removed in the next
commit. Also it doesn't mean that all tested functionality already
works...
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
2daba9e0eb core: migrate handy GUI functions and backup mixin from core2 
That's all from tests/__init__.py, so remove that file.
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
bc82643580 tests: redo "84af738 tests: support VMs cleaned up in tearDownClass..." 
Usage: VMs with name created by self.make_vm_name(name,
class_teardown=True) will be cleaned up in tearDownClass. It
should be used only in setUpClass.

Fixes QubesOS/qubes-issues#1691
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
0a45164be7 tests: few more FD leak fixes 
QubesOS/qubes-issues#1697
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
47e346efbf tests: initialize Qubes() object in SystemTestsMixin, add handy helpers 
Most of the tests needs existing template, so add helper to copy it from
"host" collection. Add also helpers to reload qubes.xml - in case of
modification of it from some other process.
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
2966456387 tests: improve assertNotRaises reporting 
Include whole exception value, not only its type.
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
f52a0f2d41 storage: get rid of 'servicevms' subdir 
Keep all the non-template VMs in 'appvms'. Having it split makes no
sense since any VM can be "converted" to NetVM by simply changing its
provides_network attribute and probably assigning some PCI device.

On the other hand, it may be useful to set this manually, for example by
management stack - based on VM function, not type. But don't care about
it for now.
 2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki
c1bab3d094 storage: reset_volatile_storage doesn't need source_template parameter 
It shouldn't touch anything not already known to the VM object.
 2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki
5f48b76b27 storage: fix vm.create_on_disk for non-template based VMs 2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki
539a46ca9a core/log: do not attach multiple handlers to the same logger 
logging.getLogger when given the same name twice, will return the same
object. Do not attach handler there every time, it will really open new
file, leading to multiplicated log entries and depleting file
descriptors.

QubesOS/qubes-issues#1697
 2016-03-03 01:18:15 +01:00