core-admin

Author	SHA1	Message	Date
Marek Marczykowski-Górecki	d0b8933374	devices: add assignment.device property Make it easy to retrieve DeviceInfo object out of DeviceAssignment object. The only missing piece of information for that is device class, so add it. Make it optional, as it can be filled on demand when passing the object through DeviceCollection (either by listing devices, or attaching/detaching). This is mostly to ease handling options in libvirt template - to get them, you need to use `assignments()`, istead of `persistent()` or `attached()`, but there were no _simple_ way of getting actual device object. This also makes DeviceCollection._device method not needed anymore.	2017-05-22 03:21:07 +02:00
Marek Marczykowski-Górecki	227010d433	pci: fix device-pre-attach event signature Now it contains also options. PCI extension do not use it, yet.	2017-05-19 18:49:22 +02:00
Marek Marczykowski-Górecki	f9fa27dacc	property: add property.__str__	2017-05-19 18:49:21 +02:00
Marek Marczykowski-Górecki	f7cb8f27c5	admin: add admin.vm.Remove method QubesOS/qubes-issues#2622	2017-05-19 18:49:21 +02:00
Marek Marczykowski-Górecki	0ef010cc70	qmemman: fix list/filter handling on python3 Filter in python3 returns a generator, can be iterated only once. This is about list of existing domains - store it as a list, otherwise domains will "disappear" after being discovered.	2017-05-18 09:44:48 +02:00
Marek Marczykowski-Górecki	1831088c9b	api/internal: don't try to start dom0 qrexec-policy do try to start a VM before executing service. Avoid this here, instead of qrexec-policy to keep the later clean.	2017-05-17 14:24:59 +02:00
Marek Marczykowski-Górecki	ae28d32af7	vm: make sure qmemman connection is closed even on failed VM startup Qmemman hold global lock while serving requests, so not closing connection will prevent any further action.	2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki	e5daf902b3	vm/net: fix IP address calculation 7 is not the same as 7 bits...	2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki	4d6d6c913a	qubesd: make sure NOTIFY_SOCKET variable is not propagated qubesd do start other daemons - make sure they will not try to signal systemd about it. In some cases such daemons (qubesdb-daemon) behave differently based on this variable.	2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki	dc2563271c	tests: very basic tests for LinuxKernel pool	2017-05-17 11:17:22 +02:00
Marek Marczykowski-Górecki	0b64e3fbe0	storage: make LinuxKernel pool track vm kernel version Do not initialize it only at qubes.xml load time, but re-read vm.kernel property each time the path is constructed. While at it, add support for vm.kernel set to 'None' - simply don't include modules.img (xvdd) then.	2017-05-17 11:17:21 +02:00
Marek Marczykowski-Górecki	595d983659	storage: make verify() asyncio aware	2017-05-17 11:17:21 +02:00
Marek Marczykowski-Górecki	d57aef96e5	admin.vm.Start: report libvirt errors to caller Convert them to QubesException for that.	2017-05-17 11:17:21 +02:00
Marek Marczykowski-Górecki	774f4c8d49	tests: admin.vm.devices.* Admin API QubesOS/qubes-issues#2622	2017-05-16 09:05:22 +02:00
Marek Marczykowski-Górecki	661752bbc1	devices: list persistent devices also when not currently attached When device extension do not return some "persistent" device as currently attached, still return it, as it will be attached at next domain startup. User can distinguish such devices by having frontend_domain=None (or other VM). Also, return a set from DeviceCollection.assignments().	2017-05-16 09:05:22 +02:00
Marek Marczykowski-Górecki	2c0d79af4a	devices: pass attach options to device handling extension Otherwise options would be ignored...	2017-05-16 09:05:22 +02:00
Marek Marczykowski-Górecki	75ac611c90	admin: admin.vm.device.* methods QubesOS/qubes-issues#2622	2017-05-16 09:05:13 +02:00
Marek Marczykowski-Górecki	f3c7fb8129	admin: add missing parts to admin.vm.Clone Clone all the things.	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	8533ec9773	devices: add clone method Copy DeviceAssignment object	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	b4ad5f0149	firewall: add clone and comparing methods, missing docstrings	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	8de82f390e	tests: Tags() and Features()	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	680dbf432c	vm: Features fixes clear: dict should not be mutated during interation.	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	ba86d6da79	vm: implement tag manager to fire events on change While at it, adjust implementation to specification: tags don't have value, only one bit of information (present/not present). Fixes QubesOS/qubes-issues#2686	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	68a426f0ba	admin: add methods for global properties QubesOS/qubes-issues#2622	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	0160d6e2f0	admin: fix handling default template in admin.vm.Create.* methods	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	fcfb2e004a	tests: more for admin.Events method	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	3c7d2289ce	vm: stubdom_xid: use canonical method for checking VM state Don't depend on xid==-1 for not running VM. This breaks some tests, as xid can be a mock object (not comparable to int).	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	e598735c3b	tests: minor formatting	2017-05-16 09:05:00 +02:00
Marek Marczykowski-Górecki	d4353c2a30	Enable linux-stubdom by default Also, make it possible to set default on a template for its VMs. QubesOS/qubes-issues#2185	2017-05-16 09:01:40 +02:00
HW42	f7094bec4f	add support for linux stubdoms (cherry picked from commit e14ac1500a05162b3c07a5f44b4f0223f43fe90c)	2017-05-15 14:43:30 +02:00
Marek Marczykowski-Górecki	625c03c504	Rename MgmtAPI to AdminAPI - part 3: Admin API QubesOS/qubes-issues#853	2017-05-12 19:28:08 +02:00
Marek Marczykowski-Górecki	258d268a3a	Rename MgmtAPI to AdminAPI - part 2: internal API QubesOS/qubes-issues#853	2017-05-12 19:28:08 +02:00
Marek Marczykowski-Górecki	cd489f46e1	Rename MgmtAPI to AdminAPI - part 1: classes QubesOS/qubes-issues#853	2017-05-12 19:28:07 +02:00
Marek Marczykowski-Górecki	81162a699b	Remove unused helper scripts	2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki	83eef56f9d	Remove tools that are moved to -client repository QubesOS/qubes-issues#853	2017-05-12 18:43:35 +02:00
Marek Marczykowski-Górecki	bb4dc91ee8	storage/lvm: add workaround for ancient lvm2 in Travis-CI environment	2017-05-12 18:16:30 +02:00
Marek Marczykowski-Górecki	2a4af124f8	qubes/app: don't fail on saving qubes.xml when 'qubes' group isn't there	2017-05-12 18:11:27 +02:00
Marek Marczykowski-Górecki	da3eed582b	tests: rearrage LVM thin tests to run at least some in devel env This requires creating LVM volume group, so create on based on loop dev in /tmp. This is rather rough, but if any of this fails, run the tests anyway - it will simply skip LVM tests.	2017-05-12 18:10:17 +02:00
Marek Marczykowski-Górecki	b91b89a341	Make pylint ♥	2017-05-12 17:41:38 +02:00
Marek Marczykowski-Górecki	0d84ee1c24	make pylint happy	2017-05-12 17:04:20 +02:00
Marek Marczykowski-Górecki	7ef020d5fe	devices: fix 'NotImplemented' usage '__eq__' function should return 'NotImplemented' _value_, not 'NotImplementedError' exception.	2017-05-12 17:04:20 +02:00
Marek Marczykowski-Górecki	4f0d72f99a	devices: adjust API documentation Device ident may contain only characters allowed in qrexec argument. This will allow using it directly in qrexec argument in Attach/Detach methods. This also means PCI extension will need to be updated (it uses ':' in ident). QubesOS/qubes-issues#853	2017-05-12 17:04:20 +02:00
Marek Marczykowski-Górecki	ef031d6f0d	qubesvm: make vm.create_on_disk cleanup after failed attempt Don't leave half-created VM - it will make harder to retry.	2017-05-12 17:04:20 +02:00
Marek Marczykowski-Górecki	2a25db7eb7	qubesvm: fix stdio handling in vm.run_service_for_stdio Do set stdin/stdout/stderr descriptors as PIPE. Otherwise would not be redirected.	2017-05-12 17:04:19 +02:00
Marek Marczykowski-Górecki	838553baf8	qubesvm: move storage.stop() to 'domain-shutdown' event handler QubesOS/qubes-issues#2256	2017-05-12 17:04:19 +02:00
Marek Marczykowski-Górecki	0dfcaa63c9	Handle libvirt events in qubesd This is required to get shutdown notification, when it wasn't initiated by qubesd (for example 'poweroff' command inside of VM). Libvirt event loop implementation must be registered before making connection to libvirt, so move it to the beginning of main(). For now, only 'domain-shutdown' event is emited.	2017-05-12 17:04:19 +02:00
Marek Marczykowski-Górecki	3e067a3ef5	vm: run_service(): require user session running only when need gui access User session may not be started at all (for example no qubes packages installed there), so don't block it in all the cases. Also this would prevent running 'qubes.WaitForSession' service... In practice, default value for 'gui' argument is False, so in most cases user session will be ignored. Which doesn't matter in most cases - especially for services called by qubesd.	2017-05-12 17:04:19 +02:00
Marek Marczykowski-Górecki	68d5ca93e1	storage: have QubesVM() handle vm.dir_path, also on clone and rename Keep it uniform - QubesVM() object is responsible for handling vm.dir_path, Storage() is responsible for handling disk volumes (which may live in that directory QubesOS/qubes-issues#2256	2017-05-12 17:04:18 +02:00
Marek Marczykowski-Górecki	53be79b3b7	mgmt: vm.Create* and vm.Clone Create methods are dynamically created based on available VM classes. QubesOS/qubes-issues#2622	2017-05-12 17:04:18 +02:00
Marek Marczykowski-Górecki	52c3753d61	storage: support asynchronous storage pool implementations Allow specific pool implementation to provide asynchronous implementation. vm.storage.* methods will detect if given implementation is synchronous or asynchronous and will act accordingly. Then it's up to pool implementation how asynchronous should be achieved. Do not force it using threads (`run_in_executor()`). But pool implementation is free to use threads, if consider it safe in a particular case. This commit does not touch any pool implementation - all of them are still synchronous. QubesOS/qubes-issues#2256	2017-05-12 17:04:18 +02:00
Marek Marczykowski-Górecki	f324d30d4e	quebs: make Label() and storage.Pool() objects hashable	2017-05-12 14:27:47 +02:00
Marek Marczykowski-Górecki	ca51e8a02a	vm: remove unused metaclass	2017-05-12 14:25:32 +02:00
Marek Marczykowski-Górecki	8c3c65f40f	vm/net: move 'netvm' value validation to setter	2017-05-12 14:25:32 +02:00
Marek Marczykowski-Górecki	3bc8facc0b	mgmt: rework mgmt.Events to use per-instance handlers QubesOS/qubes-issues#2622	2017-05-12 14:25:32 +02:00
Marek Marczykowski-Górecki	da7496794a	events: add support for per-instance handlers	2017-05-12 14:25:32 +02:00
Marek Marczykowski-Górecki	bd1f84fcec	events: clarify docstrings about add_handler/remove_handler https://github.com/QubesOS/qubes-core-admin/pull/100#discussion_r115745098	2017-05-12 14:25:31 +02:00
Marek Marczykowski-Górecki	cddc507788	mgmt: factor out filtering by mgmt-permission:... events Split actual filtering done by mgmt-permission: events into calling an event and applying returned filters. This way filtering done in mgmt.Events handler could reuse the same function.	2017-05-12 14:25:31 +02:00
Marek Marczykowski-Górecki	6926725e02	Fixes for devices API See comments in #98	2017-05-12 14:25:31 +02:00
Marek Marczykowski-Górecki	68c3d7309a	tools: remove qubes_monitor_layout_notify tool - moved to mgmt-client	2017-05-12 14:25:30 +02:00
Marek Marczykowski-Górecki	82983cc1d8	vm: minor fix in docstring	2017-05-12 14:25:30 +02:00
Marek Marczykowski-Górecki	033d2132d3	ext/gui: remove most of it, moved to qvm-start-gui tool... ...in core-mgmt-client repository. qubesd isn't the right place to start GUI applications, which will be even more important when GUI domain will be something different than Dom0. QubesOS/qubes-issues#833	2017-05-12 14:25:29 +02:00
Marek Marczykowski-Górecki	027df9567c	mgmt: add mgmt.label.Index method QubesOS/qubes-issues#2622	2017-05-12 14:25:29 +02:00
Marek Marczykowski-Górecki	278243d7dc	tools/qubesd: fix sending exception response QubesOS/qubes-issues#2622	2017-05-12 14:25:29 +02:00
Marek Marczykowski-Górecki	55037e0446	tests: tools.qubesd For now only QubesDaemonProtocol test. main function still untested QubesOS/qubes-issues#2622	2017-05-12 14:25:29 +02:00
Marek Marczykowski-Górecki	0b737b3a55	mgmt: vm.features handling (mgmt.vm.feature.*) Actual implementation and tests. QubesOS/qubes-issues#2622	2017-05-12 14:25:28 +02:00
Marek Marczykowski-Górecki	361ecdd6ac	vm: add stubdom_xid property Some parts (like GUI) needs to communicate with stubdomain. Lets expose it's XID at the API level.	2017-05-12 14:25:28 +02:00
Marek Marczykowski-Górecki	acaada3fd9	vm: fix placeholder methods signatures in Features class Some methods inherited from dict (pop and setdefault here) are covered by placeholders raising NotImplementedError. Lets fix their signatures (to match those of dict) to really get NotImplementedError, instead of TypeError.	2017-05-12 14:25:28 +02:00
Marek Marczykowski-Górecki	643cef1e65	mgmt: send artificial 'connection-established' event just after connection This allows avoid race condition between registering event handlers and performing some action. The important thing is the event sent after registering event handlers in qubesd. This means state changes (like VM start/stop) after 'connection-established' event will be included in event stream. QubesOS/qubes-issues#2622	2017-05-12 14:25:28 +02:00
Marek Marczykowski-Górecki	8cc0af1eda	tests: mgmt.Events QubesOS/qubes-issues#2622	2017-05-12 14:25:27 +02:00
Marek Marczykowski-Górecki	73df1b8ea7	tests: VM lifecycle methods tests QubesOS/qubes-issues#2622	2017-05-12 14:25:27 +02:00
Marek Marczykowski-Górecki	99be78646f	Add stateless_property decorator for properties exposed through mgmt API Management API gives access only to qubes.property. And this is actually a good thing, so instead of extending it to access also builtins.property, add a simple decorator to define read-only, stateless qubes.property. QubesOS/qubes-issues#2622	2017-05-12 14:25:27 +02:00
Marek Marczykowski-Górecki	2aa0de3d5b	tools: remove qvm-ls tool and related integration in qubes.property qvm-ls tool (as all other tools) will be accessing properties through API, so no need (nor sense) for this tool-specific attributes in qubes.property. The only somehow used was ls_width, and in fact it made the output unnecessary wide. The tool itself is already moved to core-mgmt-client repository. QubesOS/qubes-issues#853	2017-05-12 14:25:27 +02:00
Marek Marczykowski-Górecki	654e64c4a6	mgmt: implement mgmt.Events Pass (almost) all the events. Some better filtering probably will be needed. QubesOS/qubes-issues#2622	2017-05-12 14:25:27 +02:00
Marek Marczykowski-Górecki	7e4bc4249d	tools/qubesd-query: handle events-based methods Pass the data as it arrives, instead of buffering it all. QubesOS/qubes-issues#2622	2017-05-12 14:25:26 +02:00
Marek Marczykowski-Górecki	40a86efd66	mgmt: add support for events-sending methods Standard methods return only one value, after operation is completed, but events-related methods may return multiple values during the method execution time. Provide a callback for such cases. Also, according to specification, avoid sending both event and non-event values. QubesOS/qubes-issues#2622	2017-05-12 14:25:26 +02:00
Marek Marczykowski-Górecki	bd9f1d2e7c	mgmt: add support for cancellable operations Allow method handler to decide if operation could be cancelled. If yes, when connection to the qubesd is terminated (and protocol.connection_lost get called) the operation is cancelled using standard asyncio method - in which case asyncio.CancelledError is thrown inside method handler. This needs to be explicitly enabled, because cancellable methods are much harder to write, to maintain consistent system state. Caveat: protocol.connection_lost is called only when trying to send some data to it (and it's already terminated). Which makes this whole mechanism useful only for events. Otherwise, when sending some data (and possibly detecting that connection is broken), the operation is already completed. QubesOS/qubes-issues#2622	2017-05-12 14:25:26 +02:00
Marek Marczykowski-Górecki	ee967c1e72	events: add function to remove event handlers Event handlers for remote mgmt API will be registered dynamically, so allow to unregister them. QubesOS/qubes-issues#2622	2017-05-12 14:25:26 +02:00
Marek Marczykowski-Górecki	68c8d219c9	vm/qubesvm: fix daemon startup function 'command' argument is a tuple	2017-05-12 14:25:25 +02:00
Marek Marczykowski-Górecki	6383d87be9	qmemmand: fix systemd notification	2017-05-12 14:25:25 +02:00
Marek Marczykowski-Górecki	d75fc6c28b	backup: add a little more usage info, include docs in main index	2017-04-21 19:58:32 +02:00
Marek Marczykowski-Górecki	50b812190b	Merge remote-tracking branch 'kalkin/device-assignments' into core3-devel * kalkin/device-assignments: (21 commits) PCI extension cache PCIDevice objects Make pylint ♥ Fix pylint warning no-else-return Fix pylint warning len-as-conditional device-list-attached event returns a dev/options tupples list DeviceAssignment options are now a dict Remove WrongAssignment exception Rename qubes.devices.BlockDevice to qubes.storage.BlockDevice Update relaxng devices option element Fix tests broken by the new assignment api Fix qubes.tests.devices Fix pci device integration tests qvm-device add support for assignments Update ext/pci to new api BaseVM add DeviceAssignment xml serialization Migrate DeviceCollection to new API Add PersistentCollection helper to qubes.devices Add DeviceAssignment qvm-device validates device parameters qvm-device fix handling of non block devices ...	2017-04-21 17:09:33 +02:00
Marek Marczykowski-Górecki	2705e223bd	Merge branch 'core3-policy' into core3-devel * core3-policy: Make pylint happy tests: disable GTK tests on travis qubespolicy: make pylint happy qubespolicy: run GUI code inside user session and expose it as dbus object tests: plug rpc-window tests into main test runner qubespolicy: plug GUI code into qrexec-policy tool rpm: add rpc-window related files to package rpc-window: adjust for qubespolicy API rpc-window: use pkg_resources for glade file rpc-window: use 'edit-find' icon if no other is found rpc-window: adjust for python3 rpc-window: code style adjustments Import new rpc confirmation window code qubesd: add second socket for in-dom0 internal calls policy: qrexec-policy cli tool tests: qubespolicy tests qubespolicy: initial version for core3 vm/appvm: add dispvm_allowed property dispvm: don't load separate Qubes() instance when handling DispVM	2017-04-21 17:01:56 +02:00
Marek Marczykowski-Górecki	fa72679b47	Make pylint happy New pylint throw some more warnings.	2017-04-21 16:57:35 +02:00
Bahtiar `kalkin-` Gadimov	8d60f533c3	PCI extension cache PCIDevice objects	2017-04-15 23:50:11 +02:00
Bahtiar `kalkin-` Gadimov	79407a8717	Make pylint ♥	2017-04-15 23:50:11 +02:00
Bahtiar `kalkin-` Gadimov	5bc7a8f9e3	Fix pylint warning no-else-return	2017-04-15 23:49:41 +02:00
Bahtiar `kalkin-` Gadimov	eb217e0d27	Fix pylint warning len-as-conditional	2017-04-15 23:49:41 +02:00
Bahtiar `kalkin-` Gadimov	9da28c9c15	device-list-attached event returns a dev/options tupples list	2017-04-15 23:49:40 +02:00
Bahtiar `kalkin-` Gadimov	1c9636c5af	DeviceAssignment options are now a dict	2017-04-15 23:49:40 +02:00
Bahtiar `kalkin-` Gadimov	5a8cc9bdd3	Remove WrongAssignment exception Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>	2017-04-15 23:49:39 +02:00
Bahtiar `kalkin-` Gadimov	e446e7a2f4	Rename qubes.devices.BlockDevice to qubes.storage.BlockDevice Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>	2017-04-15 23:49:36 +02:00
Bahtiar `kalkin-` Gadimov	b1b005964f	Fix tests broken by the new assignment api	2017-04-15 23:48:59 +02:00
Bahtiar `kalkin-` Gadimov	827ca283f3	Fix qubes.tests.devices Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>	2017-04-15 23:48:59 +02:00
Bahtiar `kalkin-` Gadimov	e84114d3a9	Fix pci device integration tests Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>	2017-04-15 23:48:59 +02:00
Bahtiar `kalkin-` Gadimov	0f78d257d4	qvm-device add support for assignments - Add header field ASSIGNED - Persistent assignments are displayed with an asterisk - Add '-p' to attach command Fix qvm-device Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>	2017-04-15 18:05:30 +02:00
Bahtiar `kalkin-` Gadimov	0b3aebac9f	Update ext/pci to new api Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>	2017-04-15 18:05:29 +02:00
Bahtiar `kalkin-` Gadimov	2a6266887e	BaseVM add DeviceAssignment xml serialization Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>	2017-04-15 18:05:28 +02:00
Bahtiar `kalkin-` Gadimov	990cfd8ab9	Migrate DeviceCollection to new API - Use PersistentCollection as _set() - attach/detach expect DeviceAssignment as parater - attached(persistent=True) is now persistent() - attached() returns all attached devices - assigned() returns all attached device assignments `# modified: templates/libvirt/xen.xml Signed-off-by: Bahtiar `kalkin-` Gadimov <bahtiar@gadimov.de>	2017-04-15 18:05:28 +02:00
Bahtiar `kalkin-` Gadimov	23c68c5458	Add PersistentCollection helper to qubes.devices	2017-04-15 18:05:27 +02:00
Bahtiar `kalkin-` Gadimov	211e018268	Add DeviceAssignment	2017-04-15 18:05:26 +02:00
Bahtiar `kalkin-` Gadimov	e52898972d	qvm-device validates device parameters	2017-04-15 18:05:25 +02:00
Bahtiar `kalkin-` Gadimov	6caaa6f66d	qvm-device fix handling of non block devices	2017-04-15 18:05:24 +02:00
Bahtiar `kalkin-` Gadimov	9d08e4b792	Fix bug in PCIDeviceExtension: decode buffer to string	2017-04-15 18:05:12 +02:00
Marek Marczykowski-Górecki	96db9a46d1	Fix start-related asyncio integration cherry-picked commit 05fc3a4	2017-04-07 17:46:54 +02:00
Marek Marczykowski-Górecki	0ada6d0b64	tests: disable GTK tests on travis The environment there is way too old. QubesOS/qubes-issues#910	2017-04-07 17:07:31 +02:00
Marek Marczykowski-Górecki	e76ede3ad0	tests: plug rpc-window tests into main test runner QubesOS/qubes-issues#910	2017-04-07 17:07:30 +02:00
Marek Marczykowski-Górecki	83526a28d3	qubesd: add second socket for in-dom0 internal calls This socket (and commands) are not exposed to untrusted input, so no need to extensive sanitization. Also, there is no need to provide a stable API here, as those methods are used internally only. QubesOS/qubes-issues#853	2017-04-07 17:07:28 +02:00
Marek Marczykowski-Górecki	564408eb3f	tests: qubespolicy tests Fixes QubesOS/qubes-issues#2460	2017-04-06 15:43:17 +02:00
Marek Marczykowski-Górecki	1c836531c7	vm/appvm: add dispvm_allowed property Speciffy whether DispVM can be created from this AppVM Fixes QubesOS/qubes-issues#2075	2017-04-06 15:43:01 +02:00
Marek Marczykowski-Górecki	e4e92b06af	dispvm: don't load separate Qubes() instance when handling DispVM This will not work in qubesd, where there is one and only Qubes() instance.	2017-04-06 15:43:01 +02:00
Wojtek Porczyk	3bc4ec1e98	add mock libvirtaio module + pylint fixes	2017-04-05 17:06:45 +02:00
Wojtek Porczyk	64d358562b	qubes: revert async def, use @asyncio.coroutine Current pylint (any released version) stumbles on async def'ined functions. Let's use @asyncio.coroutines for now. Seems like python-3.5 is not that mature yet. QubesOS/qubes-issues#2622 QubesOS/qubes-issues#2738 PyCQA/pylint#1126	2017-04-05 17:05:52 +02:00
Wojtek Porczyk	78693c265c	qubes/vm/qubesvm: rework QubesVM.run() methods 0) All those methods are now awaitable rather than synchronous. 1) The base method is run_service(). The method run() was rewritten using run_service('qubes.VMShell', input=...). There is no provision for running plain commands. 2) Get rid of passio= arguments. If you'd like to get another return value, use another method. It's as simple as that. See: - run_service_for_stdio() - run_for_stdio() Also gone are wait= and localcmd= arguments. They are of no use inside qubesd. 3) The qvm-run tool and tests are left behind for now and will be fixed later. This is because they also need event loop, which is not implemented yet. fixes QubesOS/qubes-issues#1900 QubesOS/qubes-issues#2622	2017-04-05 17:05:22 +02:00
Wojtek Porczyk	a520e8d768	qubes/mgmt: add missing events for permission for lifecycle Thanks @marmarek for spotting this. QubesOS/qubes-issues#2622	2017-04-05 16:36:54 +02:00
Wojtek Porczyk	c437f40284	qubes/mgmt: lifecycle and class listing - mgmt.vmclass.List - mgmt.vm.Start - mgmt.vm.Shutdown - mgmt.vm.Pause - mgmt.vm.Unpause - mgmt.vm.Kill QubesOS/qubes-issues#2622	2017-04-03 21:55:01 +02:00
Wojtek Porczyk	c4ef02c377	qubes/mgmt: explicit method decorator and misc improvements - Get rid of @not_in_api, exchange for explicit @api() decorator. - Old @no_payload decorator becomes an argument (keyword-only). - Factor out AbstractQubesMgmt class to be a base class for other mgmt backends. - Use async def instead of @asyncio.coroutine. QubesOS/qubes-issues#2622	2017-04-03 21:55:01 +02:00
Wojtek Porczyk	1b9479837a	Merge remote-tracking branch 'marmarek/core3-mgmt-methods1' into core3-devel Conflicts: qubes/mgmt.py	2017-04-01 18:33:54 +02:00
Marek Marczykowski-Górecki	3388054eae	mgmt: move property value sanitization to property definition This also means we don't check if a VM with given name (in case of VMProperty) exists in the system, at this stage. But this is ok, lets not duplicate work of property setter. QubesOS/qubes-issues#2622	2017-03-31 11:55:24 +02:00
Wojtek Porczyk	cce809c2cb	qubes/vm/qubesvm: async def start QubesOS/qubes-issues#2622	2017-03-30 23:04:55 +02:00
Marek Marczykowski-Górecki	ce3bedbf2c	vm: move validate_name to qubes/vm This will be needed by VMProperty class in the next commit. QubesOS/qubes-issues#2622	2017-03-28 21:04:05 +02:00
Marek Marczykowski-Górecki	ad456a3387	mgmt: minor adjustments per @woju review QubesOS/qubes-issues#2622	2017-03-28 20:59:55 +02:00
Wojtek Porczyk	8e3621c4e5	Move libviraio to core-libvirt, may be upstreamed QubesOS/qubes-issues#2622	2017-03-22 15:30:55 +01:00
Marek Marczykowski-Górecki	2c115bb648	mgmt: allow functions to return None if no data is to be returned Don't force `return ''` everywhere. QubesOS/qubes-issues#2622	2017-03-17 12:49:28 +01:00
Marek Marczykowski-Górecki	f93674de1a	mgmt: drop ProtocolRepr Since we've added type= argument to property.Get format, it isn't useful anymore. QubesOS/qubes-issues#2622	2017-03-16 21:10:06 +01:00
Marek Marczykowski-Górecki	d21f54887d	mgmt: add decorator for method calls not accepting payload Fixes QubesOS/qubes-issues#2687	2017-03-16 20:21:11 +01:00
Marek Marczykowski-Górecki	8371ffcd0f	tests: storage and labels QubesOS/qubes-issues#2622	2017-03-16 20:04:07 +01:00
Marek Marczykowski-Górecki	2a223811a5	tests: add missing import	2017-03-16 20:04:06 +01:00
Marek Marczykowski-Górecki	f1c8f3220d	tests: deal with events containing dict in kwargs If kwargs contains dict as one of values, it isn't hashable and can't be used as value in frozenset/tuple. Convert such values into frozenset(dict.items()). Only one (more) level is supported, but it should be enough. Solution from http://stackoverflow.com/a/13264725	2017-03-16 20:04:06 +01:00
Marek Marczykowski-Górecki	beaa0e9348	tests/mgmt: check if argument/payload is rejected when should be Instead of creating such tests for each method separately, use unittest subTest functionality to handle all of them at once.	2017-03-16 20:04:06 +01:00
Marek Marczykowski-Górecki	944bb26369	tests/mgmt: VM property related functions	2017-03-16 20:04:05 +01:00
Marek Marczykowski-Górecki	35d1167893	qubes/vm/net: fix name of argument for property-del event handler Since enforcing keyword arguments for event handlers, it's important now.	2017-03-16 20:04:05 +01:00
Marek Marczykowski-Górecki	32f6bc2cd9	qubes/app: fix notifying about default_netvm change Notify every VM that is affected, not only those providing network itself.	2017-03-16 20:04:05 +01:00
Marek Marczykowski-Górecki	2c4303efc4	Prefer qubes.exc.QubesValueError over ValueError This provide clearer information for UI.	2017-03-16 20:04:05 +01:00
Marek Marczykowski-Górecki	010d40dc1e	mgmt: add label-related calls QubesOS/qubes-issues#2622	2017-03-16 20:04:04 +01:00
Marek Marczykowski-Górecki	33f3fedca1	mgmt: save qubes.xml after config-modifying calls In theory any call could modify config (through events), but lets keep writes to qubes.xml low. In any case, qubes.xml will be eventually written (either at next config-modifying call, or daemon exit).	2017-03-16 20:04:04 +01:00
Marek Marczykowski-Górecki	868dbeac3e	mgmt: implement mgmt.vm.property.Set Sanitization of input value is tricky here, and also very important at the same time. If property define value type (and it's something more specific than 'str'), use that. Otherwise allow only printable ASCII characters, and let appropriate event and setter handle value. At this point I've reviewed all QubesVM properties in this category and added appropriate setters where needed. QubesOS/qubes-issues#2622	2017-03-16 20:04:03 +01:00
Marek Marczykowski-Górecki	da51e6f032	vm/qubesvm: add validator for default_user property Don't allow characters potentially interfering with qrexec. To be on the safe side, allow only alphanumeric characters + very few selected punctuations.	2017-03-16 20:04:03 +01:00
Marek Marczykowski-Górecki	0f8fab088e	vm/qubesvm: remove pool_name property It isn't used anywhere - in core3 each storage volume have pool assigned - which may be different for each volume.	2017-03-16 20:04:03 +01:00
Marek Marczykowski-Górecki	2d2672ec58	vm/qubesvm: convert firewall_conf into dumb, read-only property Don't allow anything else than firewall.xml.	2017-03-16 20:04:03 +01:00
Marek Marczykowski-Górecki	123feced36	vm/qubesvm: forbid '/' in kernel property It would give VM access to some files outside of /var/lib/qubes/vm-kernels.	2017-03-16 20:04:02 +01:00
Marek Marczykowski-Górecki	a036e2a8a0	vm/qubesvm: improve name property setter Split it into two functions: validate_name - context-less verification, and actual _setter_name which perform additional verification in context of actual VM. Switch to qubes.exc.* exceptions where appropriate.	2017-03-16 20:04:02 +01:00
Marek Marczykowski-Górecki	dbf2066dfd	mgmt: encode property type in property.Get This also require having property.type public. QubesOS/qubes-issues#2622	2017-03-16 20:04:02 +01:00
Marek Marczykowski-Górecki	3b36e92b6d	vm/qubesvm: fix few more keyword arguments for events	2017-03-16 20:04:02 +01:00
Marek Marczykowski-Górecki	772293d0b5	vm/qubesvm: define 'updateable' as qubes.property This will allow property being accessed through management API	2017-03-16 20:04:01 +01:00
Marek Marczykowski-Górecki	f7eabf8eb0	tools/qubesd: do not close connection before sending response eof_received callback should return True, if connection should not be automatically closed just after returning from it.	2017-03-16 20:04:01 +01:00
Marek Marczykowski-Górecki	c41585e2f5	Initialize dom0 label It is required property. Additionally, define icon_path to None, otherwise it tries to access dom0.dir_path, which isn't set.	2017-03-16 20:04:01 +01:00
Marek Marczykowski-Górecki	10a07c8726	mgmt: allow vm.List call to a particular VM This allow getting info about a single VM. QubesOS/qubes-issues#853	2017-03-16 20:04:01 +01:00
Marek Marczykowski-Górecki	fb7bd6823a	mgmt: implement storage-related methods QubesOS/qubes-issues#2622	2017-03-16 20:04:01 +01:00
Wojtek Porczyk	80807fb872	qubes/libvirtaio: document and prepare for upstream QubesOS/qubes-issues#2622	2017-03-16 15:21:16 +01:00
Wojtek Porczyk	a5c59a5075	qubes/mgmt: Drop custom repr QubesOS/qubes-issues#2622	2017-03-13 22:00:15 +01:00
Wojtek Porczyk	93153da893	Add documentation for mgmt QubesOS/qubes-issues#2622	2017-03-13 21:51:52 +01:00
Wojtek Porczyk	417cb6e912	qubes/vm/mix/net: fix event arguments Arguments to events are now keyword-only and require exact names.	2017-03-06 17:20:57 +01:00
Wojtek Porczyk	4a247b1b1b	Merge remote-tracking branch 'origin/pull/90/head' into core3-devel	2017-03-02 13:19:57 +01:00
Marek Marczykowski-Górecki	f7d73893d7	qubes/storage: py3k related fixes	2017-03-01 21:50:06 +01:00
Wojtek Porczyk	c2a0d34ade	pylint: do not interpret asyncio.ensure_future compat hack	2017-03-01 18:30:49 +01:00
Wojtek Porczyk	865ab10a0c	qubesd+mgmt: convert mgmt functions to coroutines QubesOS/qubes-issues#2622	2017-03-01 17:31:37 +01:00
Marek Marczykowski-Górecki	3e0f225938	qubes: allow 'property' object comparing with str This will allow checking if a given name is valid property name, using simple `name in vm.property_list()`. QubesOS/qubes-issues#853	2017-02-27 21:57:56 +01:00
Marek Marczykowski-Górecki	2169075807	qubesd: fix response message header Type is not 16 bit big-endian. Encode it as 8bit code and \x00 as delimiter explicitly. QubesOS/qubes-issues#853	2017-02-27 21:43:14 +01:00
Marek Marczykowski-Górecki	6ab7032b11	qubes/mgmt: encode VM name without quotes That's how it is in the specification. QubesOS/qubes-issues#853	2017-02-27 21:42:30 +01:00
Marek Marczykowski-Górecki	f4616fc366	qubesd: make qubesd socket qubes-group owned QubesOS/qubes-issues#853	2017-02-27 21:42:06 +01:00
Marek Marczykowski-Górecki	c454973596	qubes/mgmt: use keyword arguments in events QubesOS/qubes-issues#853	2017-02-27 20:56:16 +01:00
Marek Marczykowski-Górecki	751415434c	backup: make hmac verification more defensive Check HMAC file size, read it as binary or with 'ascii' encoding only.	2017-02-27 02:37:52 +01:00
Marek Marczykowski-Górecki	a6c7da6061	tests: be even more defensive on cleaning up VMs Don't fail even if qubes-test.xml do not load at all because of syntax error - for example empty file.	2017-02-27 02:37:52 +01:00
Marek Marczykowski-Górecki	45709b510a	backup: minor fixes after bringing back scrypt support	2017-02-27 02:37:51 +01:00
Marek Marczykowski-Górecki	1363251438	Revert "Revert "backup: use 'scrypt' tool for backup encryption and integrity protection"" This reverts commit `0f1672dc63`. Bring it back. Lets not revert the whole feature just because required package exists only in qubes-builder, not in some online repository. Also, this revert didn't go as planned - there was a reference to a 'passphrase' local variable, but it wasn't assigned any value. Cc: @woju	2017-02-27 02:37:50 +01:00
Marek Marczykowski-Górecki	13fc810363	tests: some more fixes for core3 API	2017-02-27 02:37:50 +01:00
Marek Marczykowski-Górecki	3ecc0a9bcb	tests: improve devices API unit test Check fired events - inspired by qvm-device test.	2017-02-27 02:37:50 +01:00
Marek Marczykowski-Górecki	7f2ca33774	tests: fix importing template in non-default pool	2017-02-27 02:37:50 +01:00
Marek Marczykowski-Górecki	3726c7d9c3	python: decode xrandr output earlier, don't use regexp on bytes	2017-02-27 02:37:49 +01:00
Marek Marczykowski-Górecki	5e43d26abd	qubes: unify property ordering We already have property ordering defined in property_list(), lets move it to proper place: property.__lt__.	2017-02-27 02:37:49 +01:00
Marek Marczykowski-Górecki	9ace4e66f1	tests: more py3k related fixes	2017-02-27 02:37:49 +01:00
Marek Marczykowski-Górecki	33416f2549	qmemman: update for py3k This just make the code compatible with py3k, but nothing more. Converting to asyncio is probably the next step.	2017-02-27 02:37:49 +01:00
Marek Marczykowski-Górecki	2c3e112951	backup: one more py3k related fix	2017-02-27 02:37:48 +01:00
Marek Marczykowski-Górecki	e52d8fb051	qubes: allow passing name of class to app.add_new_vm This will allow more flexible API usage, especially when using mgmt API - we need to use VM type as string there. We don't lose any flexibility here - VM class names needs to be uniquely identified by a string (used in qubes.xml) anyway.	2017-02-27 02:37:48 +01:00
Marek Marczykowski-Górecki	cae68f64ca	tests: just one more test in vm_qrexec_gui to core3 API	2017-02-27 02:37:48 +01:00
Marek Marczykowski-Górecki	3f29345d32	tests/storage: read-only volume should not have save_on_stop=True	2017-02-27 02:37:48 +01:00
Marek Marczykowski-Górecki	5ee05e06e5	qubes/core2migration: update locking API Sync with `0141e1a` "qubes/app: Allow keeping lock after load"	2017-02-27 02:37:47 +01:00
Marek Marczykowski-Górecki	570cbe5225	qubes: py3k related fixes	2017-02-27 02:37:45 +01:00
Marek Marczykowski-Górecki	9c5c70fe25	qubes/backup: py3k related fixes - str/bytes usage - functools.reduce - dict.items instead of dict.iteritems etc	2017-02-27 02:35:50 +01:00
Marek Marczykowski-Górecki	f2a1687879	typos in comments	2017-02-27 02:35:49 +01:00
Marek Marczykowski-Górecki	9cad353939	tests: py3k related fixes - bytes/str Adjust usage of bytes vs str type.	2017-02-27 02:35:49 +01:00
Marek Marczykowski-Górecki	d68499f17f	qubes: add property ordering	2017-02-27 02:35:49 +01:00
Wojtek Porczyk	be53db4db9	qubes/events: they accept only keyword arguments Positional arguments are hereby deprecated, with immediate effect. QubesOS/qubes-issues#2622	2017-02-21 14:46:42 +01:00
Wojtek Porczyk	48f10a79c9	qubes/tools/qubesd: add response header QubesOS/qubes-issues#2622	2017-02-21 14:46:42 +01:00
Wojtek Porczyk	25d81b8ab6	Merge remote-tracking branch 'origin/pull/88/head' into core3-devel	2017-02-15 12:17:41 +01:00
Marek Marczykowski-Górecki	e50b17a6b3	tools/qvm-features: make pylint happy reduce number of return statements.	2017-02-15 00:01:33 +01:00
Marek Marczykowski-Górecki	a317e81d7e	qubes/ext/gui: adjust shm.id path It's moved to /var/run/qubes and now is built based on $DISPLAY.	2017-02-15 00:01:33 +01:00
Marek Marczykowski-Górecki	bd9300b38e	tests: copy pool configuration into qubes-test.xml If template choosen for the tests is installed in non-default storage pool, this pool also needs to be imported into qubes-test.xml.	2017-02-15 00:01:33 +01:00
Marek Marczykowski-Górecki	0c43329188	tools/qvm-features: fix handling empty list of features	2017-02-15 00:01:33 +01:00
Marek Marczykowski-Górecki	98edc9779c	tools/qvm-features: fix domain argument handling It's args.domains[0], not args.vm.	2017-02-15 00:01:33 +01:00
Marek Marczykowski-Górecki	c3fc4062d8	tests: add basic test for qvm-features	2017-02-15 00:01:33 +01:00
Marek Marczykowski-Górecki	bcab92ee64	qubes/vm: make sure to close qmemman socket after failed startup If qmemman socket isn't closed, it will block other VM startups.	2017-02-14 23:59:07 +01:00
Marek Marczykowski-Górecki	01aedb7f18	storage: fix handling snap_on_start=True file volumes Use the right cow image and apply the second layer to provide read-write access. The correct setup is: - base image + base cow -> read-only snapshot (base changes "cached" until committed) - read-only snapshot + VM cow -> read-write snapshot (changes discarded after VM shutdown) This way, even VM without Qubes-specific startup scripts will can benefit from Template VMs, while VMs with Qubes-specific startup scripts may still see original root.img content (for possible signature verification, when storage domain got implemented). QubesOS/qubes-issues#2256	2017-02-14 23:59:07 +01:00
Marek Marczykowski-Górecki	48f78dfbc8	tests: check if snap_on_start=True volumes are not persistent Content should be reset back to base volume at each VM startup. Disposable VMs depend on this behaviour. QubesOS/qubes-issues#2256	2017-02-14 23:59:07 +01:00
Wojtek Porczyk	8e678c0172	qubes/mgmt: mgmt.vm.property.Reset QubesOS/qubes-issues#2622	2017-02-14 11:37:17 +01:00
Wojtek Porczyk	ada0437f52	qubes/mgmt: mgmt.vm.property.Help QubesOS/qubes-issues#2622	2017-02-13 21:28:27 +01:00
Wojtek Porczyk	e8a5bc9b36	qubesd: improve exception handling QubesOS/qubes-issues#2622	2017-02-10 23:25:45 +01:00
Wojtek Porczyk	249d8c08e2	qubes/tools/qubesd-query: low-level interrogation tool	2017-02-10 23:25:45 +01:00
Wojtek Porczyk	02639b8d02	qubes/mgmt: mgmt.vm.property.List QubesOS/qubes-issues#2622	2017-02-10 22:55:53 +01:00
Wojtek Porczyk	c12fc744a2	qubes/mgmt: move mgmt api to separate module QubesOS/qubes-issues#2622	2017-02-09 23:29:05 +01:00
Wojtek Porczyk	5d455ac3c4	misc pylint fixes related to qubesd	2017-02-08 15:37:39 +01:00
Wojtek Porczyk	0be3b1fbb1	qubes/tools/qubesd: initial version	2017-02-07 17:07:53 +01:00
Wojtek Porczyk	1be75d9c83	misc python3 fixes	2017-02-07 17:07:52 +01:00
Wojtek Porczyk	d74567d65f	qubes: port core to python3 fixes QubesOS/qubes-issues#2074	2017-01-20 16:42:51 +01:00
Wojtek Porczyk	0f1672dc63	Revert "backup: use 'scrypt' tool for backup encryption and integrity protection" This reverts commit `418d749680`. Package `scrypt` is currently not installable (not present in any repo). Cc: @marmarek	2016-12-05 18:36:13 +01:00
Wojtek Porczyk	25912f5787	qubes/tools: add qvm-tags QubesOS/qubes-issues#865	2016-12-02 14:17:09 +01:00
Wojtek Porczyk	5f436360f7	qubes/app: Fix create_empty_store signature QubesOS/qubes-issues#1729	2016-11-30 18:34:11 +01:00
Wojtek Porczyk	68ad60c1b3	Merge remote-tracking branch 'origin/master' into core3-devel Conflicts: core/qubes.py doc/Makefile doc/manpages/qvm-prefs.rst doc/qvm-tools/qvm-add-appvm.rst doc/qvm-tools/qvm-backup-restore.rst doc/qvm-tools/qvm-backup.rst doc/qvm-tools/qvm-block.rst doc/qvm-tools/qvm-clone.rst doc/qvm-tools/qvm-firewall.rst doc/qvm-tools/qvm-ls.rst doc/qvm-tools/qvm-pci.rst doc/qvm-tools/qvm-run.rst doc/qvm-tools/qvm-shutdown.rst doc/qvm-tools/qvm-start.rst doc/qvm-tools/qvm-template-commit.rst qvm-tools/qvm-ls qvm-tools/qvm-prefs qvm-tools/qvm-remove tests/__init__.py vm-config/xen-vm-template.xml	2016-11-30 03:07:39 +01:00
Marek Marczykowski-Górecki	8f443547fb	qubes/vm: disconnect from old QubesDB when going to start new instance QubesDB daemon no longer remove socket created by new instance, so one part of VM restart race condition is solved. The only remaining part is to ensure that we really connect to the new instance, instead of talking to the old one (soon to be terminated). Fixes QubesOS/qubes-issues#1694	2016-11-26 04:09:00 +01:00
Marek Marczykowski-Górecki	3b209515c2	qubes/vm/dispvm: don't crash when DispVM is already killed This is regression of QubesOS/qubes-issues#1660 Fixes QubesOS/qubes-issues#1660	2016-11-26 04:09:00 +01:00
Marek Marczykowski-Górecki	6ff836dfa4	qubes/log: add FD_CLOEXEC to log files Don't leak log file descriptors. At least 'lvm' complains. QubesOS/qubes-issues#2412	2016-11-26 04:08:59 +01:00
Marek Marczykowski-Górecki	210cb65d1c	qubes/tools: drop requirement of qubes-prefs --force-root None of properties set there do any "dangerous thing" for filesystem permissions (at least for now), so do not require it. This is mostly to keep compatibility with %post rpm scripts (kernel-qubes-vm at least). QubesOS/qubes-issues#2412	2016-11-26 04:08:59 +01:00
Marek Marczykowski-Górecki	a318d5cea9	Don't fail on DBus connection error or opening log Especially in offline mode - like during installation, tests etc. QubesOS/qubes-issues#2412	2016-11-26 04:08:59 +01:00
Marek Marczykowski-Górecki	02a0713665	qubes/tools: better handle qvm-template-postprocess called as root This tool by design is called as root, so try to: - switch to normal user if possible - fix file permissions afterwards - if not QubesOS/qubes-issues#2412	2016-11-26 04:08:58 +01:00
Marek Marczykowski-Górecki	28d4feb0d0	qubes: fix network-related functions - cleanup_vifs crash when non-networked VM is running - type error in get_vms_connected_to (store VM objects, not qid)	2016-11-26 04:08:58 +01:00
Marek Marczykowski-Górecki	e85b0663f6	tools: fire 'template-postinstall' event for extensions Allow extensions to finish template setup. This include retrieving appmenus, settings defaults etc. QubesOS/qubes-issues#2412	2016-11-26 04:08:58 +01:00
Marek Marczykowski-Górecki	0041063b8e	qubes/config: set default labels There was a comment '# Set later', but actually values were never set. This break adding just installed template (qvm-template-postprocess). QubesOS/qubes-issues#2412	2016-11-26 04:08:57 +01:00
Marek Marczykowski-Górecki	c4e85a81fb	qubes/app: automatically enable offline mode when running in chroot Do not spray --offline-mode over every installer-related script. QubesOS/qubes-issues#2412	2016-11-26 04:08:50 +01:00
Marek Marczykowski-Górecki	d8a1216daf	Fix qubes-create in offline mode QubesOS/qubes-issues#2412	2016-11-26 04:08:09 +01:00
Marek Marczykowski-Górecki	5e15db4176	qubes/tools: accept qvm-start --no-guid Lets keep compatibility with older scripts. QubesOS/qubes-issues#2412	2016-11-26 04:08:08 +01:00
Marek Marczykowski-Górecki	964955758c	qubes/app: create 'default' storage pool as LVM when present When system is installed with LVM thin pool, it should be used by default. But lets keep file-based on for /var/lib/qubes for some corner cases, migration etc. QubesOS/qubes-issues#2412	2016-11-26 04:08:08 +01:00
Marek Marczykowski-Górecki	badc58837a	Add qvm-template-postprocess tool This is intended to call to finish template installation/removal. Template RPM package is basically container for root.img, nothing more. Other parts needs to be generated after root.img extraction. Previously it was open coded in rpm post-install script, but lets keep it as qvm tool to ease supporting multiple version in template builder QubesOS/qubes-issues#2412	2016-11-26 04:08:08 +01:00
Marek Marczykowski-Górecki	1418555346	qubes/vm: don't fail on removing VM without files VM files may be already removed. Don't fail on this while removing a VM, it's probably the reason why domain is being removed. qvm-remove tool have its own guard for this, but it isn't enough - if rmtree(dir_path) fails, storage.remove() would not be called, so non-file storages would not be cleaned up. This is also needed to correctly handle template reinstallation - where VM directory is moved away to call create_on_disk again. QubesOS/qubes-issues#2412	2016-11-26 04:08:07 +01:00
Marek Marczykowski-Górecki	cc440c62f6	qubes/tools: accept properties with '-' '-' is invalid character in python identifier, so all the properties have '_'. But in previous versions qvm-* tools accepted names with '-', so lets not break this. QubesOS/qubes-issues#2412	2016-11-26 04:08:07 +01:00
Marek Marczykowski-Górecki	880566a387	qubes/tools: do not reject --set/--get in *-prefs tools Those options are no longer needed, but lets not reject them to preserve compatibility with older scripts QubesOS/qubes-issues#2412	2016-11-26 04:08:07 +01:00
Marek Marczykowski-Górecki	91727389c4	qubes/log: ensure logs are group writable /var/log/qubes directory have setgid set, so all the files will be owned by qubes group (that's ok), but there is no enforcement of creating it group writable, which undermine group ownership (logs created by root would not be writable by normal user) QubesOS/qubes-issues#2412	2016-11-26 04:08:07 +01:00
Marek Marczykowski-Górecki	80c0093c77	qubes/tools/qubes-create: reject overriding existing qubes.xml If someone really want do to this, need to manually remove the file first. QubesOS/qubes-issues#2412	2016-11-26 04:08:06 +01:00
Marek Marczykowski-Górecki	c08766e157	qubes/features: rename 'services/ntpd' to 'service/ntpd' It makes much more sense to use singular form here - ntpd is a single service.	2016-11-26 04:08:06 +01:00
Wojtek Porczyk	2b0ad51b18	Merge remote-tracking branch 'origin/pull/68/head' into core3-devel	2016-11-15 17:41:47 +01:00
Wojtek Porczyk	37bfd0d2a3	Merge remote-tracking branch 'marmarek/core3-fake-ip' into core3-devel	2016-11-15 17:40:30 +01:00
Wojtek Porczyk	a4d50409df	Merge remote-tracking branch 'marmarek/core3-storage-fixes' into core3-devel	2016-11-15 17:36:53 +01:00
Wojtek Porczyk	fd953f4f27	Merge remote-tracking branch 'marmarek/core3-backup2' into core3-devel	2016-11-15 17:34:12 +01:00
Bahtiar `kalkin-` Gadimov	5db67fca8d	Fix init property swap in DomainVolumes	2016-11-07 23:26:53 +01:00
Marek Marczykowski-Górecki	b011cef8af	tests/storage: add tests for basic volumes properties Things like if read-only volume is really read-only, volatile is volatile etc. QubesOS/qubes-issues#2256	2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki	b59463e8e8	qvm-block: fix listing non-internal volumes In case of LVM (at least), "internal" flag is initialized only when listing volume attached to given VM, but not when listing them from the pool. This looks like a limitation (bug?) of pool driver, it looks like much nicer fix is to handle the flag in qvm-block tool (which list VMs volumes anyway), than in LVM storage pool driver (which would need to keep second copy of volumes list - just like file driver). QubesOS/qubes-issues#2256	2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki	1a7f2892d1	storage/lvm: fix logic regarding snapshots, start, stop etc There are mutiple cases when snapshots are inconsistently created, for example: - "-back" snapshot created from the "new" data, instead of old one - "-snap" created even when volume.snap_on_start=False - probably more Fix this by following volume.snap_on_start and volume.save_on_stop directly, instead of using abstraction of old volume types. QubesOS/qubes-issues#2256	2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki	ab9d7fbb76	storage: improve/fix handling extra volumes Just calling pool.init_volume isn't enough - a lot of code depends on additional data loaded into vm.storage object. Provide a convenient wrapper for this. At the same time, fix loading extra volumes from qubes.xml - don't fail on volume not mentioned in initial vm.volume_config. QubesOS/qubes-issues#2256	2016-11-04 14:18:56 +01:00
Marek Marczykowski-Górecki	4323651afb	storage/lvm: remove duplicated _reset function There were two: _reset and _reset_volume. Neither of them was working, but the later was closer. Remove the other one. QubesOS/qubes-issues#2256	2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki	37dbf29bc1	storage/lvm: don't fail on removing already removed volumes This may happen when removing not fully created VM. QubesOS/qubes-issues#2256	2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki	400e92b25a	storage/lvm: misc fixes - add missing lvm remove call when commiting changes - delay creating volatile image until domain startup (it will be created then anyway) - reset cache only when really changed anything - attach VM to the volume (snapshot) created for its runtime - to not expose changes (for example in root volume) to child VMs until shutdown QubesOS/qubes-issues#2412 QubesOS/qubes-issues#2256	2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki	0471453773	storage/lvm: call `lvm` directly, don't use qubes-lvm wrapper The wrapper doesn't do anything else than translating command parameters, but it's load time is significant (because of python imports mostly). Since we can't use python lvm API from non-root user anyway, lets drop the wrapper and call `lvm` directly (or through sudo when necessary). This makes VM startup much faster - storage preparation is down from over 10s to about 3s. QubesOS/qubes-issues#2256	2016-11-04 14:18:55 +01:00
Marek Marczykowski-Górecki	9197bde76e	storage/lvm: use dd for importing volumes ...instead of manual copy in python. DD is much faster and when used with `conv=sparse` it will correctly preserve sparse image. QubesOS/qubes-issues#2256	2016-11-04 14:18:54 +01:00
Marek Marczykowski-Górecki	38fc504ca0	qubes/vm/net: set mapped IP info before attaching network Set parameters for possibly hiding domain's real IP before attaching network to it, otherwise we'll have race condition with vif-route-qubes script. QubesOS/qubes-issues#1143	2016-11-01 00:37:43 +01:00
Marek Marczykowski-Górecki	b4fa8cdce3	qubes/vm/net: use domain's "visible IP" for a gateway address This is the IP known to the domain itself and downstream domains. It may be a different one than seen be its upstream domain. Related to QubesOS/qubes-issues#1143`	2016-11-01 00:30:11 +01:00
Marek Marczykowski-Górecki	ec81b3046f	tests: add missing app.save() before starting a domain Otherwise domain will be unknown to other processes (like qrexec services).	2016-11-01 00:28:37 +01:00
Marek Marczykowski-Górecki	d999d91049	tests: few more tests for fake/custom IP QubesOS/qubes-issues#1143 QubesOS/qubes-issues#1477	2016-10-31 03:39:46 +01:00
Marek Marczykowski-Górecki	ea33fef9cc	tests: drop dispvm_netvm tests This property no longer exists in core3.	2016-10-31 03:10:12 +01:00
Marek Marczykowski-Górecki	5072acc8f2	tests: custom VM IP QubesOS/qubes-issues#1477	2016-10-31 03:09:45 +01:00
Marek Marczykowski-Górecki	b8145595a9	qubes/vm/net: allow setting custom IP Fixes QubesOS/qubes-issues#1477	2016-10-31 03:04:13 +01:00
Marek Marczykowski-Górecki	4585f2b503	tests: add tests for fake IP feature QubesOS/qubes-issues#1143	2016-10-31 02:17:21 +01:00
Marek Marczykowski-Górecki	2c6c476410	qubes/vm/net: add feature of hiding real IP from the VM This helps hiding VM IP for anonymous VMs (Whonix) even when some application leak it. VM will know only some fake IP, which should be set to something as common as possible. The feature is mostly implemented at (Proxy)VM side using NAT in separate network namespace. Core here is only passing arguments to it. It is designed the way that multiple VMs can use the same IP and still do not interfere with each other. Even more: it is possible to address each of them (using their "native" IP), even when multiple of them share the same "fake" IP. Original approach (marmarek/old-qubes-core-admin#2) used network script arguments by appending them to script name, but libxl in Xen >= 4.6 fixed that side effect and it isn't possible anymore. So use QubesDB instead. From user POV, this adds 3 "features": - net/fake-ip - IP address visible in the VM - net/fake-gateway - default gateway in the VM - net/fake-netmask - network mask The feature is enabled if net/fake-ip is set (to some IP address) and is different than VM native IP. All of those "features" can be set on template, to affect all of VMs. Firewall rules etc in (Proxy)VM should still be applied to VM "native" IP. Fixes QubesOS/qubes-issues#1143	2016-10-31 02:06:01 +01:00
Marek Marczykowski-Górecki	b91714b204	qubes/features: handle recursive templates Have features.check_with_template() check the template recursively. The longest path (currently) is: DispVM -> AppVM -> TemplateVM.	2016-10-31 02:04:27 +01:00
Wojtek Porczyk	0141e1ac73	qubes/app: Allow keeping lock after load QubesOS/qubes-issues#1729	2016-10-28 15:43:43 +02:00
Marek Marczykowski-Górecki	8cf19e3c92	tests/backupcompatibility: verify restored VM properties	2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki	36bd834c01	core2migration: try to set properties to "default" when possible Core3 keep information whether property have default value for all the properties (not only few like netvm or kernel). Try to use this feature as much as possible.	2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki	64ac7f6e8d	tests/backup: check non-ASCII passphrase QubesOS/qubes-issues#2398	2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki	af182c4fd1	backup: fixup restore options just before restoring VMs When user included/excluded some VMs for restoration, it may be neceesarry to fix dependencies between them (for example when default template is no longer going to be restored). Also fix handling conflicting names.	2016-10-28 11:53:34 +02:00
Marek Marczykowski-Górecki	043d20c05d	backup: fix handling non-ascii characters in backup passphrase Fixes QubesOS/qubes-issues#2398	2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki	fc00dd211e	tests/backup: test backup with non-ASCII passphrase	2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki	49e718cf57	backup: mark 'encryption' option as deprecated - all backups are encrypted QubesOS/qubes-issues#971	2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki	51b66208f3	backup: verify if archive chunks are not reordered Now, when file name is also integrity protected (prefixed to the passphrase), we can make sure that input files are given in the same order. And are parts of the same VM. QubesOS/qubes-issues#971	2016-10-28 11:53:33 +02:00
Marek Marczykowski-Górecki	4ad15c082b	backup: add 'backup_id' to integrity protection This prevent switching parts of backup of the same VM between different backups made by the same user (or actually: with the same passphrase). QubesOS/qubes-issues#971	2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki	418d749680	backup: use 'scrypt' tool for backup encryption and integrity protection `openssl dgst` and `openssl enc` used previously poorly handle key stretching - in case of `openssl enc` encryption key is derived using single MD5 iteration, without even any salt. This hardly prevent brute force or even rainbow tables attacks. To make things worse, the same key is used for encryption and integrity protection which ease brute force even further. All this is still about brute force attacks, so when using long, high entropy passphrase, it should be still relatively safe. But lets do better. According to discussion in QubesOS/qubes-issues#971, scrypt algorithm is a good choice for key stretching (it isn't the best of all existing, but a good one and widely adopted). At the same time, lets switch away from `openssl` tool, as it is very limited and apparently not designed for production use. Use `scrypt` tool, which is very simple and does exactly what we need - encrypt the data and integrity protect it. Its archive format have own (simple) header with data required by the `scrypt` algorithm, including salt. Internally data is encrypted with AES256-CTR and integrity protected with HMAC-SHA256. For details see: https://github.com/tarsnap/scrypt/blob/master/FORMAT This means change of backup format. Mainly: 1. HMAC is stored in scrypt header, so don't use separate file for it. Instead have data in files with `.enc` extension. 2. For compatibility leave `backup-header` and `backup-header.hmac`. But `backup-header.hmac` is really scrypt-encrypted version of `backup-header`. 3. For each file, prepend its identifier to the passphrase, to authenticate filename itself too. Having this we can guard against reordering archive files within a single backup and across backups. This identifier is built as: backup ID (from backup-header)!filename! For backup-header itself, there is no backup ID (just 'backup-header!'). Fixes QubesOS/qubes-issues#971	2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki	d7c355eadb	backup: make wait_backup_feedback/handle_streams less ugly Have a generic function `handle_streams`, instead of `wait_backup_feedback` with open coded process names and manual iteration over them. No functional change, besides minor logging change.	2016-10-28 11:53:32 +02:00
Marek Marczykowski-Górecki	6ee200236c	tests/backup: verify migration into LVM thin pool	2016-10-28 11:53:31 +02:00
Marek Marczykowski-Górecki	673fe4423a	tests: handle LVM thin pool	2016-10-28 11:53:31 +02:00
Marek Marczykowski-Górecki	fbecd08a58	tests/backup: exclude some VMs during restore	2016-10-28 11:53:31 +02:00
Wojtek Porczyk	3553b2e1d4	Make pylint happy	2016-10-25 17:27:02 +02:00
Wojtek Porczyk	8edbf0e406	qubes: Document all the events fixes QubesOS/qubes-issues#1811	2016-10-25 17:11:38 +02:00
Wojtek Porczyk	5e62d3f7cb	qubes/tests: substitute_entry_points New context manager for temporary overriding entry point groups. fixes QubesOS/qubes-issues#2111	2016-10-24 15:16:39 +02:00
Wojtek Porczyk	4c73c1b93a	More green paint	2016-10-19 16:09:58 +02:00
Marek Marczykowski-Górecki	5babb68031	tests/backupcompatibility: verify if all files got restored There is still no verification of disk images content, nor VM properties...	2016-10-19 01:54:44 +02:00
Wojtek Porczyk	8097da7cab	Paint the project green for testbench launch	2016-10-18 19:07:20 +02:00
Wojtek Porczyk	c81346ba51	qubes/test/run: Print tracebacks of expected fails fixes QubesOS/qubes-issues#2376	2016-10-14 17:20:14 +02:00
Wojtek Porczyk	526f2c3751	Merge remote-tracking branch 'marmarek/core3-backup' into core3-devel	2016-10-14 15:29:37 +02:00
Wojtek Porczyk	e06829ab2c	Make pylint happy	2016-10-11 13:42:37 +02:00
Wojtek Porczyk	c6c0a545e6	Merge remote-tracking branch 'origin/pull/58/head' into core3-devel	2016-10-11 11:37:15 +02:00
Marek Marczykowski-Górecki	33fecd90c1	qubes/backup: misc fixes Fix restoring ProxyVM and NetVM from core2. Use correct VM class.	2016-10-05 01:58:11 +02:00
Marek Marczykowski-Górecki	339c47480e	qubes/backup: include LVM volumes content in backup Use just introduced tar writer to archive content of LVM volumes (or more generally: block devices). Place them as 'private.img' and 'root.img' files in the backup - just like in old format. This require support for replacing file name in tar header - another thing trivially supported with tar writer.	2016-10-05 01:55:30 +02:00
Marek Marczykowski-Górecki	36eb7f923f	qubes/tarwriter: add simple sparse-tar writer module tar can't write archive with _contents_ of block device. We need this to backup LVM-based disk images. To avoid dumping image to a file first, create a simple tar archiver just for this purpose. Python is not the fastest possible technology, it's 3 times slower than equivalent written in C. But it's much easier to read, much less error-prone, and still process 1GB image under 1s (CPU time, leaving along actual disk reads). So, it's acceptable.	2016-10-05 01:54:41 +02:00
Marek Marczykowski-Górecki	278a5340dc	qubes/backup: fix relative path calculation os.path.relpath strip trailing '/.' from the path, but it is important to distinguish whole-directory archive (which is tar of '.').	2016-10-04 21:38:59 +02:00
Wojtek Porczyk	72bc0506b9	Code documentation: fix 🎫 reference to query github Previously it queried long obsolete Trac at wiki.qubes-os.org.	2016-10-04 15:31:06 +02:00
Wojtek Porczyk	d7d926edaa	Fix manpage generation	2016-10-04 11:32:04 +02:00
Wojtek Porczyk	9dc37c1ee7	Add possibility to override libvirt config This is the equivalent of "custom config" from R3.x. fixes QubesOS/qubes-issues#1798	2016-10-04 11:31:31 +02:00
Marek Marczykowski-Górecki	ab69fdd7f4	qubes/backup: reduce code duplication Move inner tar process cleanup to a separate function	2016-10-03 13:43:36 +02:00
WetwareLabs	cedd822735	Fix sending monitor layout info when xrandr has one output disconnected Signed-off-by: WetwareLabs <marcus@wetwa.re>	2016-09-29 14:13:38 +02:00
Marek Marczykowski-Górecki	c4632d6be8	tests/backup: test idea	2016-09-29 02:08:30 +02:00
Marek Marczykowski-Górecki	f2d79b9379	tests/backup: use round volume size When handling LVM volumes, size must be multiply of 4MB.	2016-09-29 02:08:30 +02:00
Marek Marczykowski-Górecki	e938aa61ab	tests: cleanup test LVM volumes Handle the case when vm.remove_from_disk does not cleanup all the things.	2016-09-29 02:08:30 +02:00
Marek Marczykowski-Górecki	20590bff57	backup: adjust LVM volume size when restoring its content. Old backup metadata (old qubes.xml) does not contain info about individual volume sizes. So, extract it from tar header (using verbose output during restore) and resize volume accordingly. Without this, restoring volumes larger than default would be impossible.	2016-09-29 02:08:29 +02:00
Marek Marczykowski-Górecki	0a35bd06aa	backup: support relocating files to different storage pool To ease all this, rework restore workflow: first create QubesVM objects, and all their files (as for fresh VM), then override them with data from backup - possibly redirecting some files to new location. This allows generic code to create LVM volumes and then only restore its content.	2016-09-29 02:08:29 +02:00
Marek Marczykowski-Górecki	4d45dd5549	tests/backup: check backup+restore of LVM based VM The test fails for now...	2016-09-29 01:59:55 +02:00
Marek Marczykowski-Górecki	226695534b	tests/backup: handle non-default pool in BackupTestsMixin	2016-09-29 01:59:54 +02:00
Marek Marczykowski-Górecki	ae42308f5f	storage: improve handling volume export 1. Add a helper function on vm.storage. This is equivalent of: vm.storage.get_pool(vm.volumes[name]).export(vm.volumes[name]) 2. Make sure the path returned by `export` on LVM volume is accessible.	2016-09-29 01:59:54 +02:00
Marek Marczykowski-Górecki	9395e8fc33	storage: set only 'default' pool when creating VM on custom one Do not replace 'linux-kernel' pool for example.	2016-09-29 01:59:54 +02:00
Marek Marczykowski-Górecki	e1d9de1cc2	tests/backup: minor fix for python3	2016-09-29 01:59:54 +02:00
Marek Marczykowski-Górecki	6d5959b31d	tests/backup: use proper logging instead of print	2016-09-29 01:59:53 +02:00
Marek Marczykowski-Górecki	016c3d8e88	tests/backup: check restored disk images	2016-09-29 01:59:53 +02:00
Bahtiar `kalkin-` Gadimov	8d9b6f19fd	Add catch-all '*' to event handlers	2016-09-28 21:28:43 +02:00
Bahtiar `kalkin-` Gadimov	ef56620b6e	Make pylint happy ♥ qubes/core2migration.py	2016-09-28 21:28:33 +02:00
Marek Marczykowski-Górecki	e499b529ad	tests: move BackupTestMixin to qubes.tests.int.backup This is much more logical place, don't pollute main qubes.tests module.	2016-09-25 16:31:31 +02:00
Marek Marczykowski-Górecki	533804ebdc	Make pylint happy ♥	2016-09-21 16:39:06 +02:00
Marek Marczykowski-Górecki	96a4bb650b	qubes/tools: qvm-backup and qvm-backup-restore tools Fixes QubesOS/qubes-issues#1213 Fixes QubesOS/qubes-issues#1214	2016-09-21 16:02:50 +02:00
Marek Marczykowski-Górecki	7af3f4b19a	Make pylint happy	2016-09-19 20:36:32 +02:00
Marek Marczykowski-Górecki	d4f29bb8b7	qubes: add qvm-firewall tool Fixes QubesOS/qubes-issues#1815	2016-09-19 20:36:32 +02:00
Marek Marczykowski-Górecki	f8dd7e6cb7	qubes/firewall: make xml parameter to Rule optional QubesOS/qubes-issues#1815	2016-09-19 20:36:32 +02:00
Marek Marczykowski-Górecki	cdc97730cd	qubes/firewall: apply only IPv4 rules Currently dom0 do not assign IPv6 addresses for VMs, so there is no sense in IPv6 firewall yet. QubesOS/qubes-issues#1815	2016-09-19 20:36:32 +02:00
Marek Marczykowski-Górecki	202042bd8d	tests: update for new firewall API QubesOS/qubes-issues#1815	2016-09-19 20:36:31 +02:00
Marek Marczykowski-Górecki	d5b3d971ee	qubes/ext/r3compat: update firewall handling for new API QubesOS/qubes-issues#1815	2016-09-19 20:36:31 +02:00
Marek Marczykowski-Górecki	5123f466eb	qubes/firewall: allow listing only IPv4/IPv6 rules This will allow setting only IPv4-related rules to IPv4 address, and the same for IPv6 QubesOS/qubes-issues#1815	2016-09-19 20:36:31 +02:00
Marek Marczykowski-Górecki	e01f7b97d9	qubes/vm: plug in new firewall code, create QubesDB entries QubesOS/qubes-issues#1815	2016-09-19 20:36:31 +02:00
Marek Marczykowski-Górecki	1da75a676f	qubes/firewall: new firewall interface First part - handling firewall.xml and rules formatting. Specification on https://qubes-os.org/doc/vm-interface/ TODO (for dom0): - plug into QubesVM object - expose rules in QubesDB (including reloading) - drop old functions (vm.get_firewall_conf etc) QubesOS/qubes-issues#1815	2016-09-19 20:36:30 +02:00
Marek Marczykowski-Górecki	1af1784c69	Merge remote-tracking branch 'qubesos/pr/54' into core3-devel * qubesos/pr/54: Fix file storage resize Fix lvm reset_cache size cache	2016-09-19 20:36:20 +02:00
Marek Marczykowski-Górecki	0ea7afd245	tests: adjust LVM tests for non-default VG name	2016-09-19 20:35:25 +02:00
Marek Marczykowski-Górecki	8b408a7a70	qubes/vm: minor formating	2016-09-13 02:15:25 +02:00
Marek Marczykowski-Górecki	8ca08c7790	qubes/ext/pci: fix handling dom0 before starting first VM Before starting fist VM, backend/pci xenstore directory does not exists. Do not crash on it QubesOS/qubes-issues#2257	2016-09-13 02:15:25 +02:00
Marek Marczykowski-Górecki	2956f62bbc	qmemman: fix meminfo handling This fixes "a2d9b15 qmemman: support simple VM meminfo format"	2016-09-13 02:15:25 +02:00
Marek Marczykowski-Górecki	9ae3dc2ebf	tests/qvm-device: tolerate different 'column' tool versions Newer one do output column even if empty, but older one (in Travis-CI env) do not. Ignore trailing spaces to work with both version.	2016-09-13 02:01:22 +02:00
Marek Marczykowski-Górecki	141128e768	qubes/tools: fix 'column' input formating Include '\n' at the last line too. Otherwise the tool (depending on version) will complain about truncated line ('line too long').	2016-09-13 01:22:24 +02:00
Marek Marczykowski-Górecki	bb78eb1ce6	qubes: fix resetting already default property `getattr(instance, self.__name__)` always succeed if attribute has default value. This results in deleting attribute even when it isn't set.	2016-09-08 04:17:58 +02:00
Marek Marczykowski-Górecki	a2d9b15413	qmemman: support simple VM meminfo format Instead of excerpt from /proc/meminfo, use just one integer. This make qmemman handling much easier and ease implementation for non-Linux OSes (where /proc/meminfo doesn't exist). For now keep also support for old format. Fixes QubesOS/qubes-issues#1312	2016-09-08 04:17:48 +02:00
Marek Marczykowski-Górecki	aa0674e8bb	qubes/vm: make VM QubesDB interface as much compatible as possible All the base keys can be kept easily the same, so do it. QubesOS/qubes-issues#1812	2016-09-08 04:17:48 +02:00
Marek Marczykowski-Górecki	d5a41e838f	Revert "qubes/vm/qubesvm: remove prefixes from qubesdb keys" There is no point in changing public API for just a change without any better reason. It turned out most of those settings will be the same in Qubes 4.0, so keep names the same. This reverts commit `2d6ad3b60c`. QubesOS/qubes-issues#1812	2016-09-08 04:17:48 +02:00
Marek Marczykowski-Górecki	f6bc61997a	qubes/vm: fix handling dynamic netvm change Correctly detach network before attaching new one.	2016-09-08 04:17:48 +02:00
Marek Marczykowski-Górecki	c534b68665	qubes/vm: start VM daemons as normal user This is migration of core2 commits: commit `d0ba43f253` Author: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Date: Mon Jun 6 02:21:08 2016 +0200 core: start guid as normal user even when VM started by root Another attempt to avoid permissions-related problems... QubesOS/qubes-issues#1768 commit `89d002a031` Author: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Date: Mon Jun 6 02:19:51 2016 +0200 core: use runuser instead of sudo for switching root->user There are problems with using sudo in early system startup (systemd-logind not running yet, pam_systemd timeouts). Since we don't need full session here, runuser is good enough (even better: faster). commit `2265fd3d52` Author: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Date: Sat Jun 4 17:42:24 2016 +0200 core: start qubesdb as normal user, even when VM is started by root On VM start, old qubesdb-daemon is terminated (if still running). In practice it happen only at VM startart (shutdown and quickly start again). But in that case, if the VM was started by root, such operation would fail. So when VM is started by root, make sure that qubesdb-daemon will be running as normal user (the first user in group 'qubes' - there should be only one). Fixes QubesOS/qubes-issues#1745	2016-09-08 04:17:47 +02:00
Marek Marczykowski-Górecki	19d9edc291	qubes/ext/gui: adjust guid parameters when running on KDE5 Commit from core2: commit `94d52a13e7` core: adjust guid parameters when running on KDE5 On KDE5 native decoration plugin is used and requires special properties set (instead of `_QUBES_VMNAME` etc). Special care needs to be taken when detecting environment, because environment variables aren't good enough - this script may be running with cleared environment (through sudo, or from systemd). So check properties of X11 root window. QubesOS/qubes-issues#1784	2016-09-08 04:17:47 +02:00
Bahtiar `kalkin-` Gadimov	d63d93b0ab	Fix file storage resize	2016-09-05 16:56:57 +02:00
Bahtiar `kalkin-` Gadimov	d72448615d	Fix lvm reset_cache size cache	2016-09-04 23:25:39 +02:00
Marek Marczykowski-Górecki	2285789b49	Merge remote-tracking branch 'qubesos/pr/54' into core3-devel * qubesos/pr/54: Volume add size property wrapper	2016-09-04 22:38:06 +02:00
Bahtiar `kalkin-` Gadimov	b863152f73	Volume add size property wrapper	2016-09-04 21:55:56 +02:00
Marek Marczykowski-Górecki	16db68b053	Merge branch 'core3-devices' into core3-devel * core3-devices: Fix core2migration and tests for new devices API tests: more qubes.devices tests qubes/ext/pci: implement pci-no-strict-reset/BDF feature qubes/tools: allow calling qvm-device as qvm-devclass (like qvm-pci) qubes: make pylint happy qubes/tools: add qvm-device tool (and tests) tests: load qubes.tests.tools.qvm_ls tests: PCI devices tests tests: add context manager to catch stdout qubes/ext/pci: move PCI devices handling to an extension qubes/devices: use more detailed exceptions than just KeyError qubes/devices: allow non-persistent attach qubes/storage: misc fixes for VM-exposed block devices handling qubes: new devices API Fixes QubesOS/qubes-issues#2257	2016-09-04 21:32:01 +02:00
Marek Marczykowski-Górecki	ae72e294cd	Merge remote-tracking branch 'qubesos/pr/52' into core3-devel * qubesos/pr/52: qvm-block extend Fix help message Fix a few typo bugs in qubes.storage.lvm qvm-block extends volumes to NEW_SIZE Make pylint happy ♥ qubes.storage.lvm Fix ThinVolume _size access qvm-block extend use qubes.utils.parse_size Make pylint happy ♥ qubes_lvm Fix lvm size/usage Add ThinPool.resize() Document qubes-block extend command in the manpage Fix file storage resize qvm-block add extend sub command qvm-block refactor attach/detach sub_parser init	2016-09-04 21:21:43 +02:00
Marek Marczykowski-Górecki	dc3dd1f5e9	Merge remote-tracking branch 'qubesos/pr/53' into core3-devel * qubesos/pr/53: Fix qvm-block lists internal volumes on non default pools	2016-09-04 21:11:27 +02:00
Marek Marczykowski-Górecki	6cb624daf3	Merge remote-tracking branch 'qubesos/pr/51' into core3-devel * qubesos/pr/51: Make pylint happy ♥ qubes.storage.lvm Fix lvm AppVM startup from lvm qubes.storage.lvm.qubes_lvm use debug not info Fix qvm-create lvm based AppVM qvm-block & qvm-pool add better Exception handling Remove unexpected property qvm-start --debug don't catch exceptions	2016-09-04 21:02:19 +02:00
Bahtiar `kalkin-` Gadimov	5f586a99d4	qvm-block extend Fix help message	2016-09-04 21:01:16 +02:00
Marek Marczykowski-Górecki	5cf15ef3da	tests: improve DispVM error message	2016-09-04 21:00:02 +02:00
Marek Marczykowski-Górecki	c15dc0a392	tools/qvm_block: add missing column header QubesOS/qubes-issues#2256	2016-09-04 20:59:37 +02:00
Marek Marczykowski-Górecki	c2617663e5	qubes/vm/standalonevm: volatile image should be writable QubesOS/qubes-issues#2256	2016-09-04 20:58:11 +02:00
Marek Marczykowski-Górecki	0af5fdd267	qubes/storage/file: clone file images as normal user No need to be root here. QubesOS/qubes-issues#2256	2016-09-04 20:56:02 +02:00
Marek Marczykowski-Górecki	11191ea694	Fix core2migration and tests for new devices API QubesOS/qubes-issues#2257	2016-09-04 20:55:07 +02:00
Marek Marczykowski-Górecki	e536bcca30	tests: more qubes.devices tests QubesOS/qubes-issues#2257	2016-09-03 20:41:06 +02:00
Marek Marczykowski-Górecki	93e88e0c22	qubes/ext/pci: implement pci-no-strict-reset/BDF feature Instead of old per-VM flag 'pci_strictreset', now implement this as per-device flag using features. To not fail on particular device assignment set 'pci-no-strict-reset/DEVICE-BDF' to True. For example 'pci-no-strict-reset/00:1b.0'. QubesOS/qubes-issues#2257	2016-09-03 20:41:06 +02:00
Marek Marczykowski-Górecki	4d327fbc95	qubes/tools: allow calling qvm-device as qvm-devclass (like qvm-pci) Tool can be symlinked to `qvm-class` for particular device class (for example `qvm-pci`) - then device class can be omitted. QubesOS/qubes-issues#2257	2016-09-03 20:41:05 +02:00
Marek Marczykowski-Górecki	5c7f589330	qubes: make pylint happy	2016-09-03 20:41:05 +02:00
Marek Marczykowski-Górecki	e8d011b83f	qubes/tools: add qvm-device tool (and tests) Add a tool to manipulate various devices. QubesOS/qubes-issues#2257	2016-09-03 20:41:05 +02:00
Marek Marczykowski-Górecki	6aae6863b0	tests: load qubes.tests.tools.qvm_ls The test module was implemented, but not loaded	2016-09-03 20:41:04 +02:00
Marek Marczykowski-Górecki	fbb086aed5	tests: PCI devices tests Just basic things for now. QubesOS/qubes-issues#2257	2016-09-03 20:41:04 +02:00
Marek Marczykowski-Górecki	4bfb523974	tests: add context manager to catch stdout This will avoid code duplication in tools tests.	2016-09-03 20:41:03 +02:00
Marek Marczykowski-Górecki	aa67a4512e	qubes/ext/pci: move PCI devices handling to an extension Implement required event handlers according to documentation in qubes.devices. A modification of qubes.devices.DeviceInfo is needed to allow dynamic, read-only properties. QubesOS/qubes-issues#2257	2016-09-03 20:41:03 +02:00
Marek Marczykowski-Górecki	e1de82ea53	qubes/devices: use more detailed exceptions than just KeyError Especially inherit from QubesException, so tools will treat this as properly handled error. QubesOS/qubes-issues#2257	2016-09-03 20:41:03 +02:00
Marek Marczykowski-Górecki	e5d6c4e078	qubes/devices: allow non-persistent attach QubesOS/qubes-issues#2257	2016-09-03 20:41:02 +02:00
Marek Marczykowski-Górecki	70d3f58024	qubes/storage: misc fixes for VM-exposed block devices handling Add 'backenddomain' element when source (not target) domain is not dom0. Fix XML elemenet name. Actually set volume.domain when listing VM-exposed devices. QubesOS/qubes-issues#2256	2016-09-03 20:41:02 +02:00
Marek Marczykowski-Górecki	d7a3c0d319	qubes: new devices API Allow device plugin to list attached and available devices. Enforce at API level every device being exposed by some domain. This commit only changes devices API, but not update existing users (pci) yet. QubesOS/qubes-issues#2257	2016-09-03 20:40:39 +02:00
Bahtiar `kalkin-` Gadimov	e4433f9a65	Make pylint happy ♥ qubes.storage.lvm	2016-09-02 20:16:03 +02:00
Bahtiar `kalkin-` Gadimov	46eb4a72dd	Fix a few typo bugs in qubes.storage.lvm	2016-09-02 19:57:35 +02:00
Bahtiar `kalkin-` Gadimov	dcfc47fefe	qvm-block extends volumes to NEW_SIZE	2016-09-02 19:57:13 +02:00
Bahtiar `kalkin-` Gadimov	be48d48e56	Make pylint happy ♥ qubes.storage.lvm	2016-09-02 19:46:11 +02:00
Bahtiar `kalkin-` Gadimov	8974b2cbc0	Fix ThinVolume _size access	2016-09-02 19:45:31 +02:00
Bahtiar `kalkin-` Gadimov	1cf701ff60	qvm-block extend use qubes.utils.parse_size	2016-09-02 19:21:04 +02:00
Bahtiar `kalkin-` Gadimov	818ed20248	Make pylint happy ♥ qubes_lvm	2016-09-02 19:17:00 +02:00
Bahtiar `kalkin-` Gadimov	bbcf16410c	Fix qvm-block lists internal volumes on non default pools	2016-08-29 17:39:49 +02:00
Bahtiar `kalkin-` Gadimov	1ca3c00797	Fix lvm size/usage - Uses a size_cache, because it's faster than doing a call for each volume	2016-08-29 00:15:47 +02:00
Bahtiar `kalkin-` Gadimov	0561dfce60	Add ThinPool.resize()	2016-08-29 00:13:50 +02:00
Bahtiar `kalkin-` Gadimov	e7caa14a85	Fix file storage resize	2016-08-29 00:09:55 +02:00
Bahtiar `kalkin-` Gadimov	4733e836ce	qvm-block add extend sub command	2016-08-29 00:09:55 +02:00
Bahtiar `kalkin-` Gadimov	4c65a9c69a	qvm-block refactor attach/detach sub_parser init	2016-08-29 00:09:53 +02:00
Bahtiar `kalkin-` Gadimov	355ea0c2a8	Fix lvm AppVM startup from lvm	2016-08-29 00:05:19 +02:00
Bahtiar `kalkin-` Gadimov	24bbc971b0	qubes.storage.lvm.qubes_lvm use debug not info	2016-08-29 00:05:19 +02:00
Bahtiar `kalkin-` Gadimov	e0bf4bdb8e	Fix qvm-create lvm based AppVM	2016-08-29 00:05:18 +02:00
Bahtiar `kalkin-` Gadimov	c0f1a6b4e8	qvm-block & qvm-pool add better Exception handling	2016-08-29 00:05:18 +02:00
Bahtiar `kalkin-` Gadimov	b712d0f6f3	Remove unexpected property	2016-08-29 00:05:18 +02:00
Bahtiar `kalkin-` Gadimov	8037182dcd	qvm-start --debug don't catch exceptions	2016-08-29 00:05:17 +02:00
Marek Marczykowski-Górecki	3e30dc17cb	qubes/vm: remove special case for 'meminfo-writer' feature It should be either in a plugin, or handled entirely from inside of VM. But surely not in main QubesVM class. Fixes QubesOS/qubes-issues#2101	2016-08-18 11:59:45 +02:00
Marek Marczykowski-Górecki	0293c1c7ef	qubes/vm: move misc XML tags loading to separate method	2016-08-18 11:57:11 +02:00
Marek Marczykowski-Górecki	a5e575618c	qubes-lvm: drop sudo from lvm calls The script is already running as root QubesOS/qubes-issues#2256	2016-08-18 11:52:26 +02:00
Marek Marczykowski-Górecki	6bdc82b3a6	qubes-lvm: do not import 'qubes' module This script should run as fast as possible, so avoid importing large module. In fact the only used thing was argparse wrapper, so switch to the standard one and drop aliases. QubesOS/qubes-issues#2256	2016-08-18 11:52:26 +02:00
Marek Marczykowski-Górecki	de5d420331	qubes-lvm: drop unused lvm_image_changed function It tries to parse (untrusted) volume content, so remove it to not use it accidentally. QubesOS/qubes-issues#2256	2016-08-18 11:43:01 +02:00
Marek Marczykowski-Górecki	06f46243ec	qubes/storage: add `resize` to Pool API It was already implemented by FilePool and used by Storage object, but wasn't included in base class.	2016-08-18 11:42:00 +02:00
Marek Marczykowski-Górecki	42d6fddcd3	qubes/storage: support on-line resize Call `qubes.ResizeDisk` qrexec service after resizing volume, to adjust filesystem size there.	2016-08-18 11:40:38 +02:00
Marek Marczykowski-Górecki	6e383c703d	qubes/storage: allow passing just volume name to storage.resize It makes much more sense to call: vm.storage.resize('private', 10000000000) instead of: vm.storage.resize(vm.volumes['private'], 10000000000)	2016-08-17 22:42:15 +02:00
Marek Marczykowski-Górecki	d8286b6dab	tests: port vm_qrexec_gui to core3 API	2016-08-17 22:41:57 +02:00
Marek Marczykowski-Górecki	e19df4c5b6	tests: port dispvm tests to core3 API Some tests do not apply, as there is no savefile and attributes propagation is much simpler. Dropped tests: - test_000_firewall_propagation - test_001_firewall_propagation - test_000_prepare_dvm QubesOS/qubes-issues#2253	2016-08-17 04:15:35 +02:00
Marek Marczykowski-Górecki	46791e05f4	qubes/dispvm: do not clone 'template' property from base VM template should point at the base VM itself.	2016-08-17 02:14:19 +02:00
Marek Marczykowski-Górecki	53b2b30c0a	qubes: fix some error messages	2016-08-17 02:13:59 +02:00
Marek Marczykowski-Górecki	3a2a87839f	qubes/vm: fix TemplateVM.commit_changes Storage method was renamed to just `commit`, as it applies not only to templates now.	2016-08-17 01:47:09 +02:00
Marek Marczykowski-Górecki	d9e23d6535	qubes/core2migration: make pylint happy It doesn't see the class that is just being defined. But since it is in lambda, that's ok.	2016-08-17 01:46:13 +02:00
Marek Marczykowski-Górecki	6b546ca1e6	tests/backupcompatibility: no errors are expected on restoring R1 backup Fallback to default kernel is enabled by default.	2016-08-17 01:31:33 +02:00
Marek Marczykowski-Górecki	690727fef8	tests/backup: remove VMs in reversed order One place had missing `reversed()`.	2016-08-17 01:30:34 +02:00
Marek Marczykowski-Górecki	bbbe045b62	qubes/backup: better handle restoring kernel=None case It makes sense for HVM domains, so allow such configuration.	2016-08-17 01:18:11 +02:00
Marek Marczykowski-Górecki	5d8ecd60de	qubes: minor fixes in handling defaults - fix assigning 'template' property - do not do it if VM already have it set - cap default maxmem at 4000, as we clamp it to 10*memory anyway (and default memory is 400)	2016-08-17 00:50:38 +02:00
Marek Marczykowski-Górecki	dd93650ea7	tests/backup: fix handling different VM types Not all VM types have 'template' property.	2016-08-17 00:49:53 +02:00
Marek Marczykowski-Górecki	460f0849a7	qubes/features: handle recursive templates in check_with_template This is the case for DispVMs: DispVM -> AppVM -> TemplateVM	2016-08-17 00:47:32 +02:00
Marek Marczykowski-Górecki	1a215e47ab	qubes: add 'default_dispvm' property - both Qubes and QubesVM QubesOS/qubes-issues#2253	2016-08-17 00:46:43 +02:00
Marek Marczykowski-Górecki	ab6f961931	qubes/dispvm: misc fixes, make it actually working - DispVM is no longer a special case for storage - Add missing 'rw=True' for volatile volume - Handle storage initialization (copy&paste from AppVM) - Clone properties from DispVM template QubesOS/qubes-issues#2253	2016-08-17 00:42:17 +02:00
Marek Marczykowski-Górecki	2f977ff4f5	backup: open backup collection in offline mode Otherwise it may try to manipulate live libvirt objects with the same names (and probably different UUIDs, which would fail).	2016-08-16 00:00:14 +02:00
Marek Marczykowski-Górecki	eb637147fb	core2migration: adjust for recent storage API changes QubesVM.storage.vmdir uses QubesVM.dir_path, so can't be used directly as default value for dir_path. Use `super()`.	2016-08-15 23:47:28 +02:00
Marek Marczykowski-Górecki	660c726315	tests: add qvm-check	2016-08-09 05:14:13 +02:00
Marek Marczykowski-Górecki	da6faf83bf	qubes/tools: allow unpause multiple domains at once There is no reason to not allow it.	2016-08-09 05:14:13 +02:00
Marek Marczykowski-Górecki	a1e2888e96	qubes/tools: fix messages in qvm-check	2016-08-09 05:14:13 +02:00
Marek Marczykowski-Górecki	d4e534bc05	qubes/vm: fix pause handling 1. There is no such thing as libvirt_domain.pause(). 2. libvirt_domain.state() returns [state, reason]	2016-08-09 05:14:12 +02:00
Marek Marczykowski-Górecki	d2393f91e8	qubes/tools: adjust qvm-check for core3 API	2016-08-09 05:14:12 +02:00
Marek Marczykowski-Górecki	4d5a314e77	qubes/tools: add hidden --offline-mode options It is useful on some cases to prevent talking to hypervisor. Warning - it may have sense only when action do not access any runtime VM status. For example running the domain will fail, but changing its properties should work.	2016-08-09 05:14:12 +02:00
Marek Marczykowski-Górecki	96d032919a	tests: minor fix to revert changes tests, mark as expected failure While the test should be ok now, the functionality itself doesn't work (at least on file volumes).	2016-08-09 05:14:12 +02:00
Marek Marczykowski-Górecki	ac68ffe39f	tests: correctly compare DeviceManager objects Usefull for assertEqual(vm1.devices, vm2.devices) - clone tests, backup tests etc.	2016-08-09 05:14:11 +02:00
Marek Marczykowski-Górecki	f17f80ee9d	qubes/tools: add qvm-template-commit This tool is called from /etc/xen/scripts/block-origin script at VM shutdown.	2016-08-09 05:14:11 +02:00
Marek Marczykowski-Górecki	6490767278	qubes/storage: update vm.storage.commit Better name it 'commit', as it isn't only for template. And also fix to call `pool.commit`	2016-08-09 05:14:11 +02:00
Marek Marczykowski-Górecki	ec7a84ea9d	minor: formatting QubesOS/qubes-issues#	2016-08-09 05:14:10 +02:00
Marek Marczykowski-Górecki	af7b4087f0	tests: port few more tests to core3 API	2016-08-09 05:14:10 +02:00
Marek Marczykowski-Górecki	faa341ffa2	tests/backup: adjust for core3 API properties to verify after restore	2016-08-09 05:14:10 +02:00
Marek Marczykowski-Górecki	7d1cbd6934	tests: handle per-template tests in BackupTestsMixin	2016-08-09 05:14:09 +02:00
Marek Marczykowski-Górecki	fba6eac07f	qubes/tests: do not undefine libvirt domain twice Do not access vm.libvirt_domain after it being already removed - this will redefine it again in libvirt, just to undefine it in a moment. On the other hand, few lines below there is fallback libvirt cleanup, in case of proper one not working.	2016-08-09 05:14:09 +02:00
Marek Marczykowski-Górecki	6d45b97357	tests: workaround 'int' name being shadowed by module name qubes.tests.int shadows 'int' type in qubes.tests module. QubesOS/qubes-issues#2241	2016-08-09 05:14:09 +02:00
Marek Marczykowski-Górecki	3f6c96ee39	qubes/vm: fix handling autostart set/reset 1. Service symlink must have ".service" extension 2. Disable service on property reset to default	2016-08-09 05:14:09 +02:00
Marek Marczykowski-Górecki	5b4e30524a	qubes/vm: remove duplicated check for duplicate name on rename	2016-08-09 03:02:04 +02:00
Marek Marczykowski-Górecki	73d61bb378	qubes/events: fix event handlers ordering Since "qubes: fix event framework", handlers from extensions looks the same as from the VM class itself, so it isn't possible to order them correctly. Specification says: For each class first are called bound handlers (specified in class definition), then handlers from extensions. Aside from above, remaining order is undefined. So, restore this property, which is later correctly used to order handlers.	2016-08-09 02:58:14 +02:00
Marek Marczykowski-Górecki	5177f5e12b	qubes/core2migration: fix vm.template handling VM constructor can't be given just template QID, it must be TemplateVM object.	2016-08-09 02:57:23 +02:00
Marek Marczykowski-Górecki	067cfb7cd6	Send approximate physical screen dimensions to the VM When properly set, applications will have a chance to automatically detect HiDPI and act accordingly. This is the case for Fedora 23 template and GNOME apps (maybe even all built on top of GTK). But for privacy reasons, don't provide real values, only some approximate one. Give enough information to distinguish DPI above 150, 200 and 300. This is some compromise between privacy and HiDPI support. QubesOS/qubes-issues#1951 This commit is migrated from gui-daemon repository (dec462795d14a336bf27cc46948bbd592c307401).	2016-08-08 04:03:00 +02:00
Marek Marczykowski-Górecki	6bd127beb3	qubes/core2migration: fix handling templates	2016-08-08 04:00:37 +02:00
Marek Marczykowski-Górecki	5ab860a18c	qubes: don't fail VM removal if already undefined in libvirt This may happen when it was manually undefined for some reason. Or during tests cleanup (tearDown).	2016-08-08 00:15:46 +02:00
Marek Marczykowski-Górecki	53562799cd	Fix typo in libvirt auto reconnection	2016-08-08 00:13:26 +02:00
Marek Marczykowski-Górecki	4e022382a5	Merge remote-tracking branch 'origin/master' into core3-devel	2016-08-08 00:11:46 +02:00
Marek Marczykowski-Górecki	1604aee8d4	Merge remote-tracking branch 'qubesos/pr/42' into core3-devel * qubesos/pr/42: Make pylint happy ♥	2016-08-04 17:46:16 +02:00
Bahtiar `kalkin-` Gadimov	4477ea9678	Make pylint happy ♥	2016-08-04 17:38:39 +02:00
Wojtek Porczyk	288ff85731	Merge remote-tracking branch 'origin/pull/48/head' into core3-devel	2016-08-03 18:25:02 +02:00
Bahtiar `kalkin-` Gadimov	ea34c0ed56	Fix volume_exists in qubes_lvm	2016-08-03 02:57:46 +02:00
Bahtiar `kalkin-` Gadimov	02c8fc999c	ThinPool.import() skips not saveable volumes	2016-08-03 02:57:46 +02:00
Bahtiar `kalkin-` Gadimov	9190fdeb79	Fix lvm pylint disable=import-error	2016-08-03 02:57:45 +02:00
Bahtiar `kalkin-` Gadimov	064629b63a	Fix qubes.lvm _reset	2016-08-03 02:57:44 +02:00
Bahtiar `kalkin-` Gadimov	9d30bb132a	Fix umask	2016-08-03 02:57:44 +02:00
Bahtiar `kalkin-` Gadimov	824c359319	Fix ThinPool revision handling	2016-08-03 02:57:43 +02:00
Bahtiar `kalkin-` Gadimov	f3072e7dd0	qubes-create adds default lvm pool if it exists	2016-08-03 02:57:42 +02:00
Bahtiar `kalkin-` Gadimov	3ae6530cd8	Add lvm thin pool storage implementation	2016-08-03 02:57:42 +02:00
Bahtiar `kalkin-` Gadimov	7d11a7afbd	Add lvm tests	2016-08-03 02:57:41 +02:00
Bahtiar `kalkin-` Gadimov	f90d86fe13	Add qubes.utils.random_string()	2016-08-03 02:57:40 +02:00
Bahtiar `kalkin-` Gadimov	c559ffdeab	VMCollection reword vm already exists exception	2016-08-03 02:57:39 +02:00
Bahtiar `kalkin-` Gadimov	0dd1875fb6	Add vm.shutdown(wait)	2016-08-01 15:09:28 +02:00
Bahtiar `kalkin-` Gadimov	8972254e22	Add qvm-check	2016-07-21 23:40:24 +02:00
Wojtek Porczyk	ecb626d64b	qubes/vm/qubesvm: Simplify error message	2016-07-21 19:40:45 +02:00
Wojtek Porczyk	f915115cfb	Merge remote-tracking branch 'woju/pull/25/head' into core3-devel	2016-07-21 19:38:58 +02:00
Wojtek Porczyk	c5f44f9b07	Merge remote-tracking branch 'woju/pull/24/head' into core3-devel	2016-07-21 19:36:55 +02:00
Wojtek Porczyk	8e08dae447	Merge remote-tracking branch 'origin/pull/43/head' into core3-devel	2016-07-21 19:33:21 +02:00
Wojtek Porczyk	36e5bcd766	Merge remote-tracking branch 'origin/pull/39/head' into core3-devel	2016-07-21 16:43:25 +02:00
Bahtiar `kalkin-` Gadimov	d3f8fc96e4	Fix pci device attachment to QubesVM	2016-07-21 12:22:53 +02:00
Bahtiar `kalkin-` Gadimov	c18537439f	Make pylint really happy ♥♥♥	2016-07-21 12:21:56 +02:00
Bahtiar `kalkin-` Gadimov	c5307f3f47	qvm-ls hide stacktrace if qubes.xml is missing	2016-07-21 12:20:09 +02:00
Bahtiar `kalkin-` Gadimov	190f045c07	Fix qvm-prefs	2016-07-21 12:10:00 +02:00
Bahtiar `kalkin-` Gadimov	778cda0daa	qvm-pool show pools without volumes - Only empty `DomainPools` are skipped	2016-07-14 17:26:28 +02:00
Bahtiar `kalkin-` Gadimov	71161bfb93	Add mock libvirt objects	2016-07-13 23:28:02 +02:00
Bahtiar `kalkin-` Gadimov	d380fb4aba	qubes.tests.app make pylint happy ♥	2016-07-13 23:08:59 +02:00
Bahtiar `kalkin-` Gadimov	23e52775b6	qubes.tests.init make pylint happy ♥	2016-07-13 23:08:58 +02:00
Bahtiar `kalkin-` Gadimov	d8a90a77c2	Make pylint really happy ♥♥♥	2016-07-13 22:35:58 +02:00
Bahtiar `kalkin-` Gadimov	bcf1cfcb1f	Add qvm-clone(1)	2016-07-13 22:35:58 +02:00
Bahtiar `kalkin-` Gadimov	1467f1ede5	Storage add clone support	2016-07-13 22:35:57 +02:00
Bahtiar `kalkin-` Gadimov	496434d865	qvm-create uses new api - `-p` is now used for `--pool` instead of `--property` - Documented pool usage	2016-07-13 22:34:37 +02:00
Bahtiar `kalkin-` Gadimov	61feb0ced7	Migrate backup to new storage api	2016-07-13 22:34:37 +02:00
Bahtiar `kalkin-` Gadimov	bb8b58b04c	qubes.backup fix verify_files	2016-07-13 22:34:36 +02:00
Bahtiar `kalkin-` Gadimov	53ff88cd15	qvm-block add revert command	2016-07-13 22:34:36 +02:00
Bahtiar `kalkin-` Gadimov	9acd46bddb	qvm-block show if old revisions are available	2016-07-13 22:34:35 +02:00
Bahtiar `kalkin-` Gadimov	e07c4cc8e8	qvm-block use new storage API	2016-07-13 22:34:35 +02:00
Bahtiar `kalkin-` Gadimov	95fed1eb71	qubes.linux.kernel use new storage api	2016-07-13 22:34:34 +02:00
Bahtiar `kalkin-` Gadimov	f60ccb235d	qubes.storage.domain use new storage API	2016-07-13 22:34:34 +02:00
Bahtiar `kalkin-` Gadimov	d1c606b952	qubes.storage.file use new storage API	2016-07-13 22:34:33 +02:00
Bahtiar `kalkin-` Gadimov	1f735669bc	Migrate qubes.vm modules to new API	2016-07-13 22:34:33 +02:00
Bahtiar `kalkin-` Gadimov	ca9797bb6b	qubes.tests.int.basic use new storage API	2016-07-13 22:34:32 +02:00
Bahtiar `kalkin-` Gadimov	1cbabc79ff	qubes.vm.QubesVM use new storage api	2016-07-13 22:34:32 +02:00
Bahtiar `kalkin-` Gadimov	7e1563c88d	Add handling for old volume config	2016-07-13 22:34:31 +02:00
Bahtiar `kalkin-` Gadimov	1bccb146d8	Add qubes.storage.isodate() helper function	2016-07-13 22:34:31 +02:00
Bahtiar `kalkin-` Gadimov	7841e3f6c0	qubes.storage rework api	2016-07-13 22:34:30 +02:00
Bahtiar `kalkin-` Gadimov	3952cef556	QubesVM serialize bool values from XML	2016-07-13 22:21:32 +02:00
Bahtiar `kalkin-` Gadimov	4cc7b8d2a8	Fix qubes.tests.storage	2016-07-13 22:21:31 +02:00
Bahtiar `kalkin-` Gadimov	bba9b38e8e	Avoid libvirt access in qubes.vm.qubesvm.QubesVM	2016-07-13 22:21:30 +02:00
Bahtiar `kalkin-` Gadimov	774cbd499e	Add TestVM.is_halted() to fix failing tests	2016-07-13 21:57:12 +02:00
Bahtiar `kalkin-` Gadimov	6fad722554	Make pylint really happy ♥♥♥	2016-07-13 21:13:29 +02:00
Bahtiar `kalkin-` Gadimov	3b93fd612e	Make pylint really happy ♥♥♥	2016-07-13 20:58:45 +02:00
Bahtiar `kalkin-` Gadimov	de5487bf14	Dumb down qvm-remove - Move `vm.is_halted()` check in to VMCollection.__delitem__() - `vm.remove_from_disk()` will raise exception if is called on a running vm	2016-07-13 18:24:29 +02:00
Bahtiar `kalkin-` Gadimov	5115eef413	qvm-shutdown make pylint happy ♥	2016-07-13 18:09:32 +02:00
Bahtiar `kalkin-` Gadimov	63a1edc274	Fix qvm-remove undefines domain in libvirt - Return error if domain is not halted - Undefine the domain in libvirt, to avoid: "Domain already exists with uuid…"	2016-07-05 13:39:47 +02:00
Bahtiar `kalkin-` Gadimov	a66df9c82e	Fix bugs in qvm-shutdown --all - Exit the program if all domains are halted - Changed log.notice to log.info - Replace map with dictionary comprehensions (pylint)	2016-07-05 13:32:59 +02:00
Wojtek Porczyk	c899d1f960	qubes/tests: Show errors while loading external tests Now failure to load external tests shows in which entry point the error happened and a useful traceback. The traceback extends from the "try" statement down to the actual error line, but it does not include the frames above, ie. from the invocation to the load_tests routine. This is a limitation of Python itself and usually not a problem.	2016-07-04 16:51:35 +02:00
Wojtek Porczyk	1ff1ca37a1	Merge remote-tracking branch 'woju/pull/21/head' into core3-devel Conflicts: qubes/storage/__init__.py	2016-07-02 11:54:56 +02:00
Wojtek Porczyk	d28ab908cc	Network and storage fixes	2016-06-29 16:42:35 +02:00
Wojtek Porczyk	b53c572f1d	Merge remote-tracking branch 'woju/pull/23/head' into core3-devel	2016-06-28 15:19:57 +02:00
Wojtek Porczyk	c9accc258c	Merge remote-tracking branch 'woju/pull/15/head' into core3-devel Conflicts: qubes/storage/__init__.py qubes/storage/file.py qubes/vm/qubesvm.py	2016-06-28 15:15:00 +02:00
Bahtiar `kalkin-` Gadimov	e26932a81a	Fix bug in FilesPool.target_dir()	2016-06-23 14:23:34 +02:00
Bahtiar `kalkin-` Gadimov	7081bd433a	Make pylint happy	2016-06-23 14:23:34 +02:00
Bahtiar `kalkin-` Gadimov	0ab27d7426	qubes.vm.__init__ Remove unneeded time import	2016-06-23 14:23:33 +02:00
Bahtiar `kalkin-` Gadimov	b584d1a4bb	Remove QubesVM.is_netvm() and is_disposablevm()	2016-06-23 14:23:33 +02:00
Bahtiar `kalkin-` Gadimov	4a0b7585d1	Remove QubesVM.is_proxyvm()	2016-06-23 14:23:32 +02:00
Bahtiar `kalkin-` Gadimov	b0a7d0c283	Remove QubesVM.is_appvm()	2016-06-23 14:23:32 +02:00
Bahtiar `kalkin-` Gadimov	e08ca1ff57	Remove QubesVM.is_template()	2016-06-23 14:23:31 +02:00
Bahtiar `kalkin-` Gadimov	ba3b191702	QubesVM add docstrings & fix pylint errors	2016-06-23 14:23:31 +02:00
Bahtiar `kalkin-` Gadimov	a2668d81e3	QubesVM remove resize functions	2016-06-23 14:23:30 +02:00
Bahtiar `kalkin-` Gadimov	9132690eef	Add QubesVM.is_halted()	2016-06-23 14:23:24 +02:00
Bahtiar `kalkin-` Gadimov	4db84c42a6	Fix qubes.storage.file _remove_if_exists	2016-06-23 13:17:00 +02:00
Bahtiar `kalkin-` Gadimov	12745a4860	Qubes.add_pool() returns the added pool	2016-06-23 13:17:00 +02:00
Bahtiar `kalkin-` Gadimov	8fc3772017	Add Volume.__str__()	2016-06-23 13:16:59 +02:00
Bahtiar `kalkin-` Gadimov	3dd77719c1	Pool.create source_volume argument is optional	2016-06-23 13:16:59 +02:00
Bahtiar `kalkin-` Gadimov	b260c2cdd9	Fix pylint warnings	2016-06-23 13:16:10 +02:00
Bahtiar `kalkin-` Gadimov	bb2e6a2ad3	Add QubesVM.attached_volumes()	2016-06-23 13:16:09 +02:00
Bahtiar `kalkin-` Gadimov	b9ddc00741	Add qvm-block	2016-06-23 13:16:08 +02:00
Bahtiar `kalkin-` Gadimov	c487b5fe9b	Fix PoolsAction	2016-06-23 13:16:07 +02:00
Bahtiar `kalkin-` Gadimov	90c882610e	Storage attach volumes from other pools - Already attached volumes are ignored	2016-06-23 13:16:07 +02:00
Bahtiar `kalkin-` Gadimov	c4a506206c	Fix VmNameGroup formatting	2016-06-23 13:16:06 +02:00
Bahtiar `kalkin-` Gadimov	2f0df26606	Add subparser handling to parse_qubes_app	2016-06-23 13:16:06 +02:00
Bahtiar `kalkin-` Gadimov	4f0b17cb52	Add qubes.tools.RunningVmNameAction	2016-06-23 13:16:05 +02:00
Bahtiar `kalkin-` Gadimov	b5eb377490	Add VolumeAction for parsing POOL_NAME:VOLUME_ID	2016-06-23 13:16:05 +02:00
Bahtiar `kalkin-` Gadimov	0ac40ddd4f	Volume.__repr__ returns POOL_NAME:VOLUME_ID	2016-06-23 13:16:04 +02:00
Bahtiar `kalkin-` Gadimov	09727b1cbd	Add Volume.internal field - Linux kernel volumes are always internal volumes	2016-06-23 13:16:03 +02:00
Bahtiar `kalkin-` Gadimov	07800a0e67	Add Volume.eq, neq and hash	2016-06-23 13:16:03 +02:00
Bahtiar `kalkin-` Gadimov	f08ce2cb79	A Pool should always have a volumes property NOTE: FilesPool need some way to dynamically discover volumes	2016-06-23 13:15:54 +02:00
Bahtiar `kalkin-` Gadimov	88198fb7ac	Storage verification is part of the pool interface - LinuxKernel pool add verify method - FilePool implent verification	2016-06-23 13:11:59 +02:00
Bahtiar `kalkin-` Gadimov	87ae5a3b65	Fix qvm-start	2016-06-21 17:44:35 +02:00
Bahtiar `kalkin-` Gadimov	d56f02598a	Replace QubesVM.verify_files() with Storage.verify_files()	2016-06-21 14:58:12 +02:00
Bahtiar `kalkin-` Gadimov	296fa17322	QubesVM remove obsolete *_img methods	2016-06-21 14:58:12 +02:00
Bahtiar `kalkin-` Gadimov	e7732f8ad0	Storage.verify fires domain-verify-files event	2016-06-21 14:58:11 +02:00
Bahtiar `kalkin-` Gadimov	457c915d03	qvm-create fix access to volume paths	2016-06-21 14:58:11 +02:00
Bahtiar `kalkin-` Gadimov	732d2b33f4	Add LinuxKernel.is_outdated()	2016-06-21 14:58:10 +02:00
Bahtiar `kalkin-` Gadimov	3b441ebaad	Add Storage.outdated_volumes and Pool.is_outdated	2016-06-21 14:58:10 +02:00
Bahtiar `kalkin-` Gadimov	0a471e719b	qubes.backup fix access to volume paths	2016-06-21 14:58:09 +02:00
Bahtiar `kalkin-` Gadimov	db608f6e01	Pool add str, eq & neq	2016-06-21 12:53:31 +02:00
Bahtiar `kalkin-` Gadimov	803efa76ff	Merge FilePool._resize_loop_device() with resize()	2016-06-21 12:48:57 +02:00
Bahtiar `kalkin-` Gadimov	72df863bb9	Fix qubes.storage.file pylint warnings	2016-06-21 12:47:47 +02:00
Bahtiar `kalkin-` Gadimov	20282c17fe	Volume add docstrings	2016-06-21 12:44:21 +02:00
Bahtiar `kalkin-` Gadimov	b830cb5544	Volume add eq, neq & hash methods	2016-06-21 12:39:47 +02:00
Wojtek Porczyk	66f46d8a25	Fix test error after `bdfb85ac` (refactor storage)	2016-06-21 10:10:45 +02:00
Wojtek Porczyk	d6ad8d34a6	Merge remote-tracking branch 'marmarek/patch-1' into core3-devel	2016-06-16 21:46:53 +02:00
Marek Marczykowski-Górecki	dcdb62721b	tests: mark TC_01_FileVolumes.test_003_read_volume with expected failure	2016-06-16 21:23:18 +02:00
Marek Marczykowski-Górecki	4e797663e9	tests: make storage tests working outside of dom0	2016-06-16 21:14:35 +02:00
Marek Marczykowski-Górecki	2ff6aa456e	Provide fake CPUs count and total memory in offline mode	2016-06-16 21:08:44 +02:00
Marek Marczykowski-Górecki	5eea473725	core2migration: add a skeleton for dispvm_netvm migration QubesOS/qubes-issues#2075	2016-06-16 17:25:45 +02:00
Bahtiar `kalkin-` Gadimov	29633f43f8	qubes.vm.templatevm Add LICENSE & docstring	2016-06-16 17:11:49 +02:00
Marek Marczykowski-Górecki	9cdf994360	Minor fixes	2016-06-16 17:06:42 +02:00
Marek Marczykowski-Górecki	98effef606	backup: add option to use default kernel for restored VMs	2016-06-16 17:06:42 +02:00
Marek Marczykowski-Górecki	91404cc647	backup: collect files to backup once	2016-06-16 17:06:41 +02:00
Marek Marczykowski-Górecki	2dacb3a542	backup: drop/resolve minor "TODO" comments	2016-06-16 17:06:41 +02:00
Bahtiar `kalkin-` Gadimov	91ee455a37	NetVMMixin add docstrings & fix pylint errors	2016-06-16 17:06:19 +02:00
Marek Marczykowski-Górecki	c9a55cc198	tests: use offline mode QubesOS/qubes-issues#2008	2016-06-16 17:04:16 +02:00
Bahtiar `kalkin-` Gadimov	b77c36b224	Remove NetVMMixin.vif()	2016-06-16 17:01:34 +02:00
Bahtiar `kalkin-` Gadimov	7ccba17a38	qubes.vm.__init__ Replace TODOs with SEE: #1815	2016-06-16 17:01:34 +02:00
Bahtiar `kalkin-` Gadimov	ec9550c7cc	qubes.vm.__init__ Remove debug xml file creation	2016-06-16 17:01:34 +02:00
Bahtiar `kalkin-` Gadimov	94d9fd040f	qubes.vm.__init__ Add docstrings	2016-06-16 17:01:34 +02:00
Bahtiar `kalkin-` Gadimov	77d54b55ad	qubes.vm.__init__ Fix pylint warnings	2016-06-16 17:01:34 +02:00
Bahtiar `kalkin-` Gadimov	f318871279	qubes.vm.__init__ Remove uses_custom_config bits	2016-06-16 17:01:34 +02:00
Bahtiar `kalkin-` Gadimov	b5fe49f422	qubes.vm.__init__ Remove unneeded imports	2016-06-16 17:01:34 +02:00
Bahtiar `kalkin-` Gadimov	93e68a6796	qubes.vm.dispvm Fix XXX, add docstring & LICENSE	2016-06-16 17:01:33 +02:00
Bahtiar `kalkin-` Gadimov	d25c44ca71	qubes.vm.appvm Fix XXX, add docstrings & LICENSE	2016-06-16 17:01:33 +02:00
Bahtiar `kalkin-` Gadimov	0cd667a1f5	qubes.vm.adminvm fix XXX and add docstrings	2016-06-16 17:01:33 +02:00
Bahtiar `kalkin-` Gadimov	e5906df0c7	AdminVM remove get_private_img_sz	2016-06-16 16:59:27 +02:00
Bahtiar `kalkin-` Gadimov	9f3385296b	qubes.storage.file fix pylint warnings	2016-06-16 16:59:27 +02:00
Bahtiar `kalkin-` Gadimov	38dc74587d	Rewrote file rename_target_dir	2016-06-16 16:59:27 +02:00
Bahtiar `kalkin-` Gadimov	a4577c0dce	Rename Storage.create_on_disk to Storage.create	2016-06-16 16:59:27 +02:00
Bahtiar `kalkin-` Gadimov	930fe417a8	Remove storage size and usage methods from QubesVM	2016-06-16 16:59:26 +02:00
Wojtek Porczyk	e47043ebd7	More pylint fixes	2016-06-16 13:29:16 +02:00
Wojtek Porczyk	3cb5f031a9	Fix storage initialisation	2016-06-16 13:29:15 +02:00
Wojtek Porczyk	1f302fb776	Fix "pylint fixes" wrt dir_path	2016-06-16 13:29:15 +02:00
Wojtek Porczyk	677a3e51f4	Move libvirt XML network device to jinja	2016-06-16 13:29:15 +02:00
Wojtek Porczyk	ba20254888	Rewrite PCI attaching/detaching from xl to libvirt The only remaining part is querying vm-side BDF. That can't be done in libvirt.	2016-06-16 13:29:15 +02:00
Wojtek Porczyk	63b6674fbd	fix qvm-ls display of cpu_time QubesOS/qubes-core-admin#27	2016-06-13 19:10:01 +02:00
Wojtek Porczyk	6a10daf7be	Merge branch 'master' into core3-devel Remains to be fixed: `88cb62fc` `d2640b51` `958c2926`	2016-06-13 19:03:46 +02:00
Wojtek Porczyk	6ade5736d7	pylint fixes	2016-06-10 21:27:29 +02:00
Wojtek Porczyk	6895f34a7f	qubes/vm/qubesvm: change type of exception	2016-06-10 21:27:29 +02:00
Wojtek Porczyk	3b08e85a2d	qubes/tests/init: Fix VMCollection reference	2016-06-10 21:27:29 +02:00
Wojtek Porczyk	5a76d0b03b	qubes/vm/dispvm: Add methods for creating and destroying fixes QubesOS/qubes-issues#866	2016-06-02 19:55:42 +02:00
Wojtek Porczyk	a719e0d93d	qubes/vm/qubesvm: allow looking up by UUID Also, allow lookup using VM from other app. QubesOS/qubes-issues#866	2016-06-02 17:18:33 +02:00
Wojtek Porczyk	476b681749	qubes/vm/qubesvm: return self from methods modifying state This is helpful when writing oneliners. QubesOS/qubes-issues#866	2016-06-02 17:17:05 +02:00
Wojtek Porczyk	27d0e11872	qubes/tests/vm: fix tests	2016-06-02 15:46:01 +02:00
Wojtek Porczyk	a615a45ecd	Merge remote-tracking branch 'woju/pull/13/head' into core3-devel	2016-06-02 13:24:15 +02:00
Marek Marczykowski-Górecki	7e0af81ecc	qubes/vm: fix PCI device detach	2016-06-02 13:22:48 +02:00
Marek Marczykowski-Górecki	485e75091b	tests: fix checking rename	2016-06-02 13:22:43 +02:00
Marek Marczykowski-Górecki	077dec8d2e	tests: fix rename test	2016-06-02 13:22:40 +02:00
Marek Marczykowski-Górecki	bb2e1f2870	qubes/vm: handle VM toplevel directory in QubesVM object not Storage This directory is not only for disk images (in fact disk images may be elsewhere depending on choosen volume pool), so it would be cleaner to handle (create/remove) it directly in QubesVM class.	2016-06-02 13:22:07 +02:00
Marek Marczykowski-Górecki	ed6e69b77e	qubes/vm: minor	2016-06-02 13:14:19 +02:00
Marek Marczykowski-Górecki	2d8d78bebe	qubes/vm/standalonevm: set volume_config Otherwise VM wouldn't have any hard disk...	2016-06-02 13:14:02 +02:00
Marek Marczykowski-Górecki	5b5f290c23	qubes/vm: fix setting autostart property This is actually workaround for systemd bug reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1181922 qubesos/qubes-issues#925 This is migration of `9bfcb72722` commit to core3.	2016-06-02 13:13:21 +02:00
Marek Marczykowski-Górecki	b37bf55f5e	qubes/vm: fix handling rename Libvirt VM config is no longer named after VM.	2016-06-02 13:10:56 +02:00
Marek Marczykowski-Górecki	5e2b617c6f	qubes/vm: fix parameters for some even handlers	2016-06-02 13:10:43 +02:00
Marek Marczykowski-Górecki	2bb73ab0a1	qubes/vm: add validators for many properties	2016-06-02 13:10:28 +02:00
Marek Marczykowski-Górecki	36644f3710	qubes/vm: initialize vm.volumes in one place Move it to QubesVM, instead of each class separately.	2016-06-02 13:10:02 +02:00
Marek Marczykowski-Górecki	28591fa6be	tests: remove/skip some tests 1. It is unclear yet whether dispvm_netvm will be implemented in core3, but probably not. 2. Remove tests for setting memory/cpu above host resouces - rejecting those values at property set time would break backup restore on some machines (when migrating from bigger to smaller system).	2016-06-02 13:09:39 +02:00
Marek Marczykowski-Górecki	bf0966bfcf	tests: fix get_label mockup to return meaningful values Reject invalid argument.	2016-06-02 13:08:48 +02:00
Marek Marczykowski-Górecki	98115eb541	tests/backup: fix to core3 API - there is no verbose= argument	2016-06-02 13:08:03 +02:00
Marek Marczykowski-Górecki	0a21300872	qubes/storage: unify _remove_if_exists argument type In some places full volume object was called, in others - just file path. Since this function is also used in some volume init/teardown, use path everywhere.	2016-06-02 13:07:35 +02:00
Marek Marczykowski-Górecki	574834ac68	qubes/core2migration: rework load order To successfully load all the data, proceed in order: - set app.default_kernel - load all templates - set app.default_template - load other VMs - update network dependencies between VMs - set other global properties	2016-06-02 12:44:31 +02:00
Marek Marczykowski-Górecki	554a99610f	qubes/core2migration: use app.load_initial_values Reduce code duplication.	2016-06-02 12:43:25 +02:00
Marek Marczykowski-Górecki	692f75353e	qubes/app: split initialising empty Qubes object from saving it It may be useful to create fresh Qubes() object but not save it yet before setting other things. One such case is migration from core2.	2016-06-02 12:42:37 +02:00
Marek Marczykowski-Górecki	2909d252d6	backup: get_fisk_usage is now in storage.file	2016-06-02 12:41:59 +02:00
Marek Marczykowski-Górecki	ff78b26f66	qubes: implement offline mode Apparently the most important (the only?) property required in offline mode is "is_running". So let's patch it to return False and make sure any other libvirt usage would result in failure. Or maybe better simply returh False in vm.is_running, when libvirt connection fails? But then it would not be possible to use offline mode and have (some, probably unrelated) libvirtd running at the same time. Fixes QubesOS/qubes-issues#2008	2016-06-02 12:41:26 +02:00
Marek Marczykowski-Górecki	b119b2c36b	storage/file: simplify search for origin volume of snapshot/read-only Check directly vm.template, throwing AttributeError when not found. There may be some value in converting it to more descriptive error, but since that's mostly for internal users (not user facing actions) don't bother for now. QubesOS/qubes-issues#1842	2016-06-02 12:38:07 +02:00
Marek Marczykowski-Górecki	07b72ef3fd	tests: fix after moving VMCollection to qubes.app	2016-06-02 12:37:30 +02:00
Marek Marczykowski-Górecki	c965024287	qubes/vm: Implement Disposable VM Implement DispVM as a VM based on AppVM. QubesOS/qubes-issues#866	2016-06-02 12:37:19 +02:00
Marek Marczykowski-Górecki	54c70766a4	qubes/storage: allow snapshots of snapshots It may make sense to create 'snapshot' volume out of already 'snapshot', not only 'origin'. In pracice it will exactly the same as 'snapshot connected directly to 'origin'. QubesOS/qubes-issues#866	2016-06-02 12:37:00 +02:00
Marek Marczykowski-Górecki	48176d51f1	qubes/storage: minor error message fix	2016-06-02 12:29:39 +02:00
Marek Marczykowski-Górecki	d80f34888d	qubes/storage: fix getting origin pool volume_config parameter is about target volume, not origin. QubesOS/qubes-issues#1842	2016-06-02 12:29:30 +02:00
Marek Marczykowski-Górecki	b24ab45d00	qubes/vm: fix network attach/detach	2016-06-02 12:28:53 +02:00
Marek Marczykowski-Górecki	3b407eb79e	qubes/app: do not enable domain events when loading XML Event should be enabled only after all the domains are loaded (and domain-load event if fired). QubesOS/qubes-issues#1816	2016-06-02 12:28:14 +02:00
Bahtiar `kalkin-` Gadimov	17790c32bb	Fix DomainPool missing a continue	2016-06-01 17:28:55 +02:00
Bahtiar `kalkin-` Gadimov	90928dc4a0	qubes-pool skip DomainPools without volumes	2016-05-30 13:31:58 +02:00
Bahtiar `kalkin-` Gadimov	c8363cfc95	qvm-pool simplify options parsing	2016-05-30 13:31:57 +02:00
Bahtiar `kalkin-` Gadimov	27305dd85d	qvm-pool actions remove access to app	2016-05-30 13:31:56 +02:00
Bahtiar `kalkin-` Gadimov	ed1a6977c0	Qubes.add_pool() add name parameter	2016-05-30 13:31:55 +02:00
Bahtiar `kalkin-` Gadimov	d703f2f44b	Add qvm-pool and manpage for it - Use full import paths in qvm-pool - Add, Remove, Info and List options set `Namespace.command`. This fixes a crash when `-o dir_path=/mnt/foo` is specified after `-a foo xen`. - Remove `_List` - Remove 'added pool' and 'removed pool' messages. Unix tools are quiet - qvm-pool call app.save() - Rename create_parser in get_parser - Rename local_parser variables to just parser - qvm-pool uses print_table	2016-05-30 13:31:54 +02:00
Bahtiar `kalkin-` Gadimov	9ef9575d4e	Fix typo in "No driver FOO for pool BAR" message	2016-05-30 13:31:54 +02:00
Bahtiar `kalkin-` Gadimov	357e8125eb	Add qubes.tools.PoolsAction	2016-05-30 13:31:53 +02:00
Bahtiar `kalkin-` Gadimov	0319df25e5	Add print_table function to qubes.tools - print_table uses the `column` tool with the ASCII Unit Separator to print a pretty table	2016-05-30 13:31:51 +02:00
Bahtiar `kalkin-` Gadimov	35974a5dbf	DomainPool check the untrusted data from qubes-db	2016-05-25 17:39:34 +02:00
Wojtek Porczyk	0484be518c	Merge remote-tracking branch 'woju/pull/12/head' into core3-devel Conflicts: doc/manpages/qvm-kill.rst	2016-05-25 11:01:19 +02:00
Bahtiar `kalkin-` Gadimov	3f5a92772a	A QubesVM always has an empty DomainPool - A DomainPool is initialized by QubesVM after Storage initialization on a `domain-load` event	2016-05-22 22:09:56 +02:00
Bahtiar `kalkin-` Gadimov	ddf040ae64	Do not serialize the domain pool config	2016-05-22 22:09:55 +02:00
Bahtiar `kalkin-` Gadimov	baaac858bc	Add DomainPool - All domain pool volumes are removable volumes - DomainVolume uses device name as vid	2016-05-22 22:09:54 +02:00
Bahtiar `kalkin-` Gadimov	e30f894df9	Add Volume.removable field	2016-05-22 21:42:27 +02:00
Bahtiar `kalkin-` Gadimov	116ba64e51	Storage.remove() catch IO/OSError and log it	2016-05-21 01:35:32 +02:00
Bahtiar `kalkin-` Gadimov	c5810758c5	FilePool fix origin volume removale	2016-05-21 01:35:31 +02:00
Bahtiar `kalkin-` Gadimov	8959e5a77e	Implement qvm-remove - Remove old qvm-remove - Remove a log line from Storage, because it prints confusing lines, like: Removing volume kernel: /var/lib/qubes/vm-kernels/4.1.13-6/modules.img	2016-05-21 01:35:30 +02:00
Bahtiar `kalkin-` Gadimov	91f72dc56c	Rework argument checking when generating manpages Add the ability to handle commands having subcommands, like `qvm-block` Split the ArgumentCheckVisitor in an OptionsCheckVisitor & SubCommandCheckVisitor. The OptionsCheckVisitor checks options given in a section named 'Options' (case insensitive), while the SubCommandCheckVisitor dispatches on a section named 'Commands' (case insensitive). This also fixes finding the undocumented command arguments. The previous solution with depart_document did not work. NodeVisitor does not dispatch to depart_document() even if it's mentioned in the documentation.	2016-05-21 01:25:14 +02:00
Bahtiar `kalkin-` Gadimov	e580131465	Add AliasedSubParsersAction	2016-05-21 01:25:13 +02:00
Bahtiar `kalkin-` Gadimov	910276e898	Rename want_vm_* in vmname_nargs	2016-05-21 01:24:17 +02:00
Bahtiar `kalkin-` Gadimov	d4c74d210f	Implement vmname parsing	2016-05-21 01:24:16 +02:00
Bahtiar `kalkin-` Gadimov	3549a9d4ec	Add VmNameGroup	2016-05-21 01:24:14 +02:00
Bahtiar `kalkin-` Gadimov	7fe827d858	Add VmNameAction	2016-05-21 01:24:13 +02:00
Bahtiar `kalkin-` Gadimov	c22d9e88c9	Add QubesAction	2016-05-21 01:24:12 +02:00
Wojtek Porczyk	e757444c35	qubes/tools/qvm-features: add tool for managing qvm-features QubesOS/qubes-issues#1637	2016-05-19 03:02:24 +02:00
Bahtiar `kalkin-` Gadimov	a65b0edcd4	Add QubesArgumentParser.print_error()	2016-05-19 03:02:24 +02:00
Wojtek Porczyk	786884ad7a	qubes: fix netvm properties and tests fixes QubesOS/qubes-issues#1816	2016-05-19 03:02:23 +02:00
Wojtek Porczyk	d728f4b9ff	qubes/app: reconnect to libvirtd after crash Sometimes libvirt crashes. After that the connection (and all vm.libvirt_domain-s) were unusable. fixes QubesOS/qubes-issues#990	2016-05-05 17:33:00 +02:00
Wojtek Porczyk	e3aae7bc17	qubes: Minor fixes Fix mock TestApp object and argument parsing in qvm-run.	2016-05-05 16:58:43 +02:00
Wojtek Porczyk	c76790fbde	qubes/tests/vm/qubesvm: fix test for vm.internal On flipping .internal vm (re)creates appmenus. They need label.	2016-05-05 14:38:04 +02:00
Wojtek Porczyk	63c09a090c	qubes: Combat import cycles This commit eliminates import statements happening in the middle of the file (between two classes definition). The cycles are still there. The only magic module is qubes itself.	2016-05-05 14:33:09 +02:00
Wojtek Porczyk	f8270a07bb	qubes: add filename= argument to Qubes.save() fixes QubesOS/qubes-issues#1846	2016-05-05 00:40:45 +02:00
Bahtiar `kalkin-` Gadimov	1d5b89f0d5	LinuxKernel keep track of own volumes This is squashed woju/qubes-core-admin#8 by @kalkin - LinuxKernel.volumes() lists all available kernels - LinuxKernel use kernel version as vid - LinuxKernel add docstrings - Linux.kernel use os.listdir instead of os.walk - LinuxKernel dynamically list available kernels	2016-05-04 17:21:05 +02:00
Bahtiar `kalkin-` Gadimov	7200e6153b	Rename default storage driver from xen to file - Rename XenPool ⇒ FilePool - Rename XenVolume ⇒ FileVolume	2016-04-30 20:42:46 +02:00
Bahtiar `kalkin-` Gadimov	b2c1017488	Fix missing parameter in Pool.__init__()	2016-04-30 20:42:00 +02:00
Wojtek Porczyk	7f2f4a4e75	Fix GUI extension	2016-04-27 15:27:01 +02:00
Bahtiar `kalkin-` Gadimov	04a3e80311	SizeMixIn first assert than call super()	2016-04-25 07:17:21 +02:00
Bahtiar `kalkin-` Gadimov	29f4be0f10	If vm doesnt support volume_config raise TypeError	2016-04-25 07:17:21 +02:00
Bahtiar `kalkin-` Gadimov	37ca33b0d1	Add docstring to xen volumes implementations	2016-04-25 07:17:21 +02:00
Bahtiar `kalkin-` Gadimov	8f060a8746	Fix Pool and Volume __init__	2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov	591134833b	Replace Volume.__str__ with enhanced __repr__	2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov	04536c5950	Don't exec app.save() after add_pool & remove_pool	2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov	d7ff4b9057	Move volume xml config from QubesVM to Volume	2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov	2e28849c90	Move pool xml config from Qubes to Pool	2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov	49b4951389	Storage move rename() logic to XenPool - Fix config renaming	2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov	e3ae6cdc1b	BackupTestsMixin.create_backup_vms uses volumes Instead of using root_img to access the path it uses now the proper volumes	2016-04-25 07:17:20 +02:00
Bahtiar `kalkin-` Gadimov	d7fd66070a	Fix revert template changes test	2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov	8cc31e86a7	qvm-create handle --pool argument	2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov	2c2a778a1d	Serialize volume_config from qubes.xml	2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov	fe6a35155e	Move kernel file checks to LinuxKernel pool	2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov	ef485ca32a	Add linux-kernel to defaults['pool_config']	2016-04-25 07:17:19 +02:00
Bahtiar `kalkin-` Gadimov	62c81044c5	Add XenPool.setup/destroy	2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov	97d04791b7	After add/remove_pool execute Pool.setup/destroy	2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov	9674d03088	Add pool LinuxKernel	2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov	d1a0542c85	Add XenPool.remove()	2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov	a37fc2464a	Add XenPool.config()	2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov	971c4ae91d	Add XenPool.driver field	2016-04-25 07:17:18 +02:00
Bahtiar `kalkin-` Gadimov	5f7cb41a21	Move Storage.clone_disk_files logic to XenPool - Add XenVolume to identify volumes which can be cloned even if they are not in the same pool	2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov	973c83cedd	Move most resize logic to XenPool	2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov	bdfb85ac19	Refactor Storage, Pool and XenPool - Remove all *_dev_config methods - Checks if a storage image exists moved to XenPool - Storage.remove wraps Pool.remove() - Stop volumes on domain sutdown/kill - Warn when using deprecated methods	2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov	3c66d4b54c	Fix storage_xen test	2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov	79ac3d3770	Fix storage test and simplify TestVM	2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov	88238c80f3	Add XenPool._reset_volume	2016-04-25 07:17:17 +02:00
Bahtiar `kalkin-` Gadimov	3dab5193c6	XenPool add snapshot handling	2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov	792d94959f	Add implementations of xen volumes	2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov	f02f9e3a41	Add XenPool init_volume	2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov	4d4b846ce8	Replace XenStorage with XenPool	2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov	32255a7916	Reverted Storage ←→ Pool dependency - Storage() operates on a pool and in future on multiple pools	2016-04-25 07:17:16 +02:00
Bahtiar `kalkin-` Gadimov	9d646aabd3	Add volume_config to AppVM and TemplateVM	2016-04-25 07:17:13 +02:00
Bahtiar `kalkin-` Gadimov	24193c4308	Add Volume class	2016-04-25 07:16:38 +02:00
Bahtiar `kalkin-` Gadimov	428dd5bc1b	QubesVM.dir_path is set independent of storage	2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov	c3d8c899cc	Add TemplateVM test for storage_xen	2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov	3c798bc825	Pool configuration include the pool name	2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov	bd4674b658	Remove obsolete tests	2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov	22d73e8fa9	Fix Pool dir_path normalization	2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov	36470310a2	Replace pool config parsing logic - Move add_pool/remove_pool to Qubes class - Add Qubes.get_pool - Remove storage.conf	2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov	c791cb1935	Serialize pool configuration to XML	2016-04-25 07:16:37 +02:00
Bahtiar `kalkin-` Gadimov	47e89d84b6	XenStorage.format_disk_dev returns now BlockDevice	2016-04-25 07:16:36 +02:00
Bahtiar `kalkin-` Gadimov	cc7dd625d9	Loop over QubesVM.block_devices in libvirt xml	2016-04-25 07:16:36 +02:00
Bahtiar `kalkin-` Gadimov	2f99efa4b8	Add BlockDevice	2016-04-25 07:16:36 +02:00
Wojtek Porczyk	62da967f78	Fix generating documentation	2016-04-22 15:23:39 +02:00
Bahtiar `kalkin-` Gadimov	ebb79e9c4f	Fix TC_01_Properties/test_030_rename_conflict_app	2016-04-20 14:02:04 +02:00
Bahtiar `kalkin-` Gadimov	d1d72466d6	Remove unneeded imports from tests/int.basic	2016-04-20 13:59:55 +02:00
Bahtiar `kalkin-` Gadimov	f74646fb1b	Assert that a TemplateVM can't have a template	2016-04-20 13:59:32 +02:00
Bahtiar `kalkin-` Gadimov	6aac0a5732	On startup-failure only force_shutdown if running This avoids losing the exception if an exception is raised in self.force_shutdown(), because the vm is not running or paused	2016-04-20 13:59:17 +02:00
Wojtek Porczyk	2a9752716c	Merge remote-tracking branch 'marmarek/core3-devel-mm' into core3-devel	2016-04-20 13:55:46 +02:00
Wojtek Porczyk	fe8fdb264b	qubes/vm/qubesvm: move is_guid_running to GUI extension	2016-04-20 13:54:56 +02:00
Wojtek Porczyk	adbca5c0f6	qubes/tools/qvm_run: fix --all/--passio exclusion	2016-04-20 13:54:56 +02:00
Wojtek Porczyk	e720e1634b	qubes/tools: rewrite qvm-shutdown fixes QubesOS/qubes-issues#1228	2016-04-20 13:52:58 +02:00
Wojtek Porczyk	94589915b0	Fix documentation	2016-04-20 13:50:42 +02:00
Marek Marczykowski-Górecki	2a46abbefd	qubes.vm: implement TemplateVM.appvms property	2016-04-20 02:28:11 +02:00
Marek Marczykowski-Górecki	7117847816	tests: fix int.dom0_update Environment must be preserved for QUBES_XML_PATH variable used to point to a secondary (test) qubes.xml.	2016-04-20 02:26:56 +02:00
Marek Marczykowski-Górecki	a707840596	tests: fix and enable external tests loader QubesOS/qubes-issues#1800	2016-04-20 02:26:16 +02:00
Marek Marczykowski-Górecki	a1575b98d4	tests: do not drop 'qubes.tests.' prefix This cause troubles with "external" tests (loaded from outside of qubes.tests module).	2016-04-20 02:25:56 +02:00
Wojtek Porczyk	540942de47	qubes/ext: convert extensions to singletons From now the extensions are instantiated once. They no longer have .app attribute, but can access it from event handlers via vm.app.	2016-04-11 15:31:41 +02:00
Marek Marczykowski-Górecki	eaf5efd814	tests: convert 'extra' test loader to core3 API QubesOS/qubes-issues#1800	2016-04-11 13:50:19 +02:00
Marek Marczykowski-Górecki	8c6fe7ed90	Merge remote-tracking branch 'origin/master' into core3-devel-mm	2016-04-11 13:03:12 +02:00
Marek Marczykowski-Górecki	4308fa7946	Fix events names in docstrings	2016-04-11 00:13:14 +02:00
Marek Marczykowski-Górecki	d9a8cd5d68	backup: minor code style fixes	2016-04-11 00:13:13 +02:00
Wojtek Porczyk	c5962910d0	qubes/events: Rework firing events for effect From now, the handlers should yield their values, not return.	2016-04-08 12:35:11 +02:00
Marek Marczykowski-Górecki	c28f50f6fe	backup: minor fixes	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	2261e6a4c7	backup: simplify auto-renaming handling Don't try to track names in restore_info_verify, instead resolve them at actual restore time.	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	71a7730168	backup: simplify ordering of restoring VMs No other change than reducing two nested loops into one.	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	20d53fbf69	tests: more VMs combinations in backup VM with custom template, non-default NetVM This requires to VMs being removed in reversed order (to first remove VM, then its template).	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	98c8b7cd22	backup: use simple classes for data storage on restore too	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	424d3054f3	backup: use vm.features to store backup-specific metadata Do not keep them in system qubes.xml.	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	ed88b5a9a3	backup: use vm.run_service instead of vm.run This should be the only way used to call services.	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	e8f21929ad	backup: by default include all the VMs with vm.include_in_backups If there is a need for more robust default, it should be handled at that property.	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	a3051b956b	backup: docstrings, don't expose internal attributes processes_to_kill_on_cancel shouldn't be initialized by user, so move below applying user settings.	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	b4227f7581	backup: use simple classes instead of hashes, use events for appmenus This makes typo errors much easier to find (also using pylint or so). While at it, also removed explicit appmenus backup, as it should be provided by appmenus extension.	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	c01f43a5d3	backup: unify handling of template and non-template VMs Handle them the same way - individual files, not the whole directory for templates. Also don't backup obsolete 'kernels' subdir - it isn't supported in core3.	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	d9cf64a41d	backup: minor code structure changes - initialize internal objects object in constructor - use 'with'	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	78dbadab57	backup: minor: compile regex once	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	e9b8b6428c	backup: remove duplicated get_disk_usage function	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	c08f5986a9	backup: use constants for queue control strings	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	ad1f1738fa	qubes: make 'store' property public (R/O) It makes sense to see from what Qubes object was loaded.	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	c55ca8004a	tests: port backupcompatibility to core3 API	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	8c5d42a095	backup: add support for restoring pre-core3 backups	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	304d2b10bc	backup: fix handline "None" kernel	2016-04-07 13:21:00 +02:00
Marek Marczykowski-Górecki	412bcfa88b	backup: fix hadling netvm property	2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki	d72d8f8445	backup: code inspection annotations	2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki	3fb0754398	backup: formatting	2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki	697a26c8da	backup: update license header Use link instead of address, as recommended by FSF.	2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki	e04ea7512c	backup: minor code style - Don't use catch-all except statement. - Use str.format instead of "%" operator. - Use static methods where applicable. - Remove unused local variables. - Don't shadow variables from outer scope	2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki	44eed25511	tests/backup: report failure when error was expected but none detected	2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki	e5af64f2bf	backup: fix restore summary table	2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki	595dfdc0a9	backup: add additional verification of backup header Ensure only alphanumeric characters are used.	2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki	3342f637f4	backup: convert logging to proper python logging API QubesOS/qubes-issues#1213 QubesOS/qubes-issues#1214	2016-04-07 13:20:59 +02:00
Marek Marczykowski-Górecki	019678bf4d	tests: fix backup test to not break default template... By default AppVM is template based. This means vm.root_img points at default template's root image. Change this to StandaloneVM to have independent root.img.	2016-04-07 13:06:25 +02:00
Marek Marczykowski-Górecki	e557fe9989	backup: restructude the code into classes Introduce two main classes Backup and BackupRestore for storing the state of the desired operation. Then a simple interface to adjust parameters. (Almost) no functional change. QubesOS/qubes-issues#1213 QubesOS/qubes-issues#1214	2016-04-07 13:06:14 +02:00
Marek Marczykowski-Górecki	fd1f8def6a	tests: do not store dom0 home backup inside of dom0 home itself	2016-04-07 13:05:30 +02:00
Marek Marczykowski-Górecki	e0686e1e02	backup: initial conversion to core3 API	2016-04-07 13:05:25 +02:00
Marek Marczykowski-Górecki	45d6ab3862	qubes/vm: store libvirt config in libvirt.xml, fix rename handling	2016-04-07 13:03:46 +02:00
Marek Marczykowski-Górecki	12df07edd0	tests: invalid property should not be saved to qubes.xml	2016-04-07 13:03:42 +02:00
Marek Marczykowski-Górecki	de86a3c1a4	tools/qvm_prefs: reject invalid properties instead of ignoring silently	2016-04-07 13:03:39 +02:00
Marek Marczykowski-Górecki	c07981da88	tests: allow real prefix notation for selecting tests	2016-04-07 13:03:35 +02:00
Marek Marczykowski-Górecki	7f5feac98b	qubes/vm: fixes for features handling Properly handle 'None' value, there is no difference between empty string (<x></x>) and no content at all (<x/>).	2016-04-07 13:03:30 +02:00
Marek Marczykowski-Górecki	06c315c94b	qubes: fix/update event names Fix few typos, then rename: - recreate-appmenus -> domain-restore: there may be more to do than only appmenus	2016-04-07 12:59:39 +02:00
Marek Marczykowski-Górecki	7cbe2dbd38	qubes/vm: add passio_stderr argument to vm.run_service This allows getting error messages from service call, to provide more meaningful error messages.	2016-04-07 12:58:05 +02:00
Marek Marczykowski-Górecki	1cd6acd875	core/events: allow handlers return some value	2016-04-07 12:58:01 +02:00
Marek Marczykowski-Górecki	6052143735	tests: qvm-prefs QubesOS/qubes-issues#1248	2016-04-07 12:57:57 +02:00
Marek Marczykowski-Górecki	e319639146	qubes/vm: add StandaloneVM class	2016-04-07 12:57:52 +02:00
Marek Marczykowski-Górecki	43435405e7	tests: update checking 'None' VM representation	2016-04-07 12:57:47 +02:00
Marek Marczykowski-Górecki	c17f86e659	qubes: fix handling DEFAULT special value for VMProperty	2016-04-07 12:03:40 +02:00
Wojtek Porczyk	267a78e4e0	Fix resetting attribute to default when it's already default Based on marmarek's WIP fix resetting attribute to default when it's already default	2016-04-06 18:09:57 +02:00
Bahtiar `kalkin-` Gadimov	b1978abce5	Use entry_points for pool driver discovery - Add qubes.storage entry point to setup.py - Removed the old pool driver class loading logic - Reworked pool tests	2016-04-06 10:11:19 +02:00
Wojtek Porczyk	42666e0ec5	Throw meaningful exception when qubes.xml is missing Based on kalkin-'s Rais QubesException if qubes xml config is missing	2016-04-05 16:33:49 +02:00
Bahtiar `kalkin-` Gadimov	156ecf4163	Add missing import in qubes/__init__.py This was: Fix qubes-create command	2016-04-05 16:25:51 +02:00
Bahtiar `kalkin-` Gadimov	49ec29a235	Fix bug in Qubes.get_vm_class()	2016-04-05 16:24:30 +02:00
Wojtek Porczyk	c375334c54	qubes/devices: calculate event names using + This make static analysis of the events easier.	2016-03-23 09:08:13 +01:00
Wojtek Porczyk	349e218638	qubes/devices: devices class framework This fixes loading PCI devices.	2016-03-23 09:08:13 +01:00
Wojtek Porczyk	a4fa1adb82	qubes/ext/gui: fix sending monitor layout at VM startup Based on marmarek's commit 4edb42c1 under the same name.	2016-03-21 11:44:59 +01:00
Marek Marczykowski-Górecki	15e032cddb	qubes/vm: fix handling "None" kernel The "None" value still makes sense for HVM domains.	2016-03-21 11:44:59 +01:00
Marek Marczykowski-Górecki	a02f7a30fc	qmemman: fix core3 API usage for sending events	2016-03-21 11:44:59 +01:00
Wojtek Porczyk	29d0baa944	qubes/vm/mix/net: By default MAC is constant MAC address was one place, where qid was leaked to the VM. Different MAC addresses are not needed, since we do routing, not switching.	2016-03-21 11:44:59 +01:00
Wojtek Porczyk	04cc2099f7	HVM part 2	2016-03-21 11:44:54 +01:00
Marek Marczykowski-Górecki	8da7416f30	vm: Remove reference to old libvirt config template This file isn't installed anymore	2016-03-21 11:44:54 +01:00
Wojtek Porczyk	5eaf03c4a2	HVM part 1	2016-03-21 11:44:46 +01:00
Wojtek Porczyk	e0210130da	qubes: Fix error in get_vms_based_on() Not every vm has .template attribute.	2016-03-21 11:43:33 +01:00
Wojtek Porczyk	d766b8e110	qubes: Fix "unify event names"	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	9567f7b40b	vm: rename 'yum-proxy-setup' feature to 'updates-proxy-setup' The 'yum-proxy-setup' is deprecated since R3.0, so finally remove old name. But add it to R3.x compatibility layer.	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	b6d8c7fb81	ext/r3compatibility: features/services QubesOS/qubes-issues#1812	2016-03-21 11:43:33 +01:00
Wojtek Porczyk	0f03b257ce	qubes/vm: Fix features coercion	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	c9fd57e7e4	ext/r3compatibility: move old firewall code to R3.x compat layer QubesOS/qubes-issues#1812	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	afd4573a02	ext/r3compatibility: create R3.x QubesDB entries This allows the user to start VM based on "old" system (from R3.x) in R4.0. For example after restoring from backup, or migration. This also makes upgrade instruction much easier - no need complex recovery instruction if one upgrade dom0 before upgrading all the templates. QubesOS/qubes-issues#1812	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	54784405bd	tests: adjust networking tests	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	3b1238f7cf	vm: stub for new firewall API - introduce 'firewall-changed' event - add reload_firewall_for_vm stub function Should that function be private, called only from appropriate event handlers? QubesOS/qubes-issues#1815	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	34fe4a2d71	tests: add function to provide real network to test case Very few tests should use it, as it makes the test result somehow dependent on outside world state. But still, some of them need it.	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	c99a47d192	vm: fix handling firewall_conf path It may be (and by default is) path relative to VM directory. This code will be gone in the final version, after merging firewall configuration into qubes.xml. But for now have something testable.	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	c9cad71d03	vm/mix/net: create vm.connected_vms dynamically It is much less error-prone way. Previous approach didn't worked because VMs weren't added here at 'domain-init'/'domain-loaded' event. And even after adding such handlers it wasn't working because of QubesOS/qubes-issues#1816. It may be a little slower, but since it isn't used so often (starting/stopping VM and reloading firewall), shouldn't be a problem.	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	3e3213e7c7	Redo "core: do not reset firewal when setting netvm=none" Apparently it got lost in merge. QubesOS/qubes-issues#862	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	736773dbec	vm: remove obsolete firewall handling code There is no vm.write_iptables_xenstore_entry(). QubesOS/qubes-issues#1815	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	a0fa8fcc8e	ext/qubesmanager: fix events signature There is event itself passed just after VM object.	2016-03-21 11:43:33 +01:00
Marek Marczykowski-Górecki	12f778b539	qubes/vm: do not recursively use repr in BaseVM.__repr__ This will easily end up in infinite recursion. For example 'sys-net'.template points at 'fedora-23', which itself has 'fedora-23'.netvm set to 'sys-net'.	2016-03-21 11:43:32 +01:00
Marek Marczykowski-Górecki	422f30b969	tests: port network tests to core3 API	2016-03-21 11:43:32 +01:00
Marek Marczykowski-Górecki	df5562d2c7	tests: fix rename test to report FAIL instead of ERROR when really failed	2016-03-21 11:43:32 +01:00
Wojtek Porczyk	394fa1e5d3	qubes/tests: tests for qubes.devices	2016-03-21 11:43:32 +01:00
Wojtek Porczyk	0f9ca47d90	qubes/ext/guid: Move gui-related code to extension	2016-03-21 11:43:32 +01:00
Wojtek Porczyk	75dd882b83	qubes: Unify event names Events will be named <object>-[pre-]-<verb>, where verb is in infinitive form.	2016-03-21 11:43:32 +01:00
Wojtek Porczyk	d09bd5ab6a	qubes: Convert QubesVM and Extension discovery to pkg_resources QubesOS/qubes-issues#1238	2016-03-21 11:43:32 +01:00
Wojtek Porczyk	93686eae06	qubes/vm: change services to features	2016-03-21 11:43:32 +01:00
Wojtek Porczyk	bf78e662f6	Import qubesdb from the new module fixes QubesOS/qubes-issues#1252	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	5dca051b32	tests: enable storage tests	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	8262f2ef4b	storage: fix handling standalone VMs Currently Standalone have template=None.	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	a73c694f54	storage: access *_img directly from vm.storage object Not through vm.something_img proxy.	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	c0c0e0022e	tests: convert storage tests to core3 API QubesOS/qubes-issues#	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	14f31134c0	tests: move make_vm_name to QubesTestCase This function is often useful also in unit tests, not only integration tests.	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	5375dce90d	Prefer human readable output in qvm-prefs and qubes-prefs tools	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	68ed50c97d	tests: enable dom0 update tests	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	fb74126e56	tests: convert dom0 update test to core3 API	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	0307ec22f2	tests: resolve template name when copying from host collection	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	81dbe3628f	tests: skip system tests if not in dom0	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	50819c726c	tests: set QUBES_XML_PATH env variable to force usage of qubes-test.xml This way even separate processes (even those started not directly - like qrexec service calls) will use correct qubes.xml file. Fixes QubesOS/qubes-issues#1730	2016-03-03 01:18:17 +01:00
Wojtek Porczyk	c80e89d47e	qubes: support QUBES_XML_PATH environment variable This is reworked core: load default qubes.xml location from QUBES_XML_PATH env variable QubesOS/qubes-issues#1730	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	e3c94f3949	core: fix handling numeric label value set Not sure if it should be supported this way, but surely it shouldn't throw AttributeError.	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	79a6da1163	tests: split off invalid values tests for better reporting Most of them fails for now...	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	c45c91c5f0	tests: improve verbosity of assertNotRaises even further Include traceback in error message.	2016-03-03 01:18:17 +01:00
Marek Marczykowski-Górecki	5a409ad417	tests: few more (disabled for now) tests for hvm related properties	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	1b69f5c213	tests: make test cleanup more defensive - qubes.xml load failure If qubes.xml doesn't even load, remove it anyway and clean up the VMs the hard way.	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	9633573408	core: adjust units comment for host.memory_total, and fix related places QubesOS/qubes-issues#1737	2016-03-03 01:18:16 +01:00
Wojtek Porczyk	88a63cbe3a	qubes/vm/qubesvm: do not clone VM-specific properties This is reworked core: do not clone VM-unique identifiers in clone_properties by Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	2cbbe0bdb6	core: abandon 'kernels' in VM directory Since we have PV Grub now (which is much more convenient), it is useless now.	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	9428c70d31	core: fix vm.create_on_disk QubesOS/qubes-issues#	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	62759fb1f6	core: move maxmem default initialization to property itself Also fix (for now?) units - memory_total is int bytes while maxmem in megabytes. QubesOS/qubes-issues#1737	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	2f3762ee68	core: minor fixes in handling properties Conflicts: qubes/vm/qubesvm.py	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	12941d67bf	tests: do not unnecessary mangle test name passed as runner parameter Otherwise `qubes.tests.run -l` output would not be useful as input for test list.	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	05d006b873	tests: minor fixes in int.tools tests	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	163ab25d4b	tests: enable qubes.tests.int.basic It is fully converted to core3 API (or tests marked as 'skip' if not), so can be reenabled.	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	9b1b82c592	tests: move properties tests to QubesVM unit tests Since qvm-prefs is now a generic tool without any property-specific stuff, we can tests properties getters/setters in QubesVM class unit tests. Now most of properties are tested. Some of those tests still fails because missing/incomplete underlying implementation.	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	81bc615bce	tests: convert tests/int/basic to core3 API Not all tests converted - some of them will be removed in the next commit. Also it doesn't mean that all tested functionality already works...	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	2daba9e0eb	core: migrate handy GUI functions and backup mixin from core2 That's all from tests/__init__.py, so remove that file.	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	bc82643580	tests: redo "84af738 tests: support VMs cleaned up in tearDownClass..." Usage: VMs with name created by self.make_vm_name(name, class_teardown=True) will be cleaned up in tearDownClass. It should be used only in setUpClass. Fixes QubesOS/qubes-issues#1691	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	0a45164be7	tests: few more FD leak fixes QubesOS/qubes-issues#1697	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	47e346efbf	tests: initialize Qubes() object in SystemTestsMixin, add handy helpers Most of the tests needs existing template, so add helper to copy it from "host" collection. Add also helpers to reload qubes.xml - in case of modification of it from some other process.	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	2966456387	tests: improve assertNotRaises reporting Include whole exception value, not only its type.	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	f52a0f2d41	storage: get rid of 'servicevms' subdir Keep all the non-template VMs in 'appvms'. Having it split makes no sense since any VM can be "converted" to NetVM by simply changing its provides_network attribute and probably assigning some PCI device. On the other hand, it may be useful to set this manually, for example by management stack - based on VM function, not type. But don't care about it for now.	2016-03-03 01:18:16 +01:00
Marek Marczykowski-Górecki	c1bab3d094	storage: reset_volatile_storage doesn't need source_template parameter It shouldn't touch anything not already known to the VM object.	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	5f48b76b27	storage: fix vm.create_on_disk for non-template based VMs	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	539a46ca9a	core/log: do not attach multiple handlers to the same logger logging.getLogger when given the same name twice, will return the same object. Do not attach handler there every time, it will really open new file, leading to multiplicated log entries and depleting file descriptors. QubesOS/qubes-issues#1697	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	64260f5844	core: handle default template Fixes QubesOS/qubes-issues#1559	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	15192ede42	core: allow not having clockvm/updatevm	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	f7f1038f57	core: add a stopgap detection for simultaneous qubes.xml access For now simply throw an exception. Proper solution require some locking QubesOS/qubes-issues#1729	2016-03-03 01:18:15 +01:00
Wojtek Porczyk	c531b0eac2	qubes: fix None serialisation for VMProperty Note that None != default (no attribute in XML at all). This is reworked 27eddcb WIP core: fix deserializing None for VMProperty by Marek.	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	843cf68c1a	core: return QubesVMNotFoundError when setting non-existent VM to VMProperty Not sure about this one. Maybe it should be rather QubesPropertyValueError?	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	380ad74494	core: fix property.__eq__	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	ee1e836523	core: fix clone_properties(proplist=something)	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	929923164e	Move libvirt_conn.close to object destructor Process exit is too late for long-running processes. Especially tests, which create a lot of such objects which leads to depleting all the file descriptors. QubesOS/qubes-issues#1697	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	ab469e4473	vm: fix calling monitor layout notifier	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	4095f3c5eb	vm: disable uses_custom_config for now	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	db20581340	storage: few more merge fixes	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	926596cba7	Initialize vm.storage after loading all the properties It needs at least vm.name to be set.	2016-03-03 01:18:15 +01:00
Marek Marczykowski-Górecki	3af71ab65d	storage: fix handling netvm and adminvm @woju what was consensus on 'servicevms' dir? Since "being network provider" is no longer separate type and can be changed during VM lifetime, maybe we should abandon that idea? Or maybe just set it as VM property (some storage pool config argument) - that way mgmt stack could put all the sys-* VMs into separate directory, regardless of the type.	2016-03-03 01:18:06 +01:00
Marek Marczykowski-Górecki	d3f9c0b169	storage: fix loading default storage pool qubes.config.defaults holds ['pool_drivers'] strings, not classes.	2016-03-03 01:14:54 +01:00
Wojtek Porczyk	c47e29ed50	Fix most errors from pylint	2016-03-03 01:14:54 +01:00
Wojtek Porczyk	9eafa57539	Merge remote-tracking branch 'marmarek/master' into core3-devel The following list is bollocks. There were many, many more. Conflicts: core-modules/003QubesTemplateVm.py core-modules/005QubesNetVm.py core/qubes.py core/storage/__init__.py core/storage/xen.py doc/qvm-tools/qvm-pci.rst doc/qvm-tools/qvm-prefs.rst qubes/tools/qmemmand.py qvm-tools/qvm-create qvm-tools/qvm-prefs qvm-tools/qvm-start tests/__init__.py vm-config/xen-vm-template-hvm.xml This commit took 2 days (26-27.01.2016) and put our friendship to test. --Wojtek and Marek	2016-03-03 01:13:51 +01:00
Wojtek Porczyk	2d6ad3b60c	qubes/vm/qubesvm: remove prefixes from qubesdb keys	2016-03-03 00:46:05 +01:00
Wojtek Porczyk	e8bedcd7ba	qubes/vm: fix starting netvm	2016-03-03 00:46:05 +01:00
Wojtek Porczyk	6bc30b2e0b	qubes/tools/qvm_ls: fix getting columns from mixin	2016-03-03 00:46:05 +01:00
Wojtek Porczyk	21d6d51cbb	qubes/tools/qvm_ls: remove fmt= argument Unneeded, added complexity.	2016-03-03 00:46:05 +01:00
Wojtek Porczyk	01319e391f	qubes: port netvm From now, there are no separate NetVM and ProxyVM class, but property "provides_network".	2016-03-03 00:46:05 +01:00
Wojtek Porczyk	638c31ce68	qubes/tests/vm/qubesvm: some more tests	2016-01-21 13:31:43 +01:00
Wojtek Porczyk	06cc064c8c	qubes/tests: split init to init1 and init2 This is to keep the correct order of the tests. The exact dependency graph is somewhat complicated and contains several cycles.	2016-01-21 13:31:43 +01:00
Wojtek Porczyk	689dd64b0d	qubes: misc pylint fixes	2015-12-29 22:04:00 +01:00
Wojtek Porczyk	bf4dbe07d5	qubes/tests/vm/qubesvm: add basic tests for QubesVM And already one fix to instantiation.	2015-12-29 20:35:04 +01:00
Wojtek Porczyk	66475e5ca7	qubes/tests/run: Fix running foregin TestCases This is needed, because when tests fail to load, unittest module generates artificial „LoadTestsFailure” class -- not based on our QubesTestCase, so it does not have .log attribute.	2015-12-29 20:08:47 +01:00
Wojtek Porczyk	13b1a7201a	qubes/tests: fix tearing down QubesTestCase	2015-12-29 16:49:17 +01:00
Wojtek Porczyk	8d55bf9016	qubes/tools/qvm_run: do not colour autostart log messages	2015-12-29 14:47:37 +01:00
Wojtek Porczyk	69e0b94c74	Fix for UUID management This fixes up `75c3f355db` "qubes/vm/qubesvm: fix domain's UUID management". Previously 'domain-init' was never really fired. fixes QubesOS/qubes-issues#1557	2015-12-29 03:50:35 +01:00
Wojtek Porczyk	ce0b927a98	qubes/tests/int/tools/qvm_run: add tests for qvm-run Missing is test for --gui/--no-gui. part of QubesOS/qubes-issues#1226	2015-12-29 03:50:35 +01:00
Wojtek Porczyk	f1a0b1af39	qubes/tools: add qvm-run, qvm-{,un}pause Also change convention of calling main(): now command returns its numeric value instead of bool. Also fixed QSB#13 fixes QubesOS/qubes-issues#1226	2015-12-29 03:43:08 +01:00
Wojtek Porczyk	7b30361fa6	qubes/tools: add qubes-prefs fixes QubesOS/qubes-issues#1209	2015-12-24 00:48:17 +01:00
Wojtek Porczyk	87f74a5ba8	qubes/storage: Fix template instantiation rootcow image was somehow dropped	2015-12-24 00:14:28 +01:00
Wojtek Porczyk	ec50a411fe	qubes/tests/int/tools/qubes_create: fix wrong assert	2015-12-23 14:34:54 +01:00
Wojtek Porczyk	5c57b4cda7	qubes/tests: fix regression in test setup Regression was caused by 648fe213 "core3: test fixing"	2015-12-23 14:34:54 +01:00
Wojtek Porczyk	e4a0cfc8c8	qubes: fix instantiating with no or empty qubes.xml There is always dom0 (domid=0) added, but it has None as self.xml.	2015-12-23 14:34:54 +01:00
Wojtek Porczyk	b54305bff1	qubes: fix qubes-create regression Previous commit, which changed the handling of qubes.xml again, broke bootstrapping.	2015-12-23 14:34:54 +01:00
Wojtek Porczyk	58608864c1	qubes: fix saving qubes.xml	2015-12-23 14:34:54 +01:00
Wojtek Porczyk	d693964ab9	qubes: fix deleting domain	2015-12-23 14:34:53 +01:00
Wojtek Porczyk	0dc0fd306f	core3: test fixing part of QubesOS/qubes-issues#1248	2015-12-23 14:34:53 +01:00
Wojtek Porczyk	dc3fcc3141	qubes/tests: port again the backported test runner part of QubesOS/qubes-issues#1248	2015-10-17 00:20:44 +02:00
Wojtek Porczyk	2e42a408e5	core3: Add documentation to qubes.exc part of QubesOS/qubes-issues#1279	2015-10-17 00:20:44 +02:00
Wojtek Porczyk	96efb4568a	core3: add different exceptions From now on there are different exceptions which can be raise on different occasions. fixes QubesOS/qubes-issues#1279	2015-10-17 00:17:12 +02:00
Wojtek Porczyk	ea44c0acf3	qubes: pylint fixes Fix bunch of errors and warnings.	2015-10-05 23:49:39 +02:00
Wojtek Porczyk	15713cbf46	qubes/tools: rewrite qvm-kill Also new function, `error_runtime` for common parser. fixes QubesOS/qubes-issues#1222	2015-10-05 18:06:02 +02:00
Wojtek Porczyk	015b01fe46	qubes/tools: unify looking up the domain This is common operation that is repeatable.	2015-10-05 17:34:13 +02:00
Wojtek Porczyk	55e4ddcd3d	qubes/tools: custom QubesArgumentParser Common functions that revolve around running the tool (like `dont_run_as_root`) will be methods of this class.	2015-10-05 14:03:11 +02:00
Wojtek Porczyk	c9cbf8ffe2	core3: fix mounting modules.img	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	15c59f938c	qubes/storage/xen: fix RO snapshot	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	3724d6fca1	qubes/vm/qubesvm: fix core2 compatibility	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	8628a984fc	qubes/vm: fix memory constraint while generating libvirt XML	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	c538d536c8	core3: move qmemman This is part of fixing qvm-start. qmemman was moved with minimal touching, mainly module names. Moved function parsing human-readable sizes from core2. This function is wrong, because it treats k/M/G as 1024-based, but leave it for now.	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	fcdb579bab	core3: qubesmanager notifying extension core/notify.py was excavated and tray notifying remnants were moved to qubes.log. They are unused as yet. Also extension events are fixed.	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	75c3f355db	qubes/vm/qubesvm: fix domain's UUID management Now we generate UUID on domain's instantiation. This is needed, because we sometimes don't update XML (like in qvm-start). fixes QubesOS/qubes-issues#1236	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	6f4951d08a	install python package with setuptools	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	ce60915fb0	qvm-ls: fix listing domains and labels	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	b0be1ad584	qubes/tools: qvm-start rewritten from original	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	17ac6cb225	core3: boolean properties can be set explicitely	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	b4d51b016b	core3: some properties can be set only once Some properties should not be changed by user at will (like UUID). The solution is to make them write-once, so they will be set when loading from XML and frozen for the lifespan of the object holding the property. When desperately needed, users may edit XML by hand. fixes QubesOS/qubes-issues#1235	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	a017d78174	Dumb down dir_path property From now, dir_path cannot be set and is calculated from domain's class and name. fixes QubesOS/qubes-issues#1234	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	12329e7b35	qubes/vm/qubesvm: fix log initialisation	2015-10-05 12:46:14 +02:00
Wojtek Porczyk	9da5ae8169	qvm-create: fix --template option	2015-10-05 12:46:13 +02:00
Wojtek Porczyk	d3425142bc	qubes/tools: hide --xml option It requires user to know, what he is doing	2015-10-05 12:46:13 +02:00
Wojtek Porczyk	b0af95fa2b	qvm-prefs: support resetting values	2015-10-05 12:46:13 +02:00
Wojtek Porczyk	af1a6a49bd	qvm-ls: fix printing label	2015-10-05 12:46:13 +02:00
Wojtek Porczyk	e7cba0214f	qubes/tools: qvm-prefs	2015-10-05 12:46:13 +02:00
Wojtek Porczyk	0fbee4dc10	core3: more fixes	2015-10-05 12:46:13 +02:00
Wojtek Porczyk	80d664441d	core3: fixes from Marek This is adapted from commit 90a50dca406e3d40c88ea338566e0460589df7a3.	2015-10-05 12:46:13 +02:00
Wojtek Porczyk	e9b998400d	qubes/vm/qubesvm: fix __init__ wrt .storage attr	2015-09-28 18:20:28 +02:00
Wojtek Porczyk	6241859dc8	qubes: Fix PropertyHolder.property_get_default And test cases, some of them stubs.	2015-09-28 18:20:28 +02:00
Wojtek Porczyk	067f033af6	qubes/tools/qvm-create: Add internal --no-root option And documentation fixes.	2015-09-28 18:20:28 +02:00
Wojtek Porczyk	136b3b36b8	qubes/tools/qubes-create: add --property option	2015-09-28 18:20:28 +02:00
Wojtek Porczyk	84eb082ed2	qubes/vm/qubesvm: fix label setting/loading	2015-09-28 18:20:28 +02:00
Wojtek Porczyk	f536935632	qubes/tools: add --verbose and --quiet to tools	2015-09-28 18:20:28 +02:00
Wojtek Porczyk	ea9a984fa7	qubes/vm: Fix instantiating QubesVM vm.app attribute needs to be accessible before setting properties	2015-09-28 18:20:28 +02:00
Wojtek Porczyk	cb62545007	qubes: fix qid generation	2015-09-28 18:20:07 +02:00
Wojtek Porczyk	12092c3aa5	core3: port TemplateVM	2015-09-28 18:15:24 +02:00
Wojtek Porczyk	ff7d89700a	qubes/tools: port qvm-create	2015-06-30 17:18:24 +02:00
Wojtek Porczyk	669a976d4e	qubes/tools: add common action for setting properties	2015-06-29 17:40:43 +02:00
Wojtek Porczyk	b11b8cdbde	qubes/tools: add common --force-root option	2015-06-29 17:39:29 +02:00
Wojtek Porczyk	6dc5f328f3	qubes/tools: ArgumentParser prepopulated with common options	2015-06-29 17:39:29 +02:00
Wojtek Porczyk	7832f23d5d	qubes: minor fix	2015-06-29 17:39:29 +02:00
Wojtek Porczyk	7886695c63	qubes/vm/qubesvm: fix qid setter to accept str Mainly for loading from XML.	2015-06-29 17:39:29 +02:00
Wojtek Porczyk	d4147c52bd	qubes: Fix property-set events signature	2015-06-29 17:39:29 +02:00
Wojtek Porczyk	7f16e0a73b	qubes: disable events by default not to interfere with loading from XML	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	f2d9d628c4	qubes/vm/qubesvm: Fix loading/instantiating	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	fd0107f11a	new tool: qubes-create At present it just creates default qubes.xml (empty barring labels and AdminVM). In the future it can be evolved into backend for firstboot.	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	8e690c6858	qubes: fix saving Label	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	d5f2e6ab74	qubes: rework qubes.xml handling Now it more closely aligns with old core2 semantics. New semantics will have to wait for qubesd.	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	7c8cc14bec	core3: fix import	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	9a5703d5ea	pylint fix	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	142521317c	qubes: Documentation for qubes.tools And first documentation for qubes.tools.qvm_ls.	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	6a4820c381	qubes/tools: qvm-ls	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	e83d21c671	qubes: more documentation and doc fixes	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	5d9b92a039	qubes: pylint fixes (minor programming errors)	2015-06-29 17:39:28 +02:00
Wojtek Porczyk	186b277418	qubes/vm: fix logging	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	45977fc873	qubes: fix VM instantiation and loading	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	8c437f4053	qubes: unify PropertyHolder API get_props_list() -> property_list() _init_property() -> _property_init()	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	6798790e1f	qubes: pylint fixes (fix signatures)	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	8d3edbf133	qubes/vm: fix kernels_dir	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	a686e810b9	qubes/tests: enhance test runner report	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	adb144acfe	qubes: pylint fixes (small mistakes and wrong names)	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	4a85c823c3	qubes: pylint fixes (qualify imported variables)	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	bf29d5e5b5	qubes: pylint fixes (disable unfounded messages)	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	ac1739df6b	qubes: pylint fixes (fix imports)	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	ee06e7d7a2	qubes: documentation and licence fixes	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	cdc3df66c8	qubes: pylint fixes (mostly whitespace)	2015-06-29 17:39:27 +02:00
Wojtek Porczyk	d937d39c9e	qubes: fix Qubes instantiation	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	8e16dd2b28	qubes/storage: fix storage instantiation	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	8805db5e5f	core3 move: AdminVM class	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	8afba4c5e9	core3 move: storage/*	2015-06-29 17:39:26 +02:00
Jason Mehring	0dbcdb8c0d	qubes: pep8 fixes ------------------------------------------------------------------------------- ISSUES: ------------------------------------------------------------------------------- - Some auto-corrected code (when line is too long) may still be over-indented. It can be manually chaged and it will be left alone, or is it acceptable as I am not sure how strict your rule is for under-indented lines for which context. If you want this only indented 4 spaces, I can work on it some more. [Also @ ~line:385 in new file] For example, __init__.py:382 OLD: def __contains__(self, key): return any((key == vm or key == vm.qid or key == vm.name) for vm in self) NEW: def __contains__(self, key): return any((key == vm or key == vm.qid or key == vm.name) for vm in self) - will not detect if there are more than 2 spaces between function methods ------------------------------------------------------------------------------- FIXED: ------------------------------------------------------------------------------- - Now uses the most horizontial space and does not use excessive lines when splitting a line - __init__:489 - '#' comments being indented for some lines and not others; would like no indent - Only happens if line preceeding comment ends in a ':' E128 - Fix visual indentation E128 - Fix a badly indented line [Now allows under-indented lines] E309 - Add missing blank line (after class declaration) [No longer adds it] E303 - Remove extra blank lines [Now allows 2 blank lines between function defs] [TODO: Create definition to enforce this] Conflicts: qubes/__init__.py	2015-06-29 17:39:26 +02:00
Jason Mehring	6504da9524	qubes: changed a test for None from == None to is None Conflicts: qubes/__init__.py	2015-06-29 17:39:26 +02:00
Jason Mehring	be3e888bbe	Fixed typos	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	a13a41fbaf	qubes/tests: fix testrunner dependency on being run in specific directory	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	e5d2b49fd6	qubes/tests: fix colourful testrunner	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	7e12d0485d	add core3 to Makefiles and spec	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	04c221e924	qubes/vm/qubesvm: fix env manipulation on qrexec-daemon start	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	eda2f7cf73	qubes/tests: mark one test as expected failure This is temporary, to be fixed in the future.	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	af154b53fe	qubes: change names of XML generating methods Methods returning lxml.etree.Elements are called xml_ or lvxml_, meant for qubes.xml or libvirt respectively. Acknowledgement: This commit is a result of core3 review by Marek.	2015-06-29 17:39:26 +02:00
Wojtek Porczyk	92eca8edb9	qubes: Fix comments accross the code Acknowledgement: This commit is a result of core3 review by Marek.	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	2e1696cb16	qubes: Fix XML validation test	2015-06-29 17:39:25 +02:00
Wojciech Zygmunt Porczyk	42d8e67556	doc: Add autogenerated qubes.xml documentation	2015-06-29 17:39:25 +02:00
Wojciech Zygmunt Porczyk	118edb6ac4	First RelaxNG schema of qubes.xml Schema is most likely incomplete.	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	091ffa5444	qubes: Add parser for property docstring From now, docstrings in properties cannot contain sphinx-specific features, because there is no sphinx in dom0.	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	0a94762508	doc: Tutorial for qubes.events and fix	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	99edcb56c1	qubes: fix event framework Two important fixes are in this commit: handlers from decorators are added when class is defined (and not when class is instantiated); also multiple events can be specified in the decorator.	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	7971342811	qubes: Make get_props_list a classmethod Same for get_property_def.	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	c0e3281d04	qubes: fix changing domain name Changing name of running VM is wrong. Acknowledgement: This commit is a result of core3 review by Marek.	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	1deb3221c7	qubes: fix netvm semantics WRT default values Automatic acquiring default*_netvm, default_template and {clock,update}vm is no more. This will be moved to firstboot. Advanced users (those, who elect not to autoconfig their initial VMs) will have to deal with that. Acknowledgement: This commit is a result of core3 review by Marek.	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	c2a35c02b4	qubes: Cache QubesHost requests, fix xen-specific members Acknowledgement: This commit is a result of core3 review by Marek.	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	2a62780ea2	qubes: add property-del events	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	6ec86ec9f7	qubes: property may be unset by assinging DEFAULT Introducing qubes.property.DEFAULT special value, which may be assigned to any property. Result is the same as del'ing a property.	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	713ced8cd2	qubes/tests: add event asserts for qubes module	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	74c3126b80	qubes/tests: add some tests for qubes.property	2015-06-29 17:39:25 +02:00
Wojtek Porczyk	eabc571102	qubes/tests: colourful test runner	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	52c1be49ec	qubes/vm: remove unneccessary import	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	de9eb60f61	Developer's documentation for qubes.tests	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	8ace0fa634	qubes/tests: skipping tests outside dom0 New variable and decorator enable skipping tests outside dom0, that is, without connection to libvirtd.	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	613b03d277	qubes/tests: Common TestEmitter class qubes.tests.TestEmitter is intended to check whether specific event fired on given emitter.	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	a82bf7cc54	qubes/tests: Move unit tests inside qubes/, add runner	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	9fa3d60d0b	qubes/events: fix event handling order Events are divided into "pre" and "post" events. "Pre" events fire handlers in MRO, "post" fire them in reverse.	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	41fef46db2	core3 move: QubesVM This is a big commit and probably incomplete. Tests will follow.	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	6b7860995b	qubes/events.py: Fire events from parent classes too	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	f9658ae338	qubes/vm: remove old event methods that were overlooked	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	f149c7b59b	qubes/vm: fixed __repr__ for BaseVM Previously it could fail with AttributeError when any of the properties was unset.	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	ef4f00dac0	qubes/vm: DeviceManager class for herding devices collections.defaultdict was not enough, because it cannot pass any arguments to factory. We need to pass domain object and device class to fire events on attach and detach.	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	1a032ecf2a	core3: basic global events and their documentation	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	855a434879	core3: event framework adjusted for global Qubes object From now, global events are emitted by qubes.Qubes object and handlers are registered there.	2015-06-29 17:39:24 +02:00
Wojtek Porczyk	b623a71d87	core3 move: QubesVmCollection This got split to qubes.Qubes and qubes.VMCollection. From now on, VMCollection is a stupid bag. Some parts went elsewhere.	2015-06-29 17:39:23 +02:00
Wojtek Porczyk	6146c8e466	QubesVmLabel: XML parsing	2015-06-29 17:39:23 +02:00
Wojtek Porczyk	6c68bd062e	qubes.dochelpers: Version check for manual pages	2015-06-29 17:39:23 +02:00
Wojtek Porczyk	96bff66546	qubes.dochelpers: Helpers for Sphinx documentation Currently it is possible to refer to Qubes' tickets via 🎫`#no`	2015-06-29 17:39:23 +02:00
Wojtek Porczyk	87ae0112eb	qubes/vm: New XML format loading	2015-06-29 17:39:23 +02:00
Wojtek Porczyk	f3673dd34c	core3 move: class QubesVmLabel	2015-06-29 17:39:23 +02:00
Wojtek Porczyk	778571fe8d	core3 move: class QubesHost	2015-06-29 17:39:23 +02:00
Wojtek Porczyk	c3dd13c0ab	qubes/log: logging routines	2015-06-29 17:39:23 +02:00
Wojtek Porczyk	320cb096f6	qubes: drop dry_run	2015-06-29 17:39:22 +02:00
Wojtek Porczyk	57d35fbc4c	qubes: fix qubes.QubesVMMConnection fix logical error in resetting offline_mode and drop redundant _common_getter()	2015-06-29 17:39:22 +02:00
Wojtek Porczyk	cec3db993d	core3 move: class QubesVMMConnection	2015-06-29 17:39:22 +02:00
Wojtek Porczyk	e1a6fb2859	core3 move: class QubesException	2015-06-29 17:39:22 +02:00
Wojtek Porczyk	65595e3b39	apidoc stub	2015-06-29 17:39:22 +02:00
Wojtek Porczyk	7f27d987cc	import framework for core3	2015-06-29 17:39:22 +02:00

... 20 21 22 23 24 ...

2028 Commits