Do not abort VM restarts due to inconsistent info

Prior to this commit, there have been occasional issues with the usage
of the Qubes Manager's VM restart button where the restart procedure
is interrupted due to an exception thrown after the VM in question is
shut down. The exception has the following backtrace:

  ----
  line: assert not vm.is_running()
  func: start_vm
  line no.: 1198
  file: /usr/lib64/python2.7/site-packages/qubesmanager/main.py
  ----
  line: self.caller.start_vm(vm)
  func: check_if_vm_has_shutdown
  line no.: 308
  file: /usr/lib64/python2.7/site-packages/qubesmanager/main.py

Upon investigation, the root cause of the issue appears to be
inconsistent information provided by Xen regarding a recently-shut-down
VM's start-up timestamp and its state (i.e., running or shut down).

In some cases Xen would report that the VM is running whereas the
start-up timestamp would be returned as None, due to unknown reasons.
This inconsistency would then cause the code modified by this commit to
call the Qubes Manager's "start_vm" method, which would attempt to
assert that a VM is shut down, which would raise the aforementioned
exception.

This commit aims to resolve this issue by checking whether the VM has
fully shut down according to Xen and by calling "start_vm" only if the
VM has fully shut down.

This commit also slightly refactors the affected code.

Fixes: QubesOS/qubes-issues#2438
This commit is contained in:
M. Vefa Bicakci 2017-05-13 14:17:58 -04:00
parent 23fc7df23c
commit d551bdc3fa
No known key found for this signature in database
GPG Key ID: 1DF87CE3B3A5DFAF

View File

@ -273,13 +273,26 @@ class VmShutdownMonitor(QObject):
self.shutdown_started = datetime.now()
self.caller = caller
def restart_vm_if_needed(self):
if self.and_restart and self.caller:
self.caller.start_vm(self.vm)
def check_again_later(self):
# noinspection PyTypeChecker,PyCallByClass
QTimer.singleShot(self.check_time, self.check_if_vm_has_shutdown)
def timeout_reached(self):
actual = datetime.now() - self.shutdown_started
allowed = timedelta(milliseconds=self.shutdown_time)
return actual > allowed
def check_if_vm_has_shutdown(self):
vm = self.vm
vm_is_running = vm.is_running()
vm_start_time = vm.get_start_time()
if vm.is_running() and vm_start_time and \
vm_start_time < self.shutdown_started:
if (datetime.now() - self.shutdown_started) > \
timedelta(milliseconds=self.shutdown_time):
if vm_is_running and vm_start_time and vm_start_time < self.shutdown_started:
if self.timeout_reached():
reply = QMessageBox.question(
None, self.tr("VM Shutdown"),
unicode(self.tr("The VM <b>'{0}'</b> hasn't shutdown within the last "
@ -290,22 +303,22 @@ class VmShutdownMonitor(QObject):
self.shutdown_time / 1000))
if reply == 0:
vm.force_shutdown()
if self.and_restart:
if self.caller:
self.caller.start_vm(vm)
self.restart_vm_if_needed()
else:
# noinspection PyTypeChecker,PyCallByClass
self.shutdown_started = datetime.now()
QTimer.singleShot(self.check_time,
self.check_if_vm_has_shutdown)
self.check_again_later()
else:
QTimer.singleShot(self.check_time,
self.check_if_vm_has_shutdown)
self.check_again_later()
else:
if vm_is_running:
# Due to unknown reasons, Xen sometimes reports that a domain
# is running even though its start-up timestamp is not valid.
# Make sure that "restart_vm_if_needed" is not called until
# the domain has been completely shut down according to Xen.
self.check_again_later()
return
if self.and_restart:
if self.caller:
self.caller.start_vm(vm)
self.restart_vm_if_needed()
class VmManagerWindow(Ui_VmManagerWindow, QMainWindow):
row_height = 30