Преглед изворни кода

qubes/doc: Test's documentation improvement

Wojtek Porczyk пре 9 година
родитељ
комит
b442fb0fab
1 измењених фајлова са 21 додато и 5 уклоњено
  1. 21 5
      doc/qubes-tests.rst

+ 21 - 5
doc/qubes-tests.rst

@@ -42,14 +42,30 @@ First of all, testing is art, not science. Testing is not panaceum and won't
 solve all of your problems. Rules given in this guide and elsewhere should be
 followed, but shouldn't be worshipped.
 
-When writing test, you should think about order of execution. Tests should be
-written bottom-to-top, that is, tests that are ran later may depend on features
+Test can be divided into three phases. The first part is setup phase. In this
+part you should arrange for a test condition to occur. You intentionally put
+system under test in some specific state. Phase two is executing test condition
+-- for example you check some variable for equality or expect that some
+exception is raised. Phase three is responsible for returning a verdict. This is
+largely done by the framework.
+
+When writing test, you should think about order of execution. This is the reason
+of numbers in names of the classes and test methods. Tests should be written
+bottom-to-top, that is, test setups that are ran later may depend on features
 that are tested after but not the other way around. This is important, because
 when encountering failure we expect the reason happen *before*, and not after
 failure occured. Therefore, when encountering multiple errors, we may instantly
 focus on fixing the first one and not wondering if any later problems may be
-relevant or not. This is the reason of numbers in names of the classes and test
-methods.
+relevant or not. Some people also like to enable
+:py:attr:`unittest.TestResult.failfast` feature, which stops on the first failed
+test -- with wrong order this messes up their workflow.
+
+Test should fail for one reason only and test one specific issue. This does not
+mean that you can use one ``.assert*`` method per ``test_`` function: for
+example when testing one regular expression you are welcome to test many valid
+and/or invalid inputs, especcialy when test setup is complicated. However, if
+you encounter problems during setup phase, you should *skip* the test, and not
+fail it. This also aids interpretation of results.
 
 You may, when it makes sense, manipulate private members of classes under tests.
 This violates one of the founding principles of object-oriented programming, but
@@ -86,7 +102,7 @@ class which uses :py:class:`qubes.tests.TestEmitter` as mix-in::
    import qubes.tests
 
    class TestHolder(qubes.tests.TestEmitter, qubes.PropertyHolder):
-      pass
+       pass
 
    class TC_20_PropertyHolder(qubes.tests.QubesTestCase):
        def test_000_event(self):