Qubes/core-admin
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

qubes/doc: Test's documentation improvement

Browse Source
This commit is contained in:
Wojtek Porczyk 2015-01-08 16:48:19 +01:00
parent eabc571102
commit b442fb0fab
1 changed files with 21 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
doc/qubes-tests.rst
View File

@ -42,14 +42,30 @@ First of all, testing is art, not science. Testing is not panaceum and won't
solve all of your problems. Rules given in this guide and elsewhere should be solve all of your problems. Rules given in this guide and elsewhere should be
followed, but shouldn't be worshipped. followed, but shouldn't be worshipped.
When writing test, you should think about order of execution. Tests should be Test can be divided into three phases. The first part is setup phase. In this
written bottom-to-top, that is, tests that are ran later may depend on features part you should arrange for a test condition to occur. You intentionally put
system under test in some specific state. Phase two is executing test condition
-- for example you check some variable for equality or expect that some
exception is raised. Phase three is responsible for returning a verdict. This is
largely done by the framework.
When writing test, you should think about order of execution. This is the reason
of numbers in names of the classes and test methods. Tests should be written
bottom-to-top, that is, test setups that are ran later may depend on features
that are tested after but not the other way around. This is important, because that are tested after but not the other way around. This is important, because
when encountering failure we expect the reason happen *before*, and not after when encountering failure we expect the reason happen *before*, and not after
failure occured. Therefore, when encountering multiple errors, we may instantly failure occured. Therefore, when encountering multiple errors, we may instantly
focus on fixing the first one and not wondering if any later problems may be focus on fixing the first one and not wondering if any later problems may be
relevant or not. This is the reason of numbers in names of the classes and test relevant or not. Some people also like to enable
methods. :py:attr:`unittest.TestResult.failfast` feature, which stops on the first failed
test -- with wrong order this messes up their workflow.
Test should fail for one reason only and test one specific issue. This does not
mean that you can use one ``.assert*`` method per ``test_`` function: for
example when testing one regular expression you are welcome to test many valid
and/or invalid inputs, especcialy when test setup is complicated. However, if
you encounter problems during setup phase, you should *skip* the test, and not
fail it. This also aids interpretation of results.
You may, when it makes sense, manipulate private members of classes under tests. You may, when it makes sense, manipulate private members of classes under tests.
This violates one of the founding principles of object-oriented programming, but This violates one of the founding principles of object-oriented programming, but
@ -86,7 +102,7 @@ class which uses :py:class:`qubes.tests.TestEmitter` as mix-in::
import qubes.tests import qubes.tests
class TestHolder(qubes.tests.TestEmitter, qubes.PropertyHolder): class TestHolder(qubes.tests.TestEmitter, qubes.PropertyHolder):
pass pass
class TC_20_PropertyHolder(qubes.tests.QubesTestCase): class TC_20_PropertyHolder(qubes.tests.QubesTestCase):
def test_000_event(self): def test_000_event(self):