After last week’s introduction to doctest it is now time to have a closer look at dynamic output and how we can test that with doctest.
This post is part of my journey to learn Python. You can find the other parts of this series here.
Test random numbers
Random numbers should be unpredictable, something that makes testing nearly impossible. We can expect to get different numbers from our documentation whenever our code runs and doctest will not be happy when they do not match.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
import random def random_int(): """ Returns a random integer between 0 and 10 >>> random_int() 3 >>> random_int() 5 >>> random_int() 7 """ return random.randint(0, 10) |
If we run doctest it will most likely fail because it gets other random numbers as in our example:
|
1 |
python -m doctest .\dynamicoutput.py |
***************************************************
File “.\dynamicoutput.py”, line 7, in dynamicoutput.random_int
Failed example:
random_int()
Expected:
3
Got:
7
***************************************************
File “.\dynamicoutput.py”, line 9, in dynamicoutput.random_int
Failed example:
random_int()
Expected:
5
Got:
9
***************************************************
File “.\dynamicoutput.py”, line 11, in dynamicoutput.random_int
Failed example:
random_int()
Expected:
7
Got:
1
***************************************************
1 items had failures:
3 of 3 in dynamicoutput.random_int
***Test Failed*** 3 failures.
We can use the seed() function before the first example to set the random number generator to a well-known starting position:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
def random_int(): """ Returns an integer between 0 and 10 >>> random.seed(111) >>> random_int() 3 >>> random_int() 5 >>> random_int() 7 """ return random.randint(0, 10) |
Run doctest again, capture the numbers from the doctest output and update the examples. From now on doctest will get the same random numbers as we had in our example:
|
1 |
python -m doctest .\dynamicoutput.py -v |
Trying:
random.seed(111)
Expecting nothing
ok
Trying:
random_int()
Expecting:
3
ok
Trying:
random_int()
Expecting:
5
ok
Trying:
random_int()
Expecting:
7
ok
1 items had no tests:
dynamicoutput
1 items passed all tests:
4 tests in dynamicoutput.random_int
4 tests in 2 items.
4 passed and 0 failed.
Test passed.
Exceptions
Depending where you put your modules and how you load them, your stack trace may be different. Doctest knows this and as long as your indentation of the exception is correct, doctest will match the important parts of the exception and ignores all the noise:
|
1 2 3 4 5 6 7 8 9 10 11 |
def exception_demo(): """ >>> exception_demo() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "D:\Python\dynamicoutput.py", line 37, in exception_demo 1 / 0 ZeroDivisionError: division by zero """ 1 / 0 |
You can go and modify the line number or the file name and doctest is still happy:
|
1 |
python -m doctest .\dynamicoutput.py -v |
Trying:
exception_demo()
Expecting:
Traceback (most recent call last):
File ““, line 1, in
File “D:\Python\dynamicoutputNOOOO.py”, line 0, in exception_demo
1 / 0
ZeroDivisionError: division by zero
ok
Test varying output
Date and time are just to examples of output that constantly changes. This is not so much a problem for the developer that wants to use our function, but a big problem for doctest:
|
1 2 3 4 5 6 7 8 9 10 |
from datetime import datetime def dynamic_text(): """ Returns the current time: >>> dynamic_text() 'It is now 14:52:45' """ now = datetime.now().strftime("%H:%M:%S") return f"It is now {now}" |
Immediately after we wrote our example the time passed and doctest will find an error:
|
1 |
python -m doctest .\dynamicoutput.py |
**************************************************
File “.\dynamicoutput.py”, line 23, in dynamicoutput.dynamic_text
Failed example:
dynamic_text()
Expected:
‘It is now 14:52:45’
Got:
‘It is now 14:52:58’
***************************************************
1 items had failures:
1 of 1 in dynamicoutput.dynamic_text
***Test Failed*** 1 failures.
There are two main ways we can handle this method. The simplest one is to say we ignore this test in doctest and add a # doctest: +SKIP at the end of the input:
|
1 2 3 4 5 6 7 8 9 10 |
from datetime import datetime def dynamic_text(): """ Returns the current time: >>> dynamic_text() # doctest: +SKIP 'It is now 14:52:45' """ now = datetime.now().strftime("%H:%M:%S") return f"It is now {now}" |
When doctest ignores our test, we do not get any test failures.
Another approach is to use ellipses and tell doctest that there are varying parts in our output that it can ignore. For that we use # doctest: +ELLIPSIS comment and replace the dynamic output with … (3 dots):
|
1 2 3 4 5 6 7 8 9 10 11 |
from datetime import datetime def dynamic_text(): """ Returns the current time: >>> dynamic_text() # doctest: +ELLIPSIS 'It is now ...' """ now = datetime.now().strftime("%H:%M:%S") return f"It is now {now}" |
Make sure that there are no spaces after the closing ‘ in which you use the … or it will not match the expected output. It took me a while to find this annoying bug in my documentation.
Which approach should you choose? If the varying output is not that important, like with the date of a log message or the memory address of an object, then the ellipsis is a good help. When the output matters I would rather use skip and have a meaningful documentation than an obscure test.
Conclusion
Doctest is a great help to ensure your documentation is up to date and matches the real output of your code. You can even test dynamic outputs and make sure that it matches or at least is similar to the expected output. If you use these parts in combination you can even skip a few tests that would reduce the readability of your documentation and still be able to release your code with confidence.
1 thought on “Python Friday #69: Dynamic Output and Doctest”