Code coverage tools are a great help to find code that is not yet covered by your tests. Let us look how we can find those spots with pytest.
This post is part of my journey to learn Python. You can find the other parts of this series here.
Prerequisites
You need to install the packages coverage and pytest-coverage into your virtual environment:
|
1 |
$ pip install coverage |
|
1 |
$ pip install pytest-cov |
Gilded Rose Kata as an example
Code coverage metrics depend on two things: code and tests. In this post I use the Python example of the Gilded Rose Kata. I like this kata as a starting point for its realistic code. This code was made so bad on purpose and what works here will work with your code base as well.
First steps
Let us start wit a new empty test file called test_gilded_rose.py next to the gilded_rose module and add this test code:
|
1 2 3 4 5 6 7 8 9 10 |
import pytest from gilded_rose import Item, GildedRose def test_foo(): items = [Item("foo", 0, 0)] gilded_rose = GildedRose(items) gilded_rose.update_quality() assert "foo" == items[0].name |
If we run pytest as in the posts before, it reports that our test passes. To get a coverage report on the command line, we need to run pytest with the --cov option:
|
1 |
$ pytest --cov |
=========== test session starts ============
platform win32 — Python 3.8.1, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: D:\GildedRose-Refactoring-Kata\python
plugins: cov-2.10.1, html-2.1.1, metadata-1.10.0
collected 1 itemtest_gilded_rose.py . [100%]
----------- coverage: platform win32, python 3.8.1-final-0 -----------
Name Stmts Miss Cover
-----------------------------------------
gilded_rose.py 36 17 53%
test_gilded_rose.py 7 0 100%
-----------------------------------------
TOTAL 43 17 60%
============ 1 passed in 0.10s =============
This report shows us that gilded_rose.py has 36 statements covered by our test, yet 17 are missing. This gives us a code coverage of 53%. Our test file with the one test got a coverage of 100%.
The more test files and modules you have, the bigger this report gets. You can limit the coverage report to a specific module by appending it to the cov option (--cov=your_module):
|
1 |
$ pytest --cov=gilded_rose |
=========== test session starts ============
platform win32 — Python 3.8.1, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: D:\GildedRose-Refactoring-Kata\python
plugins: cov-2.10.1, html-2.1.1, metadata-1.10.0
collected 1 itemtest_gilded_rose.py . [100%]
----------- coverage: platform win32, python 3.8.1-final-0 -----------
Name Stmts Miss Cover
------------------------------------
gilded_rose.py 36 17 53%============ 1 passed in 0.10s =============
This high-level overview is good to see the overall view, but it lacks the actionable details.
HTML report with line-by-line coverage
The coverage package comes with an HTML report that shows exactly what lines are covered and which are not. We can generate the report by adding the --cov-report html option:
|
1 |
$ pytest --cov-report html --cov=gilded_rose .\test_gilded_rose.py |
=========== test session starts ============
platform win32 — Python 3.8.1, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: D:\GildedRose-Refactoring-Kata\python
plugins: cov-2.10.1, html-2.1.1, metadata-1.10.0
collected 1 itemtest_gilded_rose.py . [100%]
———– coverage: platform win32, python 3.8.1-final-0 ———–
Coverage HTML written to dir htmlcov
============ 1 passed in 0.10s =============
This creates a report in the htmlcov folder that looks like this:
The thin green line between the line numbers and the code shows the covered lines, while the missing lines are marked in red. We now got a visual help to find the missing parts for which we need to write additional tests.
To change the report folder to myreport, you can replace html with html:myreport in the command line:
|
1 |
$ pytest --cov-report html:myreport --cov=gilded_rose .\test_gilded_rose.py |
Branch coverage
The way we used the coverage tool so far has one caveat that you need to be aware of. It just looks at the lines of your code but ignores branches. If you have code that only has an if statement but not an else, you may not notice that you do not cover both possibilities. Let us use this code as an example to illustrate this problem:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
def a(number): if (number % 2 == 0): return "even" else: return "odd" def b(number): if (number % 2 == 0): return "even" def test_a(): assert "even" == a(2) def test_b(): assert "even" == b(2) |
If we create the coverage report as before, we expect the else part of method a to be missing:
|
1 |
$ pytest --cov-report html --cov . .\test_branches.py |
As expected, the else part is marked red because it is missing. However, in method b we get no indication that we miss a test case for an odd number. We can change that with the option --cov-branch:
|
1 |
$ pytest --cov-report html --cov . --cov-branch .\test_branches.py |
When we turn branch coverage on, we get a yellow marker on the missing branches. That allows us to find missing test cases when there is no else statement (as in function b).
Next
The coverage report is a great help to find missing test cases. While the details are great for developers, they are not that useful for other stakeholders. Next week I show you a tool to create an overview for your test suite.
1 thought on “Python Friday #53: Code Coverage for Pytest”