Python Friday #51: Parametrised Tests With Pytest

Parametrised tests are a great help to create many test cases without the need to write many tests. Let us look how they work in pytest.

This post is part of my journey to learn Python. You can find the other parts of this series here.

 

FizzBuzz

I like the FizzBuzz kata for its minimalistic requirements. This helps us to focus on the task at hand and not on the business logic. The basic requirements are these:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

I wrote this function to fulfil the requirements:

 

How not to test FizzBuzz

With all the parts I so far explained of pytest, we could come up with a lot of test functions like these ones:

This is too much to write and if we change the fizz_buzz() function, we need to update all our tests. Considering these downsides, we may try to write something like this condensed test:

These tests have multiple asserts for different test cases. As soon as the first test fails, pytest stops the test function and we do not get any feedback about other errors. As long as everything works this kind of tests may be useful. But as soon as this is no longer the case, we have a big problem and need a lot of time to figure out what went wrong.

 

Parametrised tests to the rescue

The concept of parametrised tests helps us to solve our problem in an elegant and maintainable way. Instead of pushing multiple test cases together, we use them as parameters to the test function. To do that, we need the @pytest.mark.parametrize() decorator with these two values:

1. The name of the parameters separated by ,
2. A list of values (or tuples, if you have multiple parameters)

Using parametrised tests we can test fizz_buzz() for values from 1 to 50 with just 4 test functions:

Pytest translates these 4 functions into 50 test cases:

=========== test session starts ============
platform win32 — Python 3.8.1, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: D:\Python, inifile: pytest.ini
plugins: cov-2.10.1, html-2.1.1, metadata-1.10.0
collected 50 items

test_fizzbuzz.py ………………………………………….. [100%]

============ 50 passed in 0.12s ============

If one of those parametrised tests fails, pytest will report us exactly which one has a problem and continues to run all other tests. If I add 3 to the list of values that should return the number, I get this error message:

================= FAILURES =================
_____ test_otherwise_returns_input[3] ______

input = 3

@pytest.mark.parametrize(“input”, [1,2,3,4,])
def test_otherwise_returns_input(input):
> assert input == fizz_buzz(input)
E AssertionError: assert 3 == ‘Fizz’
E + where ‘Fizz’ = fizz_buzz(3)

test_fizzbuzz.py:62: AssertionError

This means that the input 3 failed and that fizz_buzz(3) returned ‘Fizz’ instead the expected 3. It is not the most understandable output, but you get used to it.

 

Multiple parameters for your test function

To use multiple parameters in our tests, we need to replace the list of values with a list of tuples – and make sure that they are in the order we specify the parameter names (in the first argument):

If we spell a parameter wrong (like ‘expeted’ instead of expected), pytest will throw us an exception like this one:

=========== test session starts ============
platform win32 — Python 3.8.1, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: D:\Python, inifile: pytest.ini
plugins: cov-2.10.1, html-2.1.1, metadata-1.10.0
collected 0 items / 1 error

================== ERRORS ==================
_________ ERROR collecting test_fizzbuzz.py __________
In test_multiple_inputs: function uses no argument ‘expeted’
========= short test summary info ==========
ERROR test_fizzbuzz.py

This safety net is a great help and prevents you from hours of debugging your tests.

 

Next

With parametrised tests we can save ourselves a lot of typing. But how do we know what parts of our application are covered by tests? Next week we will look at code coverage to answer this question.