A few weeks back we were running a data preparation script that took a long time to complete. The main part of the work should not take that much time, but somehow that script was sloooow. I expected that the print() statement for each record would have an impact on performance, but I had no indication of the magnitude. Is it significant? Or is it neglectable? Let us find out how big the impact can be.
This post is part of my journey to learn Python. You find the code for this post in my PythonFriday repository on GitHub.
Timeit to measure our script
With %timeit we could measure the runtime of a Jupyter cell. The same tool can help us to measure a function or a statement without the need to run it inside Jupyter.
Timeit is part of Python; therefore, we do not need to install a package.
A sample script
Since I cannot share the original script, I created this code to show the problem:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
import timeit """ A simplified example on how much print() can cost if you call it in huge numbers. """ def with_prints(): a = 0 for i in range(0,1000): for y in range(0, 1000): a += 1 print(a) return a def without_prints(): a = 0 for i in range(0,1000): for y in range(0, 1000): a += 1 #print(a) return a if __name__ == '__main__': print(timeit.repeat(with_prints, number=1, repeat=5)) print(timeit.repeat(without_prints, number=1, repeat=5)) print(f"number of loops: {without_prints()}") |
The two methods with_prints() and without_prints() only differ in the print() statement. In both methods we loop 1 million times through our logic, but only in with_prints() do we print a value to the console.
Measure it
If we run this script with python XXXX.py, timeit will run the method one time and repeats the measurement 5 times. This gives us an output like this one:
|
1 2 |
[73.02711110000018, 71.45730600000024, 76.76619669999945, 73.35993109999981, 64.34289660000013] [0.02264809999996, 0.02240359999996, 0.02250490000005, 0.02240470000015, 0.02237599999989] |
The 5 repetitions show us that range in which the runtime differs even with the same code and on the same machine. The method with the print() statement took between 64 and 76 seconds, while the method without the print() statement run in 0.02* seconds. For this demo code the print() statement in the loop slowed the code down by a factor of 3000 – or in other words, 99% of the runtime is spend on writing to the console.
Conclusion
The print() statement takes time to write to the console, but when you use it sparely it will not have a big impact on the performance. However, if you call print() a million times, then you get the full negative impact on the performance and you waste a lot of time with waiting on the terminal.
To get a better understanding on how big this problem is with your code, use timeit and measure the performance, once with the print statements and then without it. That will give you a much better position to make a fact-based decision.