Yield is a funny little keyword that allows us to create functions that return one value at a time. Let us look how yield works and how we can use it to create a generator.
This post is part of my journey to learn Python. You can find the other parts of this series here.
A little repetition of loops
When we have a list and want to loop through all elements, we can write something like this:
|
1 2 3 4 5 6 7 |
>>> mylist = [1,2,3] >>> for x in mylist: ... print(x) ... 1 2 3 |
This works because the list is an iterable, that means it has an __iter__() method that gives back an iterator. On the iterator we can use the __next__() method to go through the elements one by one.
The same principle comes to play when we use list comprehension:
|
1 2 3 4 5 |
>>> for i in [x*x for x in [1,2,3,4,5] if x % 2 == 0]: ... print(i) ... 4 16 |
Or when we iterate through a string, character by character:
|
1 2 3 4 5 6 7 8 9 |
>>> s = 'Hello' >>> for i in s: ... print(i) ... H e l l o |
In other words: whenever you work with a loop you probably use an iterable behind the scenes. If we want to use our own code in this way, we can create an iterator or a generator.
What are generators?
Generators are a convenient way to create a function that behaves like an iterator but without writing all the code of an iterator (like __iter__(), __next__() and StopIteration). All we need to do is to use the yield keyword at the place we normally use return (see PEP 255):
|
1 2 |
>>> def simple(): ... yield 1 |
If we use this simple() method in a for loop, we get 1 as the output:
|
1 2 3 4 |
>>> for i in simple(): ... print(i) ... 1 |
So far, so good. What looks rather useless will be exactly what we need to create our own fixtures in pytest. To understand yield better, we need a more verbose example:
|
1 2 3 4 5 6 7 8 9 |
def generator(): print("**A**") yield 1 print("**B**") yield 2 print("**C**") yield 3 print("**D**") yield 4 |
When we iterate over this generator, we get a more interesting output:
|
1 2 3 4 5 6 7 8 9 10 11 |
>>> for id in generator(): ... print(id) ... **A** 1 **B** 2 **C** 3 **D** 4 |
The for loop went multiple times through the generator() method, but the output is only printed once. The yield statement pauses the function and allows the Python interpreter to continue at exactly the position where it stopped the last time. Therefore, the loop prints **A** only once and not 4 times.
When it comes to big data structures and machine learning, the concept behind yield and generators allows you to work with large data sets that exceed the memory of your machine. How that works will be a topic for another post.
Next
With the knowledge of yield we can continue with pytest and create our own fixtures.
2 thoughts on “Python Friday #48: Yield and Generators”