With the user access tokens stored, we can now explore the Twitter API. A good starting point is to look at the accounts you follow and on who follows you.
This post is part of my journey to learn Python. You can find the other parts of this series here. You find the code for this post in my PythonFriday repository on GitHub.
Rate limits are everywhere
Twitter uses rate limits on all its API endpoints. They vary from 15 requests per 15 minutes (for most endpoints) and go up to 2 million requests per month for search. You find the limit in the documentation for each endpoint.
Especially the 15 requests per 15 minutes is a limit you run into in no time. If that happens while you try to fetch data, Tweepy will throw an exception that will kill your application:
tweepy.errors.TooManyRequests: 429 Too Many Requests
88 – Rate limit exceeded
Since this is a common problem, there is a simple solution: You can set the property wait_on_rate_limit to True when you instantiate the API client:
|
1 |
api = tweepy.API(auth, wait_on_rate_limit=True) |
If you now hit the rate limit, Tweepy will wait until it can send the next request:
Rate limit reached. Sleeping for: 821
Iterate through your friends
In Twitter terms friends are the people who you an account. You can use the method get_friends and set the screen_name parameter to the account you want to check:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
# list friends friends = [] screen_name = 'j_graber' print(f"Friends (accounts {screen_name} follows)") for page in tweepy.Cursor(api.get_friends, screen_name=screen_name, count=200).pages(10): for user in page: name = f"{user.id} - {user.name} (@{user.screen_name})" # print(name) friends.append(name) print(len(page)) print(f"Friends: {len(friends)}") |
This method may need many calls to the Twitter endpoint; therefore, we use a cursor where we can specify how many results we want to get per request. By default, Tweepy asks for 20 accounts, but we can change that up to 200 (with count=200). If you ask for more than 200, Tweepy will reduce it to 20.
If I run the code above with my Twitter account, I get this output:
Friends (accounts j_graber follows)
200
200
91
Friends: 491
As you can see, it took 3 requests to get 491 accounts back.
Iterate through your followers
The followers are the accounts that follow a specific user on Twitter. As with the friends, we can use the method get_followers and use a cursor to get the followers back:
|
1 2 3 4 5 6 7 8 9 10 11 |
# list followers followers = [] print(f"Followers (accounts who follow {screen_name})") for page in tweepy.Cursor(api.get_followers, screen_name=screen_name, count=200).pages(10): for user in page: name = f"{user.id} - {user.name} (@{user.screen_name})" followers.append(name) print(len(page)) print(f"Followers: {len(followers)}") |
When I run this code I get an output like this:
Followers (accounts who follow j_graber)
200
200
90
Followers: 490
Who does not follow back?
The two code samples from above created lists for friends and followers. We can use the set difference from NumPy and find out which accounts are in the friends list but not in the follower list and put them into a file:
|
1 2 3 4 5 6 7 |
friends_who_not_follow_back = np.setdiff1d(friends,followers) with open(f"not_follow_back_{screen_name}.txt", "w") as f: for people in friends_who_not_follow_back: f.write(f"{people}\n") print(f"Friends but not Followers: {len(friends_who_not_follow_back)}") |
The combination of Tweepy and NumPy gives us great possibilities, even if NumPy knows nothing about Twitter. This flexibility is what I like so much about Python.
Examples with the V2 endpoint client
I find the V1 API a bit simpler to use when you start learning Tweepy. If you like to see the V2 examples, you can find them in my Python Friday GitHub repository. The files are in the Twitter folder and have a V2 suffix.
Next
Iterating through friends and followers is an important part to analyse networks of people. Next week we take a closer look at how we can debug Tweepy should something go wrong.