In this post we explore 3 helpful ways to figure out what went wrong if Playwright cannot find the elements we are looking for.
This post is part of my journey to learn Python. You can find the other parts of this series here. You find the code for this post in my PythonFriday repository on GitHub.
Activate tracing
Playwright offers a great tracing tool that helps you to figure out where exactly something went wrong. We can activate the collection of the necessary information with this code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
from playwright.sync_api import Playwright, sync_playwright, expect def run(playwright: Playwright) -> None: browser = playwright.chromium.launch(headless=False) context = browser.new_context() # Start tracing before creating / navigating a page. context.tracing.start(screenshots=True, snapshots=True, sources=True) page = context.new_page() page.goto("https://github.com/") title = page.title(); page.get_by_role("heading", name="Let’s build from here").click() page.get_by_placeholder("Email address").click() page.get_by_placeholder("Search GitHub").fill("Playwright") page.get_by_placeholder("Search GitHub").press("Enter") page.get_by_role("link", name="Sign in").click() # Stop tracing and export it into a zip archive. context.tracing.stop(path = "trace_demo.zip") # --------------------- context.close() browser.close() with sync_playwright() as playwright: run(playwright) |
You can use the created *.zip file and run it through the trace viewer:
|
1 |
playwright show-trace .\trace_demo.zip |
Inside the trace viewer you can work with the shadow DOM and use your developer tools as you would in a regular browser.
Create videos
Playwright can create videos of our interactions with a web page. That is a great help should we encounter a strange behaviour and need to see exactly what happened. We can create the video with this code:
|
1 2 3 4 5 6 7 8 9 10 11 |
context = browser.new_context( record_video_dir="videos/", record_video_size={"width": 1280, "height": 720}, viewport={"width": 1280, "height": 720} ) page = context.new_page() page.goto("https://github.com/") path = page.video.path() print(f"video located in: {path}") |
The created videos will be in the folder videos.
Take screenshots
If we only need to know how a page looked at a specific time, we can use the screenshot() method to create a screenshot:
|
1 2 |
page.screenshot(path="screenshot.png") page.screenshot(path="screenshot_full.png", full_page=True) |
The first method call captures the web page as we would see it in the browser, while the second method call with the option full_page=True captures the whole web page, including the parts that are only reachable when we scroll through the page.
Next
If something does not work as we expect it, tracing and videos will make all the difference to figure out what went wrong. Next week we can take our newly gained knowledge of Playwright and revisit the web crawling problem for the WordPress statistics.