After my series of 13 posts about Playwright spanning C# and Python, it is time to answer the two question I got asked a lot in the past few weeks:
Is Playwright ready for production?
Yes
Should we switch to Playwright?
It depends.
The second one is a boring answer, but if you want to have my honest opinion, then that is it. Let’s recapitulate what we covered in this blog series and add the context to the favorite answer of consultants.
Playwright in a nutshell
What works great?
The installation is quickly done by adding a NuGet package or run pip for Python. No need to find drivers and especially no figuring out what version of Chrome requires what driver.
Playwright gives you a fast start into the world of web testing. The test recorder codegen is a great help and quickly records the interactions on your web application. To turn that into tests, you may need to check the selectors and add a few asserts.
Creating videos out of the box is another aspect that Playwright does exceptionally well. The only challenge is to figure out where exactly you need to inject the video request in NUnit. Otherwise, it does what it should without a problem.
The tracing feature is again a great help if you need to figure out why your tests no longer work. You need to write a bit too much code for my taste, but as soon as you have it, you can reuse it in your projects.
If you do not care about testing and only want to use Playwright to automate JavaScript-heavy pages, you can do that without any problems. I have the code only for the Python version, but you can adapt that to C# by replacing a few method calls.
If you already have a BrowserStack license, you can keep that and combine it with Playwright. Again, a bit too much code for your NUnit tests, but doable in an hour or two.
What is a pain?
If you have Selenium Grid and want to keep it around, you are up for a painful experience. You can get it to work, but it is such a fragile configuration that I expect it will break a lot.
What is missing?
The JavaScript / Node.js version of Playwright has a test report, which has similar features as the tracing file. Unfortunately, that feature does not work in C# or Python (yet).
Another feature I hope will find its way from JavaScript to C# and Python is the auto-retry. It is especially in complex environments a great help when Playwright automatically retries a failed test.
On what it depends to switch to Playwright
No existing test automation? Start with Playwright.
If you have no test automation yet, then Playwright is a good tool to change that. Especially the test recorder can help you to get a good coverage of your application in a short time. It will be much more work than you imagine, but a lot less than using a test recorder for Selenium.
Existing tests in Selenium? Keep it.
If you already have invested in test automation (based on Selenium or another framework), the question is not that easy to answer. If you are happy with the solution you have, you should keep it. At the end of the day the features of Playwright are nice, but when you already get the safety net of test automation, it is hard to justify starting over and do everything again in a new technology.
Unhappy with Selenium? Figure out the reason.
If you are unhappy with your existing test automation, you should make sure that it really is the framework you use that makes you unhappy. Otherwise, you run into the same problems with Playwright in no time.
Still on Selenium 3? Take the time to decide.
If you are on Selenium 3 and need a lot of work to port your tests to Selenium 4, then that would be a good moment to decide if you want to switch to Playwright. Whatever you want to do, there is a significant amount of work needed. You can spend that work on Selenium 4, or you invest it into Playwright.
Keep in mind that there is usually a lot of infrastructure around Selenium. Do your test reports work with Playwright? Or are you willing to recreate them for Playwright? Are your dashboards and your build pipelines easy to switch to Playwright? All that infrastructure must change as well and that can be a time-consuming switch. Therefore, do not rush the decision.
Parting thoughts
Over the last weeks I got a deep insight on Playwright. I saw a lot that I like and a few things I will not try in the future (like Selenium Grid). It was worth it to keep going after I run into the problems with codegen right at the beginning. Playwright is actively developed by Microsoft and the community and errors are quickly fixed. Nevertheless, prepare yourself for a different pace of changes, especially when you are used to Selenium 3.