Playwright, a Time-Saving End-to-End Testing Framework
How confident are you that your sites are up and running right now?
I’m fairly sure my projects are up, but I always face this hidden fear of production issues that I just don’t know about yet.
Because things break. Unnoticed. All the time.
Sure, a high and automated test coverage helps, but this is easier said than done because creating a well-running test suite is challenging.
What tests should I write? Unit tests? End-to-end tests? Should I test the APIs or the applications using them? Or both? And then, when and where should I run all these tests?
I don’t have all the answers, because, similar to software engineering, there’s no silver bullet in testing. It’s always a matter of tradeoffs.
But let me tell you how I approach the testing problem after building for the web for a decade and share my tool of choice today.
Playwright – A Modern End-to-End Testing Solution
While I was an obsessed “100% unit test coverage” advocate for most of my career, today, I lean towards user experience testing end-to-end. A thousand unit-tested components won’t help when they fail in combination. End-to-end testing, on the other hand, guarantees that all parts work correctly hand in hand.
My end-to-end testing experience has been a very rocky road, though. In the early days, I used Selenium, PhantomJS, and CasperJS, and constantly failed to create stable test suites, only to end up with unreliable systems reporting useless false positives.
Puppeteer, Cypress and other modern tools were a huge step in the right direction, but there’s a tool that the developer community doesn’t talk about enough.
Let me tell you why Microsoft’s open source Playwright is the underrated player in the space!
Cross-Browser, -Language, and -Platform
First off, Playwright is “cross-everything.”
Control all modern browser rendering engines: Chromium (Chrome/Edge), WebKit (Safari), and Gecko (Firefox).
Also, Playwright is cross-platform. Run it on Windows, macOS, Linux, your CI, or locally.
And lastly, Playwright works across programming languages, too. Write your tests in your favorite language: TypeScript, JavaScript, Python, .NET, or Java are supported.
These three points are already a solid foundation, but there are at least two more things to consider when choosing a testing solution: speed and developer experience.
Let’s dive into these!
A Testing Framework Built to Save Time
If you bet on automated tests, you’ll constantly create new ones. Every added test will make your test suite slower because more code has to run. Slow test suites can be a burden disrupting your delivery process. When it becomes too slow, your developers will only run them “when needed”. I’ve been in this situation, and trust me, this is the moment when you lost.
Keep an eye on your test execution time, or you won’t succeed.
But how can Playwright make a difference when it comes to speed?
Control the Browser with the DevTools Protocol
Playwright leverages the Chrome DevTools protocol to communicate with browsers directly. The protocol allows for a faster and less flakey execution than its alternatives. We only talk about fractions of seconds here, but these add up!
Auto-Waiting, Actionability, and Web-First Assertions
End-to-end tests usually include many waitFor statements. Click a button, and wait for something to appear. Submit a form, and wait for a new page to load. There is always something to wait for when you test your product end-to-end.
It’s refreshing that waiting is baked into Playwright itself. UI actions such as clicking an element or filling an input automagically wait.
Suppose you want to click a button that shows up after a UI action; you could now perform the UI action, wait for the button to appear, and click it.
In Playwright, you can “just” click the button and skip the waiting step. The Framework waits for the element to be actionable, which means: it’s in the DOM, visible, not animated, enabled, and not obscured by other elements.
|
1 2 3 4 5 6 7 |
```javascript // click this button when it is actionable await page.click('[data-test=cta]') ``` |
This auto-waiting mechanism streamlines UI tests tremendously so that you can focus on your behavior testing instead of writing code to wait. But that’s not all!
The team behind Playwright baked auto-waiting into their assertion library, too. Web-first assertions look like some convenience methods, but there’s more to them.
Let’s look at an example.
|
1 2 3 4 5 6 7 |
```javascript // wait for the .status element to have the text “Submitted” await expect(page.locator('.success)).toHaveText('You made it!'); ``` |
The above assertion queries a DOM element and checks if it holds the text “Submitted”. But this assertion is not a single check. “expect” reruns your check until the condition is met or it times out.
And while this seems like an unimportant detail, it leads to cleaner code because you don’t have to worry about when elements reach a particular state. Forget all these manual “waitFor” statements, and let Playwright figure out the rest!
Auto-waiting is a wonderful way to avoid wasting time with random timeouts and create the leanest tests possible. But what if that’s not enough and your test suite still cracks the unacceptable execution time?
Parallelize, Parallelize, Parallelize
Playwright allows you to parallelize your tests and run them in orchestrated OS processes (worker processes). Configure how many workers you want to start, define what should run in parallel, and call it a day. There’s really not much to it, but parallelization will speed up your tests massively.
Fast tests are the first part of the story, but what about the developer experience?
Run, Record and Debug
While Playwright is primarily a command line tool, it comes with handy “side apps” to ease the complicated end-to-end testing topic.
Record and Debug Your Scripts
Writing new tests takes time, and this is when code generation helps. Run a single CLI command and start recording the foundation of your next end-to-end test.
|
1 2 3 4 5 |
```sh <span class="s3">npx playwright codegen playwright.dev</span> ``` |
The command will open a browser window and the Playwright Inspector. Browser actions such as clicking links or filling out forms will be recorded in a generated script. When you finish your recording, take the script, adjust the selectors, add assertions and move the new script to your codebase.
But the Inspector is not only helpful when you create new scripts. There will be situations where you have to debug an existing script to find out why your tests aren’t working. The debug mode provides a simple way to step through your browser scripts. Playwright will highlight the selected elements so it’s easier to find out where things go wrong.
But maybe you’re more of an IDE person; Playwright has you covered there, too.
A Powerful VS Code Extension
To not run additional applications, you can install the Playwright VS Code extension and debug, record, and run your test code right from within VS Code.
That’s the creation and local debugging, but what do you do when scripts work on your machine but fail in other environments?
Travel Back in Time with Tracing
To debug failed scripts in a remote environment, Playwright supports tracing. A generated zip archive can be inspected via another CLI command or Playwright’s online trace inspector. You’ll find a timeline, all instructions, and screenshots of your failed test right in front of you.
Get notified about failing tests, download the trace and find out what happened. There’s nothing better than some good old time-traveling!
Developing tests with all these tools make a stellar development experience. What’s next?
When and Where Should You Run Your Tests?
With modern testing tools in your toolbelt, a final question remains. When should you run your tests to ensure that your entire stack is functioning?
Development teams ideally follow DevOps principles covering feature development, delivery, and testing. Your automated test suites should at least run in your CI/CD pipeline on every deployment to avoid regressions. The testing of a new production deployment should be the minimum to gain some confidence. But how can you reach 100% confidence?
Developers Come for the Tests but Stay for the Monitoring
I learned that deployment-driven test execution doesn’t give me enough confidence for all my projects. Let’s look at two examples.
Consider a static Jamstack site without any runtime computation. By design, fewer things can go wrong with this tech stack. And testing things on deployment might be good enough for it.
But on the other hand, think of a system with regular database updates, user-generated data, and multiple involved third-party providers. The number of error possibilities in these projects goes through the roof. Deployment-driven testing won’t cut it.
It’s common practice to monitor APIs to fulfill uptime SLAs. Why don’t we test our UIs the same way? How often do you use a service guaranteeing 99.999% API uptime but find glitches in their web product? I see these things too often.
So here’s this post’s hot take: UIs should be monitored with the same care as the APIs powering them to ship the best possible products. Products should be tested end to end. All the time!
And there are tools like Checkly that extend Playwright’s capabilities, enabling you to test your deploys and monitor your entire tech stack. Use Playwright to test your sites end-to-end, locally, on deployment, and on a schedule. Raise the bar and be sure that things are working at any time.
Test What Matters and Ship Stellar Products!
The only thing that matters is that products work without surprises. Issues must be caught before they hit production, and developers like myself should eliminate this constant fear of production issues.
There’s lots of movement in the testing space. Playwright and all the other new testing solutions are a significant step towards shipping better software. So, if you don’t test your products end-to-end yet, maybe today’s the day to get started! Trust me, end-to-end testing has come a long way, and it’s the way to ship with 100% confidence. Finally!
