Do you run browser tests with Playwright? Is your test suite getting slower the more tests you add? Do you feel like you don't understand why some tests are flaky or failing? Then you should probably keep on reading about why we're building Endform.
Hi 👋 , I'm Jakob, and together with my co-founder Oliver I'd like to introduce Endform – a platform we built to run your Playwright tests faster than anywhere else.
It all started when we worked together in the platform team of a Stockholm based scale-up called Mentimeter a few years ago. The product was starting to have quality issues because of the quick growth that had been achieved. There were some browser based end-to-end tests written in Cypress, but they only covered a tiny part of the product's surface area, and most developers were too afraid to touch them. So it was decided to invest heavily into quality assurance in the form of end-to-end tests. We decided to switch over to Playwright because it seemed developers got a hang of it much quicker than Cypress. Then we hosted workshops and started pushing hard for product teams to write tests for their different product areas.
To a start, this worked out great. Teams really took to heart writing Playwright tests, and the tests prevented a lot of critical incidents. But fairly soon we started running into a wall. The more tests developers wrote, the longer the test suite took to run, and when the run time of the test suite started reaching over 20 minutes, it was really noticeably slowing down the feedback loop for developers. So we went down a rabbit hole...
We started looking into techniques for speeding up a test suite and looked into partitioning the test suite, sharding, stricter timeouts, and speeding up the application itself in the staging environment. While we got some quick wins doing this, we realized that at the rate the test suite was growing we would soon be back at the same total time to run the suite, and what we were doing was not enough.
About this time Oliver came back saying, "So I've been working on something I think can help", and he introduced an early version of what came to be the foundational ideas behind Endform. If you want to read more about the technical details of how Endform works, you can read more in this blog post, but the gist of it is that we massively parallelize the test suite by running each and every test in a micro-VM of its own, and then merge back the output so we can display the results.
This means we achieved a way of growing our test suite, while decoupling the number of tests to the wall time of running it. Because we ran all tests fully in parallel, we could cap the total runtime of the test suite to that of the longest running test.
The biggest upside of this is that developers can now write as many new tests as they want, without worrying that they'll slow down the test suite. This had a big impact on the engineering culture at Mentimeter which now has a test suite of several hundred tests.
And that, is one of the core value propositions of Endform. Allowing developers to focus on writing more tests instead of worrying about the total run time of the test suite.
Going forward we want to focus on insights into test flakes and failing tests, to truly empower developers to take control over their test suite.
Does this sound interesting to you? Sign up to the waitlist below, or send me an email at example@example.com if you want to participate in early testing.
Signed,
Jakob & Oliver