
A Playwright suite that ran clean at twenty tests starts dragging at eighty. By a hundred and fifty, it becomes fragile. Tests fail for reasons nobody can trace, and new engineers break flows they never touched. CI stretches past twenty minutes, and every pull request waits behind it.
Most teams reach this point without warning because the habits that kept a small suite manageable stop scaling once it grows. What passed for good practice at twenty tests, a shared login script, a few workers, one flat spec folder, becomes the reason the suite slows down and hides what is actually stable. Few teams plan for this shift. Most stumble into it, then spend months treating a structural issue as a series of individual bugs.
Patching individual failures rarely solves the underlying problem. The suites that hold up under growth all have one thing in common. Someone identified which of three problems they were dealing with, efficiency, isolation, or determinism, before acting on it. Each one calls for a different fix, and applying the wrong one burns real engineering time. This piece walks through all three, so the fix matches the problem.
Why the usual best practices lists don’t help teams at scale
Most best practices lists treat every recommendation as equally urgent, whether it’s using page objects, isolating state, or mocking the network. In practice, pain at scale is not distributed evenly. Three problems account for most of it. CI time that keeps climbing until every pull request waits behind it, tests that quietly depend on the order they run in, and tests that fail in CI then pass on re-run with nothing changed.
A review of why Playwright setups fail found that login flows alone consume more than half of total suite time in a simple setup. Logging in is not complicated, but repeating it in every test is what gets expensive.
Scale, in this context, has nothing to do with reaching some large test count. It starts when a suite needs coordination across multiple engineers and runs long enough to delay pull requests. That threshold is much lower than most teams expect, which is why these problems appear earlier than typical scaling advice suggests.
Before going further, one question identifies the right place to start. Is the suite slow, does it fail differently depending on the order tests run in, or does it pass and fail without any code changes? The answer points to one of three categories. Speed is the most visible complaint, so that is where this starts.
Category 1: Efficiency (the suite is too slow)
Efficiency problems often show up as login running before every test, a CI pipeline that takes twenty minutes to report back, browser reinstalls eating ninety seconds on every push, and tests queuing behind each other while half the machine’s cores sit idle. The goal here is to cut the time a suite spends on work that isn’t testing anything. Most of that waste comes from three places: auth overhead, CI setup, and execution that runs serial when it could run in parallel.
Authenticate once with storageState, not before every test
UI-based login before each test adds unnecessary authentication overhead across
the entire suite. storageState removes that cost. The login runs once, and the
resulting cookies, local storage, and IndexedDB are saved to a JSON file that
other tests load to start in an authenticated state. Apps that keep auth tokens
in IndexedDB, as Firebase Authentication does, need to include IndexedDB in the
saved state because it is not captured by default. Pass indexedDB: true to
storageState, an option Playwright added in v1.51.
The usual approach is a dedicated setup project that runs first, with the rest
of the suite declared as dependent on it in playwright.config.ts. Because the
file contains live session tokens, playwright/.auth belongs in .gitignore,
not version control. The exception is tests that cover the login flow, which
still need to log in through the UI on every run.
// tests/auth.setup.ts
import { test as setup } from "@playwright/test";
const authFile = "playwright/.auth/user.json";
setup("authenticate", async ({ page }) => {
await page.goto("/login");
await page.getByLabel("Email").fill(process.env.TEST_USER_EMAIL!);
await page.getByLabel("Password").fill(process.env.TEST_USER_PASSWORD!);
await page.getByRole("button", { name: "Sign in" }).click();
await page.waitForURL("**/dashboard");
await page.context().storageState({ path: authFile });
});
// playwright.config.ts
import { defineConfig, devices } from "@playwright/test";
export default defineConfig({
projects: [
{ name: "setup", testMatch: /.*\.setup\.ts/ },
{
name: "chromium",
use: {
...devices["Desktop Chrome"],
storageState: "playwright/.auth/user.json",
},
dependencies: ["setup"],
},
],
});
Manage multi-role auth with separate state files
When a suite covers multiple user roles, such as an admin, standard user, or viewer, each role needs its own saved session. Extend the setup by creating one setup project per role, storing each session in its own state file, and configuring each group of tests to load the state file it requires.
// playwright.config.ts
export default defineConfig({
projects: [
{ name: "setup-admin", testMatch: /admin\.setup\.ts/ },
{ name: "setup-viewer", testMatch: /viewer\.setup\.ts/ },
{
name: "admin-tests",
testMatch: /admin\/.*\.spec\.ts/,
use: { storageState: "playwright/.auth/admin.json" },
dependencies: ["setup-admin"],
},
{
name: "viewer-tests",
testMatch: /viewer\/.*\.spec\.ts/,
use: { storageState: "playwright/.auth/viewer.json" },
dependencies: ["setup-viewer"],
},
],
});
Use fresh accounts when tests modify server-side state
Tests that create, edit, or delete data on the server need more care than read-only tests. Shared login sessions can become a problem here because multiple tests may modify the same account at the same time. For tests that change server-side state, the safest default is a fresh account created specifically for that test. It adds some setup time, but it removes shared state as a variable completely.
A fixture is the right home for this setup. It ties the account’s lifecycle to
the tests that request it: the account is created before the test runs, handed
to the test through await use(), and deleted afterward, even when the test
fails.
Setup and teardown live in one function, so there is no separate cleanup block to forget, and tests that never touch server-side state never pay the provisioning cost. If creating accounts is slow, provisioning them through an API endpoint rather than the UI or direct database access usually keeps the cost small.
// fixtures/fresh-user.ts
// A new account created and torn down for every test
import { test as base } from "@playwright/test";
export const test = base.extend<{ freshUser: string }>({
freshUser: async ({}, use) => {
const userName = `user-${Date.now()}-${Math.random()}`;
await createUserInTestDatabase(userName);
await use(userName);
await deleteUserFromTestDatabase(userName);
},
});
Cache browser binaries with the OS-deps conditional on cache hit
Repeatedly reinstalling browsers during CI wastes time when the binaries have
not changed. Caching them with a key tied to package-lock.json keeps the cache
valid until dependencies change and invalidates it when they do.
One detail is easy to miss. Even on a cache hit, npx playwright install-deps
still needs to run because it installs OS-level system libraries that the cache
does not include. Skipping that step causes dependency errors that look
unrelated to caching. The steps below slot into an existing workflow after
checkout and dependency installation, which are trimmed here to keep the focus
on caching. A closer look at Playwright on GitHub
Actions covers the full
caching setup in more depth.
- name: Cache browsers
id: playwright-cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ hashFiles('**/package-lock.json') }}
- name: Install Playwright with browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install OS dependencies only
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps
Run Chromium on PRs and the full browser matrix on merge to main
Run Chromium on every pull request and reserve the full browser matrix for
merges to main. Chrome, Edge, and other Chromium-based browsers share the same
engine and account for more than three-quarters of global web traffic, according
to StatCounter’s browser share
data. Testing that engine on
every PR catches most issues users will encounter while cutting CI time roughly
in half.
The tradeoff is coverage because a Safari- or Firefox-specific bug can still
pass a PR and only surface when the full matrix runs at merge. For most teams,
that is a reasonable balance between faster feedback on every PR and full
browser coverage before anything ships. Since the matrix passes each browser
name to --project, playwright.config.ts needs a project defined for each
entry: chromium, firefox, and webkit.
jobs:
test-pr:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- run: npx playwright test --project=chromium
test-main:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
strategy:
matrix:
browser: [chromium, firefox, webkit]
steps:
- run: npx playwright test --project=${{ matrix.browser }}
Set worker count based on your runner, not a hardcoded default
Leaving workers undefined uses half the machine’s logical CPU cores, but the
right value depends on the runner and the workload. Some suites perform better
at 75 or even 100 percent CPU utilization, while others slow down as contention
increases. Only benchmarking on the runner where tests actually execute reveals
the right balance.
Workers and fullyParallel improve how efficiently a suite uses available
cores, but they do not increase the number of cores themselves. Long end-to-end
flows reach their limit sooner than lightweight checks, and once measurements
show a single runner cannot keep up, the next step is usually scaling
infrastructure rather than continuing to tune workers.
// playwright.config.ts
export default defineConfig({
workers: process.env.CI ? "75%" : undefined,
});
Tag tests by speed and criticality and split CI jobs accordingly
Not every test needs to run on every pull request. Tag tests as @smoke,
@regression, or @slow, then run only the smoke suite on PRs while reserving
regression and slow tests for merges to main or nightly runs. This keeps
feedback fast while still running broader coverage where it matters. The same
tagging pattern shown below for @smoke applies directly to @regression and
@slow.
test("user can complete checkout", { tag: "@smoke" }, async ({ page }) => {
// ...
});
# PR pipeline: fast feedback only
npx playwright test --grep @smoke
# Merge to main: full regression
npx playwright test --grep-invert @smoke
Auth overhead, CI setup, and execution parallelism account for most efficiency gains, and reducing CI cycle time usually comes from several smaller improvements working together rather than one major change. A faster suite does not automatically mean a more reliable one, though. The next problem shows up once tests start interfering with each other.
Category 2: Isolation (tests interfere with each other)
Tests that pass when run alone but fail during parallel execution usually point
to an isolation problem. Results may change depending on test order, whether the
suite runs serially or a single test is isolated with test.only. That usually
points to tests depending on state another test created or left behind rather
than a flaky test. Isolation practices ensure any test can run on any worker
without relying on state from another test.
Playwright handles part of this automatically. Each test gets its own browser context with separate cookies, local storage, and session storage, so tests do not accidentally share login sessions or cached browser values. What it does not isolate is anything outside that context, including database records, server-side session state, files written to disk, or external API state. Those require explicit teardown or an API-level reset because a fresh browser context does not affect them.
Set up test state via API calls, not UI flows
Creating test state through the UI is slower and less reliable than using an API call because it introduces timing dependencies before the test even starts. API-level setup makes parallelism easier because tests no longer depend on slow UI flows to reach their starting state.
// Before: UI setup introduces timing risk before the test begins
await page.goto("/admin");
await page.getByLabel("Product name").fill("Test Product");
await page.getByRole("button", { name: "Create product" }).click();
// After: state exists before the test touches the browser.
// request is Playwright's built-in fixture, destructured alongside page:
// test('...', async ({ page, request }) => { ... })
const response = await request.post("/api/products", {
data: { name: "Test Product" },
});
const product = await response.json();
Use fixtures for teardown, not afterEach hooks
Fixtures tie setup and teardown to the tests that request them. afterEach runs
after every test in a file, even when that cleanup is not needed, and it
separates one logical concern into different blocks that can drift apart as the
file grows. With fixtures, setup runs before await use() and cleanup runs
after it in the same function.
The bigger risk with afterEach is not that Playwright skips it on failure,
because it does not. The risk is teams adding their own conditions inside the
hook, checking test status and assuming a failed test left nothing to clean up.
That assumption is often wrong.
// Fixture: setup and teardown live together, runs only when requested
export const test = base.extend<{ testUser: string }>({
testUser: async ({}, use) => {
const userId = await createUser();
await use(userId);
await deleteUser(userId);
},
});
// Anti-pattern: shown for comparison, avoid this approach.
// Cleanup lives in a separate block, easy to accidentally skip
let userId: string;
test.beforeEach(async () => {
userId = await createUser();
});
test.afterEach(async ({}, testInfo) => {
if (testInfo.status !== "passed") return; // often the wrong assumption
await deleteUser(userId);
});
Mock external services selectively with page.route()
Mocking replaces a real service response with a controlled one, which helps when a third-party dependency introduces instability or latency that hides what the test is verifying. It becomes the wrong choice when that integration is what the test needs to validate, since a mocked response cannot confirm the real service still works. Not every third-party call belongs behind a stub, and treating mocking as a default instead of a deliberate decision trades coverage for convenience.
await page.route("**/api/shipping-rates", (route) =>
route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({ rate: 4.99, carrier: "Standard" }),
}),
);
Use test.describe to manage shared state within a group
Tests that share the same context can scope that setup to a test.describe
block instead of relying on a global beforeAll that couples every test in the
file to the state created first.
test.describe("checkout with saved card", () => {
let cardId: string;
test.beforeAll(async () => {
cardId = await createSavedCard();
});
test("applies saved card at checkout", async ({ page }) => {
// uses cardId, scoped to this group only
});
});
Use test.only to diagnose isolation failures before touching the suite
A test that passes alone but fails during the full run usually points to an
isolation problem, and confirming that before changing code saves time. Run the
suspect test with test.only locally or use --grep in CI, where test.only
should never ship, to prove the dependency exists before deciding how to fix it.
Setting forbidOnly: !!process.env.CI in the config
enforces this automatically, failing the CI run if a test.only slips through.
Isolation is what makes a suite trustworthy under real conditions, allowing tests to run in any order, on any worker, without leftover state. A suite can be isolated correctly and still fail for other reasons, which leads into the next category.
Category 3: Determinism (tests pass and fail inconsistently)
Tests that fail in CI, then pass on re-run without a code change, slowly erode trust in the pipeline while retries accumulate and hide the underlying problem. If isolation is already solid and failures remain inconsistent, the cause often lies elsewhere in the system under test, which is why no single fix resolves it outright. The practices below reduce the surface area, but they do not eliminate it.
Two baseline expectations belong here briefly rather than in full. Locators
built on getByRole survive markup changes that would break CSS or XPath
selectors because they reflect how users find elements rather than the current
DOM structure.
// Fragile: breaks on any markup refactor
await page.locator(".btn-primary.submit-form").click();
// Resilient: survives markup changes
await page.getByRole("button", { name: "Submit" }).click();
waitForTimeout stays banned in code review, not discouraged, because a fixed
wait advances the test based on elapsed time rather than application state.
// Machine-dependent, moves on regardless of app state
await page.waitForTimeout(2000);
// Retries automatically until the condition is true
await expect(page.getByText("Order confirmed")).toBeVisible();
Enable trace on first retry, or retain-on-failure
The tradeoff between these settings is trace coverage versus overhead. trace: 'on-first-retry' skips tracing on the initial attempt, so tests that pass
cleanly never incur that cost. The downside appears when a retry passes. The
only trace available is from the successful run rather than the original
failure.
retain-on-failure traces every attempt and keeps the file only when a test
fails. That guarantees a trace of the original failure, but it adds tracing
overhead to every run, whether a test is retried or not. Teams running with few
or no retries need it regardless, since on-first-retry never triggers without
a retry to attach to.
export default defineConfig({
use: {
// Skips tracing on a clean first pass, but a passing retry
// leaves no trace of what actually failed the first time
trace: "on-first-retry",
// Traces every attempt, guaranteeing a trace of the real failure
// trace: 'retain-on-failure',
},
});
Either option works, and traces open directly in trace.playwright.dev without any local installation.
Treat flaky tests as defects, use retries as a safety net
Retries are useful for the occasional test failure, but they stop helping when flakiness becomes a pattern. Once one to two percent of a suite is flaky, the problem is systemic, and retries hide it rather than fix it. Confirmed flaky tests deserve a logged defect, an owner, and a deadline, with retries reserved for a small number of edge cases instead of replacing the investigation.
The fastest way to confirm a suspected flaky test is to run it twenty times
locally with --repeat-each=20 instead of waiting on CI. A companion piece on
diagnosing and fixing flaky Playwright
tests covers the full
workflow, including the quarantine pattern for tests that cannot be resolved
immediately.
Determinism is the last of the three categories, and fixing all three rarely happens at once. The next section covers what to fix first.
What to fix first
The right priority depends on the problems already showing up in the suite. Trying to tackle efficiency, isolation, and determinism at the same time usually spreads effort too thin.
Speed is usually the first complaint teams notice, but it isn’t always the right place to start. In smaller suites, apparent slowness often traces back to auth overhead, and apparent flakiness often traces back to an isolation gap, which is why those areas usually come first no matter which symptom got noticed initially.
When fixtures start duplicating interaction logic for the same page, structure becomes urgent regardless of test count. As suites grow, CI time often becomes the biggest cost, making caching and browser-scope optimizations worth the effort. The table below maps those signals to the areas that typically deserve attention.
| Suite size | Team size | Highest-leverage category |
|---|---|---|
| Under 50 tests | Any | Isolation and auth overhead |
| 50 to 150 tests | 5+ engineers | Isolation first, then structure (fixtures and page objects) |
| 150 to 300 tests | 10+ engineers | Efficiency (CI caching, browser scope) |
| 300+ tests | Multiple teams | Infrastructure, once all three are genuinely addressed |
A small team building a complex multi-role product may need multi-role auth at thirty tests, while a single engineer running two hundred tests may never need a page object. The priority is matching the fix to the problem already slowing the suite down instead of applying every recommendation at once, and trusting the other two categories to wait their turn until the suite’s own behavior says otherwise.
Beyond three hundred tests, infrastructure often becomes the limiting factor. Efficiency, isolation, and determinism solve the problems they are designed to address, but they do not change how much work a single machine can execute. If a suite is already well isolated, deterministic, and tuned, yet CI remains too slow, the machine itself is what’s holding it back, and infrastructure is the last resort left to consider.
Last resort when the three categories are not enough
If efficiency, determinism, and isolation have genuinely been addressed, and the suite is still slowing PR feedback or producing unreliable results, a single machine may no longer be enough to execute the workload efficiently.
Sharding is the standard native CI approach. Splitting a suite across multiple parallel CI jobs reduces wall-clock time, but it comes with additional upkeep. That includes extra CI minutes, a merge-reports step to combine results, and a shard count that needs revisiting as the suite grows. That tradeoff is worth making when the need is clear rather than becoming the first response to a slow pipeline.
A managed runner removes most of the infrastructure overhead that comes with sharding. Running every test simultaneously on its own isolated machine skips configuration changes, shard tuning, and report merging entirely. Runtime settles around the duration of the slowest test in the suite instead of the number of shards configured in CI.
Neither approach belongs at the start. Reaching for sharding or a managed runner before fixing auth overhead, isolation gaps, or determinism issues means spending more resources while those problems remain, and they’re cheaper to solve before infrastructure is added.
If the suite is on solid footing and execution capacity is the remaining constraint, Endform runs the entire Playwright suite in parallel with a one-line change:
npx endform test
Fix the suite first, and scaling execution becomes a configuration decision instead of an infrastructure one.
Wrapping up
Matching the fix to whatever is actually causing the pain, then building from there, beats trying to tackle efficiency, isolation, and determinism all at once. Solid auth and real isolation make efficiency improvements worthwhile, while optimizing execution too early only makes existing problems run faster.
As coverage grows, the same thinking still applies. Fix the problem that’s showing up now rather than jumping ahead to the next one. If the suite is already stable, isolated, and running efficiently, yet execution time is still a problem, Endform runs the whole suite in parallel with no configuration changes.
Whatever the suite’s size, the discipline stays the same. Identify the problem first, then choose the fix that addresses it.