Skip to content

Investigate test history with AI

Connect your coding agent to the Endform MCP server, then ask questions about the test history for the repository you are working in. The agent can compare trends, inspect individual executions, compare a failed attempt step by step with a passing run, and link you to a replay of what it found, without you copying data out of the dashboard.

The conversations below are representative examples. Your results will use your own tests and Endform history.

Example conversationWhat are our flakiest tests?
You

What were our flakiest tests over the last seven days? Look into the worst one and tell me why it flakes.

Tool call:analytics_querypassing test runs with retries, grouped by test, last 7 days
Tool call:samples_queryrecent retried executions of the flakiest test
Tool call:test_attempt_inspectcompare a failed attempt with its passing retry
Tool call:test_attempt_inspectcheck another failed attempt of the same test
Tool call:test_attempt_replay_linkreplay the failed click beside the passing retry
AI response

checkout.spec.ts › saved card › completes payment was the flakiest test this week. It passed in 18 runs, but 6 of those runs needed a retry.

The failed attempts have the same pattern: the test clicks Pay now immediately after the saved card request finishes, but before the form enables the button. The successful retries wait long enough for that state transition. In both failed attempts, the first change from the passing retry is the click on a disabled button, and there is no corresponding POST /payments request. I’ve linked a replay that plays a failed attempt beside its passing retry, opened on that click.

I would replace the fixed delay before the click with an assertion that the button is enabled:

await expect(page.getByRole("button", { name: "Pay now" })).toBeEnabled();
await page.getByRole("button", { name: "Pay now" }).click();
Example conversationWhich tests are slowest?
You

Which individual tests have the biggest effect on our test runtime? Check whether any have become slower recently.

Tool call:analytics_queryaverage test duration grouped by test, last 30 days
Tool call:samples_queryrecent executions of the slowest tests
Tool call:analytics_querydaily duration trend for the leading slow test
AI response

reports.spec.ts › exports a yearly CSV is the largest individual bottleneck at an average of 48 seconds. It was averaging 29 seconds at the start of the month and has climbed steadily since August 8.

The sample runs show that browser setup is stable; almost all of the increase is spent waiting for the export request. organization.spec.ts › deletes a workspace is second at 33 seconds, but its duration has not changed materially.

I would investigate the yearly export endpoint first. Because Endform runs tests in parallel, reducing this longest test is more likely to shorten the overall feedback loop than optimizing several already-fast tests.

Example conversationTriage today's failures
You

Summarize today’s test failures. Group related failures together and tell me what to investigate first.

Tool call:analytics_queryfailed test count grouped by test, last 24 hours
Tool call:samples_querylatest failed executions and their ordered attempts
Tool call:test_attempt_inspectinspect a representative checkout failure
AI response

There are 14 failed test runs across three groups:

  1. Checkout API unavailable: 9 failures. Three checkout tests fail after the same POST /payments 503 response. This is the highest-priority issue because it affects every browser project and retries do not recover.
  2. Profile menu timing: 4 failures. These pass on retry and share a locator timeout. Treat this as one flaky-test issue.
  3. Visual assertion: 1 failure. The settings-page screenshot differs only in WebKit and did not retry.

Start with the payments service or its test environment. The representative attempt shows the UI behaving normally up to the failed API response, so changing the Playwright assertions would only hide the underlying outage.

You can make these questions more specific by naming a branch, commit, Playwright project, test file, title, or time window. The complete historical data tool reference is in Endform MCP server.