feat(html-reporter): write the run as one HTML page

Merged
KR
12d ago
+3487-5

e2e reports a run as terminal output, JSON, JUnit, markdown, or a pull request comment. None of them shows a test's steps next to its screenshots and video, and the Playwright and Maestro migration guides list reporter: 'html' as missing.

What changed

  • New package @e2e-dev/html-reporter. htmlReporter() writes <output>/report.html, one page that opens from disk with no server:

    import { htmlReporter } from '@e2e-dev/html-reporter';
    
    export default { targets, reporters: ['list', htmlReporter()] } satisfies E2EConfig;
    
  • The page is a React app in packages/html-reporter/app/. Vite and vite-plugin-singlefile build it into one HTML file with the fonts and icons inlined. The reporter (packages/html-reporter/src/reporter.ts) fills that template with the report-1 document, the media, and the test-file lines around each step.

  • media picks how screenshots and videos reach the page: 'link' (default, relative paths into results/), 'embed' (one file to send), or 'none'.

  • The page shows the tests as a tree (file, describe blocks, tests, failures first) with status tabs and search. Each test shows its attempts, the error with expected and received values diffed, the code frame, its steps synced to the video, agent step details, and a copy prompt button that puts the failure on the clipboard for a coding agent.

  • Core: FinishedRun.redact(text) (packages/e2e/src/types.ts). It masks configured secret values in text a reporter brings from outside the report, built from the same ledger as TestFilterContext.redact. The page uses it on the test-file lines it embeds. This needs the next e2e release, so the reporter's e2e peer floor is >=0.23.1.

  • media: 'embed' leaves out downloads the runner could not scan for secrets (redaction: 'incomplete'), and download links save the file instead of opening it.

  • Fonts are Geist and JetBrains Mono (OFL 1.1) and the icons are Lucide (ISC). The licence texts ship in the package and are listed in NOTICE.

flowchart LR
  A[run finished] --> B["writeReport(run)"]
  B --> C[report-1 document]
  B --> D["media: link / embed / none"]
  B --> E["source lines, run.redact"]
  C & D & E --> F["dist/app/index.html template"]
  F --> G[".e2e/report.html"]

Not covered yet (documented in docs/reference/reporters.mdx): after --last-failed the page shows only the rerun's tests, and under e2e explore it shows no findings.

Verified

Ran it locally: yes

  • Testbed, built CLI, with htmlReporter() on the todos and forms suites plus one deliberately failing test, run with --repeat-each 2:

    HTML report  .e2e-html/report-link.html
    
         Failed  × |web| tests-html-scratch/broken.e2e.ts:10 > scratch > fails on a wrong remaining count
                   ASSERTION_FAILED: expect.toHaveText failed
                 × |web| tests-html-scratch/broken.e2e.ts:5 > scratch > fails on a wrong remaining count (repeat #1)
    
  • The testbed's reporter stress suite (hostile titles, a 20 KiB message, timeouts, flakes, skips), plus a test that fills the configured admin password and then fails, with media: 'embed'. Every one of the 16 test pages opened in Chromium with no page errors and no dialogs. The password value is nowhere in report.html, and the frame shows fill('<secret:admin.password>').

  • Pages from the web benchmark agent suite and the mobile benchmark on an iOS simulator, in both themes and at phone width.

Run listFailed test
run listfailed test
Light themeReporter stress suitePhone recording
lightstressmobile player

Review

  • A fresh-context review found 9 issues. Fixed: unscanned downloads embedded in a page meant for sending, --repeat-each rows that looked identical, download links that opened in the report's origin, emoji splitting in the expected/received diff, and no runner-level test for run.redact (packages/e2e/tests/integration/reporters.test.ts, which fails when redact passes text through).

  • Declined:

    • Folding --last-failed and explore findings into the page: documented above as a follow-up.
    • Streaming 'embed' past V8's string limit (about 400 MB of media): the docs already warn that device video makes an embedded page large.
    • A test against the built template: the root test script builds first, and embedPayload already throws when the placeholder is not there exactly once.
  • Cubic: fixed 15 threads (copied-prompt fence, cache-only step details, headline counted from the listed tests, the reporter abort signal, video part clock, serial member error fallback, a doubled screen-reader status, and docs wording). Two threads about the secrets invariant are left open for a human: symlinked artifacts, and multiline secrets in code frames.

Reviewed with sol-loop: 2 rounds with gpt-6.1-sol. Sol verdict: approved (before the fresh-context fixes above).

Closes ENG-1079.

Written with Claude Code (claude-opus-5-5).