fix(web): omit Next.js route announcer from observations

Merged
ET
13d ago
+135-2

Fixes #1057.

What and why

  • Next.js App Router's <next-route-announcer> echoes document.title into a shadow <div id="__next-route-announcer__" role="alert"> on a hydration schedule of its own. The web engine recorded it as an outcome alert, so an identical navigation replayed stale (REPLAY_STALE, end-mismatch) depending on whether the echo had landed.
  • Fix: next-route-announcer joins SKIP_TAGS in packages/web/src/in-page/read-semantics.ts. The skip runs before the walk descends into light children or a shadow root, so the whole announcer subtree is never observed. A direct locator still reads it.
  • An app's own role="alert", even with the title's text, is still observed and still stops a replay that raised it.
  • No REPLAY_POLICY_VERSION bump: older entries that recorded the echo re-record on the next read-write run.
  • Tests: a web integration test (announcer empty and populated yield no alert node, app alert kept, locator still reads the text) and a three-row record/replay matrix in packages/e2e against a fixture page with the real announcer shape.
  • Maintainer follow-up commit: collapsed the per-node check into SKIP_TAGS and trimmed the tests. The author's commit and credit stay as they were.

Verified

Ran it locally: yes

pnpm --filter @e2e-dev/web exec vitest run tests/integration/route-announcer.test.ts tests/integration/crosscheck.test.ts tests/integration/conformance.test.ts
 Test Files  3 passed (3)
      Tests  19 passed | 1 skipped (20)

pnpm --filter e2e exec vitest run tests/integration/route-announcer-cache.test.ts
 Test Files  1 passed (1)
      Tests  3 passed (3)

Same route-announcer-cache.test.ts with the SKIP_TAGS entry removed (main's reader):

 ❯ |integration| tests/integration/route-announcer-cache.test.ts (3 tests | 3 failed)
   × records announced, replays silent, passes=true
   × records silent, replays announced, passes=true
   × records announced, replays app-alert, passes=false
Casemain readerthis branch
record announced, replay silentREPLAY_STALEreplayed
record silent, replay announcedREPLAY_STALEreplayed
record announced, replay app alert with title textreplayed (echo masks the new alert)REPLAY_STALE, end-mismatch

pnpm check passed (lint, dead code, typecheck, docs). Built CLI:

cd apps/testbed && node node_modules/e2e/dist/cli/bin.js run tests/todos.e2e.ts
 Test Files  1 passed (1)
      Tests  7 passed (7)

cd apps/web-benchmark && pnpm run build && node node_modules/e2e/dist/cli/bin.js run tests/login-form.e2e.ts
 Test Files  1 passed (1)
      Tests  4 passed (4)

node node_modules/e2e/dist/cli/bin.js run --config e2e.agent.config.ts   (no AI_GATEWAY_API_KEY)
      Tests  7 failed | 26 passed (33)
      Cache  26 replayed · 2 handed off · 4 missed

The 7 agent failures are MODEL_PROVIDER_FAILED on steps that need a model: 4 judgment steps with no entry, 2 hand-offs and 2 misses on recordings that type a run-time value. None is an end-mismatch. On the built Next.js 16.3.8 benchmark app, a capture after a client-side route change with the announcer populated with the title lists 173 nodes and zero alert nodes while next-route-announcer's shadow div reads e2e Web Benchmark.

Not run: a live model recording.

Checklist

  • Changeset for @e2e-dev/web.
  • docs/cache.mdx and skills/e2e/references/agent.md updated.
  • Breaking change: not applicable.