#583 test(web): cross-check the tree against Chrome and Playwright

What and why

• Most recent web engine fixes (#549, #550, #551, #553, #560) began as a hand-written repro showing the tree disagreeing with Playwright or Chrome. This check finds that class without anyone writing the repro: every node of the engine's tree is compared with two oracles on the same live page.
  • Chrome: Accessibility.getFullAXTree over CDP. Role, name, checked, disabled, expanded, selected, pressed, heading level, plus interactive nodes Chrome exposes that the tree left out (missing) and tree nodes Chrome has no node for (unknown).
  • Playwright: getByRole(role, { name, exact }) must resolve to the same element, the round trip #553 did for 15 hand-picked cases.
• Nodes are matched by tagging each element the engine published and resolving Chrome's backendDOMNodeIds back to the tag, so shadow roots match too. packages/web/tests/crosscheck/crosscheck.ts.
• Two corpora: ten fixture pages, one per family of HTML and ARIA semantics (inputs, buttons, checked, disabled, expanded, landmarks, tables, names, hidden, shadow), under pnpm test, and every web-benchmark scenario in benchmark.yml after the deterministic suite.
• Known disagreements are committed in tests/crosscheck/expected.txt, one line each with a reason, the way Playwright keeps its BiDi expectations. A new disagreement fails, and so does a listed one that stopped happening. E2E_CROSSCHECK_UPDATE=1 rewrites the file keeping reasons and marks new lines TODO, which fails until someone writes one.
• Deliberate vocabulary choices (a listitem, status, or alert named from content, box, an editor host as textbox, Chrome's Iframe and ColorWell) are policies at the top of crosscheck.ts, each with its reason. Without them the file had 113 lines. Now it has 9.

flowchart LR
  A[captureDocument] --> B[tag each element<br/>data-e2e-crosscheck=id]
  B --> C[Chrome AX tree<br/>backendDOMNodeId to tag]
  B --> D[getByRole role, name, exact]
  C --> E[disagreements]
  D --> E
  E --> F{in expected.txt?}
  F -- new --> G[fail: unexpected]
  F -- listed, not seen --> H[fail: stale]

What it found

Page	Disagreement	Read
inputs, control-inventory, file-round-trip	input type=file reads as textbox; Chrome and Playwright read button "Choose File"	bug, follow-up
hidden content	an inert button is listed; Chrome drops it (Playwright lists it too)	bug, follow-up: the agent can aim at a control that takes no input
disabled states	a disabled fieldset group is itself disabled; both oracles mark only its controls	kept, reason in the file
disabled states	an option of a disabled select is selected; Chrome drops it, Playwright agrees with the tree	kept, Chrome quirk

Negative control: dropping checkbox from NAME_FROM_CONTENT_ROLES (the #553 bug) fails the run with fixture: checked states | chrome name | checkbox "" | ours | theirs Custom agree.

Checklist

• Changeset added (pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only. Tests and CI only.
• Breaking change: title carries !, the changeset body names what breaks and what replaces it, and the deprecation warning landed a window earlier (stability policy in CONTRIBUTING.md).
• Docs updated in the same PR: the docs/**/*.mdx page for the behavior, and skills/e2e/ if the skill describes it. AGENTS.md; no product behavior changes.
• Contract change: emitted .d.ts reviewed, tests/types/sdk-types.ts updated; wire change edits the schema and both fixtures.
• Engine contract change (e2e/engine): changesets for e2e, @e2edev/web, and @e2edev/mobile.

---