fix(cache): name the end-state checks that failed in end-mismatch details
Merged
HE
14d ago+176+176+012345678901234567890123456789-43-43-01234567890123456789
Refs #1057 (the diagnostics part; the route announcer itself is not touched here)
What and why
- On an
end-mismatch,step.cache.detailprintedendAnchors.slice(0, 2): the first recorded anchors, not the ones that failed. In #1057 it namednode "Acme Notes", which was on screen, next to the announcer alert that was not. deltaHoldsbecomesdeltaMismatch(packages/e2e/src/cache/anchors.ts). It runs the same checks and returns what failed: appeared anchors that are missing, gone anchors still present, and alerts the recording never saw.undefinedmeans the delta holds.endStateMatches(packages/e2e/src/agent/step-cache.ts) keeps the mismatch from the last look on the recorded route. It also reports whendeltaEvidencedfailed, so an outcome that was already on screen is no longer reported as "did not show".- The detail joins its parts with commas.
cacheLinealready uses;between its own parts. - When the end route was never read, the detail still falls back to
expected <first anchors>. - Docs: one sentence in
docs/cache.mdxnext to the existingstep.cache.detailexample. Patch changeset fore2e.
| Case | main | this branch |
|---|---|---|
| announcer alert recorded, missing on replay (#1057) | ...did not show on /customers: expected alert "Acme Notes", status "Marker" text="saved" | ...did not show on /customers: missing alert "Acme Notes" |
| replay raised an alert the recording never saw | ...: expected status "Marker" text="saved" (which was on screen) | ...: unrecorded alert "Card declined" |
| gone anchor still there, 3 anchors missing | ...: expected link "Alpha", link "Beta" | ...: missing link "Alpha" and link "Beta" (+1 more), still showing status "Marker" text="saved" |
| outcome already on screen before the replay acted | ...did not show on /customers: expected status "Marker" text="saved" | ...the recorded outcome already showed on /customers before the replay acted there, so nothing proves the replay produced it |
The root cause in #1057 is a framework live region that echoes document.title and gets recorded as an anchor. That is a matching decision, so I left it out of this PR. One option is to treat an alert/status whose text equals the document title as navigation noise on steps that changed route. I can follow up once you pick a direction.
Verified
Ran it locally: no - I did not run the built CLI. Reproducing the announcer race needs a recorded run against a Next.js app with a model. Each row above is a StepTraceSession replay with that anchor shape in tests/unit/step-cache.test.ts. The "main" column is the output of the same tests with packages/e2e/src reverted to main.
Checklist
- Changeset added (
pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only. - Breaking change: title carries
!, the changeset body names what breaks and what replaces it, and the deprecation warning landed a window earlier (stability policy in CONTRIBUTING.md). - Docs updated in the same PR: the
docs/**/*.mdxpage for the behavior, andskills/e2e/if the skill describes it. - Contract change: emitted
.d.tsreviewed,tests/types/sdk-types.tsupdated; wire change edits the schema and both fixtures. - Engine contract change (
e2e/engine): changesets fore2e,@e2e-dev/web, and@e2e-dev/mobile.