fix(cache): replay steps on nodes known only by a generated test id
Merged
PI
11d ago+445+445+012345678901234567890123456789-72-72-01234567890123456789
Part of ENG-845.
A flow editor names each node rf__node-<uuid> and mints a new id on every run. When that test id was all a node had, the replay cache recorded it verbatim, so the next run could never find it. A step whose effect included such a node ended in end-mismatch, and a click on one in target-not-found, on every run (REPLAY_STALE under --strict-cache).
testIdShapeTier(packages/e2e/src/cache/relocate.ts) reads a test id that is a node's only identity as a shape: each uuid, hex id, cuid, or recent epoch time in it becomes#(rf__node-#,row-#-delete). Indexed ids such astodo-item-12orrf__node-dndnode_0stay exact.- End check (
packages/e2e/src/cache/anchors.ts): such a node is skipped while a stable anchor exists, like volatile text. When it is the only change, any node of its shape that was not on screen before the replay counts. A vanished one is still checked exactly. - Click targets: the recording counts the nodes of that shape (
position). After an exact miss, replay matches by shape only where a place means something, the same rule as an unnamed control: the node was alone, or it sits in a named row or group. - No
REPLAY_POLICY_VERSIONbump. Exact matching runs first and is unchanged, the shape paths only run where main always missed, and none of the 389 test id descriptors in the committed benchmark recordings has a minted part.
flowchart LR
T[recorded target] --> E{exact match}
E -- found --> A[act]
E -- none --> M{only a minted test id?}
M -- no --> H[hand off]
M -- yes --> P{alone when recorded, or in a named container?}
P -- no --> H
P -- yes --> C{same count of that shape?}
C -- yes --> A
C -- no --> H
What the "add a node" step records:
// main
"endAnchors": [{ "role": "status", "name": "Node added" }, { "role": "box", "testId": "rf__node-712f7aa3-f7cc-4bbf-bf52-1ddf74…" }]
// this branch
"endAnchors": [{ "role": "status", "name": "Node added" }]
Known limits:
- nanoid's alphabet includes
-and_, so its pieces often look like words. 79% of nanoids are recognized and skipped as end anchors, but two nanoids share a shape only about 20% of the time; the rest hand off as on main. - Two nodes of one shape with no named container still hand off: their order is not an identity.
- A click recorded before this release has no count, so it hands off once and re-records.
Verified
Ran it locally: yes
- Scratch testbed project: "Add node" appends
<div data-testid="rf__node-${crypto.randomUUID()}">and sets a status; clicking a node setsNode <n> selected. Built-in agent (openai/gpt-6-luna-fast), one recording run, thene2e run --strict-cache, the same commands on main (ea7cdb51) and this branch:
| Step | main | this branch |
|---|---|---|
| add a node | REPLAY_STALE (end-mismatch): missing box rf__node-712f7aa3-… | replayed in 172 ms, no model call |
| click the only node | REPLAY_STALE (target-not-found) | replayed in 152 ms, no model call |
| click the second of two nodes, no container | REPLAY_STALE (target-not-found) | REPLAY_STALE (target-not-found), on purpose |
- The same recordings replayed against a broken app on this branch: an "Add node" that does nothing fails with
end-mismatch, and an extra node on the canvas withtarget-ambiguous. - Web benchmark
tests-agentwithCI=1 --strict-cache:26 replayed · 2 handed off · 4 missedon main and on this branch, the same six steps (two never recorded, four with a run-time value).