fix(cache): replay same-app navigations where the app runs now

Merged
TH
18d ago
+175-21

Fixes ENG-999.

  • The agent's navigate recorded its argument verbatim (packages/e2e/src/agent/action-dispatcher.ts), so an absolute URL on the app's origin kept the recording run's port. startPath and endPath were already app-relative (appLocation), so a replay on a fresh port with a stable app.identity passed the start check, then opened the dead origin and stopped with action-failed: the agent redid the step, or --strict-cache failed it with REPLAY_STALE.
  • appNavigation (packages/e2e/src/cache/route.ts) cuts the app's origin off such a URL and keeps the rest as typed, so redaction and unique() slots read the same text as before.
  • Kept as given: another origin, a relative reference, a URL with credentials, and any rest that would not open the same URL on the recorded origin (//host/x, a backslash).
  • Replay is unchanged: it already resolves a path against the current app.url. Entries recorded before this still replay on their own origin and keep the old address until re-recorded. A deployment under another base path still misses, as startPath does.
  • Step text for such a navigation reads navigate to "/todos" instead of the full URL.
  • The links the agent reads use the same rule (renderHref and treeHref in packages/e2e/src/agent/observation.ts): an app link http://127.0.0.1:3000//settings showed as href="//settings", which a navigate resolves to the host settings; it now shows whole. In Chromium on a page with that link, the agent's line read href="//settings" before and the full URL now.
 "actions": [{
   "name": "navigate",
-  "summary": "navigate to \"http://127.0.0.1:4291/todos\"",
-  "url": "http://127.0.0.1:4291/todos"
+  "summary": "navigate to \"/todos\"",
+  "url": "/todos"
 }],
 "startPath": "/",
 "endPath": "/todos"

Verified

Ran it locally: yes

  • Built-in agent, live (openai/gpt-6-luna-fast): a scratch app on port 0 with no app.identity shows an invite link as text, and the step is "Open the invite link shown on the page". The model called navigate with {"url":"http://127.0.0.1:50034/invite/team-blue"} (--ai-trace). Then a strict replay on a new port:
mainthis branch
recorded navigate.urlhttp://127.0.0.1:49979/invite/team-blue/invite/team-blue
replay on a new port, --strict-cache✗ REPLAY_STALE, ERR_CONNECTION_REFUSED✓ Cache 1 replayed, no model call
  • testbed, a scratch config with a stable app.identity, the port from PORT, and a deterministic executor that navigates to the Todos link's href from observe({ tree: true }), an absolute URL on the app. Recorded on port 4291, then replayed on 4292 with --strict-cache while 4291 was down.
mainthis branch
recorded navigate.urlhttp://127.0.0.1:4291/todos/todos
replay on 4292, --strict-cache✗ REPLAY_STALE✓ Cache 1 replayed

main:

× opens todos through the main nav > agent.act "Open the "Todos" page from the main nav" 115ms failed
REPLAY_STALE: the recording of this step no longer replays (action-failed), and cache.strict hands no step to the agent; ...
  - cache: no replay: a recorded action failed; at action 1 of 1, navigate to "http://127.0.0.1:4291/todos": navigation failed: page.goto: net::ERR_CONNECTION_REFUSED
  • An entry holding the absolute URL, replayed on its own port with --strict-cache: Cache 1 replayed, and the file is unchanged.
  • The new integration test in packages/e2e/tests/integration/agent-trace-cache.test.ts (two fixture apps sharing one app.identity) fails on the main build with url: "http://127.0.0.1:61091/about", and without that check with the reported REPLAY_STALE (action-failed, ERR_CONNECTION_REFUSED). It passes here.