fix(web): answer native dialogs an agent step's action opens
OK
10d ago+549+549+012345678901234567890123456789-43-43-01234567890123456789
An agent never sees a native alert, confirm, prompt, or leave-page dialog: they sit outside the page. With no browser.onDialog handler, the web engine dismissed each one and failed the next operation with INVALID_STATE. So an agent step could not leave a form with unsaved changes or confirm a delete, live or from the replay cache.
DialogRouterasks its surface whether an agent step made the newest action (packages/web/src/dialogs.ts). If so, a dialog no handler takes is answered: alerts, confirms, and leave-page dialogs are accepted, and a prompt is dismissed, since the agent never sees its text. The app log gets onesystemline, such asconfirm dialog "Delete this label?" accepted for the agent.- Ownership moves only on actions.
PlaywrightSurface.guardAction()(packages/web/src/surface.ts) wraps every operation that acts: navigation, input,browser.evaluate, keyboard, mouse,browser.setViewport,browser.emulateMedia. It claims the dialog once the operation runs, so one refused by a pending error claims nothing. Reads (observe,locate, screenshots) and a route handler'sroute.fetchnever take a dialog. So a test's own late dialog still fails its next step while an agent judgment polls, and a test route handler fetching mid-action can't steal the agent's dialog. OperationContext.agentStep(new, optional) marks an agent step's operations, replays included (packages/e2e/src/agent/phases.ts). Replayed actions keeporigin: 'test'for settling, so without it a recorded dialog step failed under--strict-cache. It crosses the engine protocol. An older core omits it and the engine falls back toorigin.- A dialog a test's own step opens behaves as before: dismissed, and the next step fails with
INVALID_STATE. A registered handler always wins. docs/reference/web.mdxsaid an unhandled dialog throwsAPP_NOT_OPEN. The code throwsINVALID_STATE, so the table now says that.- The web benchmark gets an agent scenario on
/e/deferred-dialog, whose confirm opens 900 ms after the click, with its committed recording.
Why
In 201 e2e explore runs on 0.22.0 against a self-hosted Gitea (their report.json and AI traces):
- 226 agent observations and actions failed with
unhandled beforeunload dialog. - 114
navigatecalls failed withnet::ERR_ABORTED. 113 of them were the dismissed leave-page dialog. - 59 steps ended blocked. The model saw only
Action failed: unhandled beforeunload dialog:and retriedbackuntil the step timed out.
For review
- Confirms are accepted. The agent's action asked for the destructive step and the agent can't see the dialog to decline it. That also removes the app's last guard if an explorer clicks "Delete workspace" against its instructions. The alternative is to dismiss confirms, tell the model the text, and give it an explicit way to accept, which needs a model-visible notice channel first.
- The model and MCP callers don't see the app log line. An MCP action's test line registers no handler, so the docs and the MCP skill now say a scripted test of that flow needs
browser.onDialog.run_codeis test code, so its dialogs still fail the next call. Surfacing answered dialogs in action results is a follow-up. - Porting to #1503: a trial cherry-pick onto its head conflicts only in
driver/dialogs.tsimports.BidiSessionbuilds the router there and would pass the same two callbacks, and its action entry points would takeguardAction's ownership line.
Verified
Ran it locally: yes
packages/web/tests/integration/agent-dialogs.test.ts, real Chromium, 8 cases: an agent leaves a dirty form; agent confirm, alert, and prompt; a replayed agent action (origin: 'test',agentStep: true); a test route handler'sroute.fetchbefore the agent's alert; a test'ssetViewportwhose resize handler confirms after an agent tap, which must fail the test's next step; a test action refused by a pending route error, which must leave the agent's alert with the agent; a test's 300 ms delayed confirm while agent reads poll, which must still fail withINVALID_STATE; and a test's own confirm. On main, both live agent cases fail withunhandled ... dialog. The viewport and refused-action cases fail without their fixes.- Web benchmark,
tests-agentdeferred-dialog: recorded live once, then replayed twice withCI=1 --strict-cache:Cache 1 replayed, no model call, and the app log showsconfirm dialog "Really submit the report?" accepted for the agent. - The whole web benchmark agent suite under
--strict-cache, with the same counts on main for the existing recordings:
| main | this branch | |
|---|---|---|
| Tests | 33 passed | 34 passed (+ deferred-dialog) |
| Cache | 26 replayed · 2 handed off · 4 missed | 27 replayed · 2 handed off · 4 missed |
- Testbed
tests/dialog.e2e.tsandtests/dialogs-frames.e2e.ts(15 tests) and web benchmarktests/native-dialogs.e2e.tsandtests/deferred-dialog.e2e.ts(4 tests) pass, so scripted handlers behave as before. e2e explore(openai/gpt-6-luna-fast,--max-steps 4) on a scratch notes app with a leave-page guard on its editor and aconfirm()before each delete. Its delete removes the wrong note, a planted bug:
| main | this branch | |
|---|---|---|
| Steps | 3 of 4, all blocked, ended "steps kept failing" | 4 of 4, the step limit was reached |
| Failed agent actions | 7 (unhandled beforeunload dialog) | 0 |
| Planted delete bug | not reached | found: "Deleting Trip plan removes Groceries instead" |
| Cost | $0.0107 | $0.0106 |
- Agentic repro tests against the same app (
e2e run --tag bugbash):
| Repro | main | this branch |
|---|---|---|
| Delete removes the chosen note (real bug) | AUTOMATION_UNSUPPORTED: blocked by the confirm | ASSERTION_FAILED: confirmed |
| Leaving the editor unsaved keeps the list (not a bug) | ACTION_FAILED: blocked by the leave-page dialog | passes |