feat: bridge web provider to agent-browser

Merged
TH
5mo ago
+849-23

Summary

Bridge the semantic web provider seam to the local agent-browser CLI with session-scoped command execution, JSON envelope/error handling, and snapshot normalization for refs, labels, textbox values, parents, rects, and selector-compatible roles.

flowchart TD
  AD["agent-device web command\nopen / snapshot / press / fill / close"] --> Daemon["daemon routing and session lease"]
  Daemon --> Interactor["createWebInteractor\ncoordinate-first Interactor contract"]
  Interactor --> Provider["WebProvider seam"]
  Provider --> Adapter["AgentBrowserWebProvider"]

  Adapter --> AB["agent-browser CLI\n--json --session <name>"]
  AB --> Chrome["local Chrome session"]
  AB --> Envelope["success/error JSON envelope"]
  Envelope --> Adapter

  Adapter --> Snapshot["normalizeAgentBrowserSnapshot\naria snapshot text + refs map + bounded get box calls"]
  Snapshot --> Nodes["RawSnapshotNode\ntype, role, label, value, parent, rect"]
  Nodes --> ADOutput["agent-device user output\nPage/Snapshot header + @eN [role] lines"]
  Nodes --> Selectors["shared selector runtime\nrole selectors and replay healing"]

  AB -. direct agent-browser usage .-> ABOutput["agent-browser native output\naria tree lines + native refs"]
  Daemon --> Cleanup["web close cleanup"]
  Cleanup --> Adapter

User-visible output is intentionally agent-device output, not raw agent-browser output. Direct agent-browser prints aria-tree lines such as - textbox "Email" [ref=e3]: ada@example.com; agent-device reprojects the same state as Snapshot: 3 nodes and @e2 [text-field] "ada@example.com" [editable] so refs/selectors fit the existing agent-device command model.

Add normal web session close cleanup so agent-device close calls agent-browser close even without a positional target.

Closes #820

Touched files: 12. Scope stayed within the web provider adapter, web close cleanup, and web snapshot text presentation.

Validation

Format passed with pnpm format. Typecheck passed with pnpm typecheck. Focused tests passed for src/platforms/web, session-close-shutdown, and output formatting. Fallow passed with pnpm check:fallow --base origin/main. Build passed.

Manual comparison passed on https://vercel.com/: agent-browser returned native aria-tree refs, while agent-device returned normalized Page: https://vercel.com/, Snapshot: 15 nodes, and agent-device refs for the same visible controls. Manual comparison passed on https://example.com/ with 2 normalized nodes. Manual form fixture verified textbox value projection: agent-browser showed [ref=e3]: ada@example.com; agent-device showed @e2 [text-field] "ada@example.com" [editable]. Sessions codex-web-vercel-817 and codex-web-example-817 were closed.