#4314 feat(service,web): live view and takeover of an agent's Browser tab

PR E of the cloud browser rollout (plan "Browser agent tab automation setup", §5 A live view and takeover, §6 Privacy, §7 Cost and limits). This is the service side: a WebSocket relay a member's panel opens to watch one of an agent's cloud Browser tabs live, and to take it over. The panel itself is PR F, which stacks on this one.

What it does

• The socket: GET /api/browser/live?task=<id>&tab=<id> is a WebSocket function of its own, standalone like the terminal relay, with an 800 s bound. One socket covers one tab of one task.
• Admission:
  • With no Origin, the upgrade must carry the account's bearer. That's the desktop's main process, as for a terminal.
  • With an Origin, the origin must be one of the deployment's own (authDeployment(...).ownOrigins, the list origin-guard.ts reads; never trustedOrigins). The account is then read from the session cookie through requestUserId, as every hosted route reads a browser's.
  • The account must reach the tab (readTaskBrowserTab). Otherwise the upgrade is refused (401 / 403 / 404 / 503) before any socket stands.
• Frames out:
  • The relay connects with AgentBrowser.connect, which resumes a stopped sandbox and starts the browser.
  • It attaches to the tab's target (Target.attachToTarget, flatten) and runs Page.startScreencast: JPEG, quality 60, at most 1280×800.
  • Each frame goes to the panel. Its Page.screencastFrameAck is sent no sooner than 125 ms after the previous one, so Chromium sends at most 8 frames a second.
  • A frame is skipped while the panel's socket still has more than 1 MB to send.
  • The address, title and loading state are sent as they change (Target.targetInfoChanged, Page.frameStarted/StoppedLoading).
• Input in, over the same socket (C's REST contract is not used for driving):
  • take-over and hand-back use C's store and terms: BROWSER_TAKEOVER_HOLD_MS, and controlsAgent decides who may drive.
  • Mouse, wheel, key and text messages become Input.dispatchMouseEvent, Input.dispatchKeyEvent and Input.insertText. Each point is mapped from the panel's viewport onto the page's CSS size.
  • Input moves the takeover on, with a write at most every 10 s.
  • Input from a member whose takeover doesn't stand is refused with a refused message and never reaches the page.
  • Mouse moves that queue up behind the browser are collapsed to the latest one.
  • The tab is read again every 5 s. Driver changes made in another window reach this socket, and a tab that closed closes the socket as gone.
  • However the socket closes, it hands back any takeover it took.
• Holding the sandbox:
  • For a coding agent's tab, the socket holds the sandbox through the terminal's lease (holdSandbox) from the moment it connects. Each panel message moves the lease on, written at most once a minute.
  • The socket closes as idle after 5 minutes with no message from the panel. The sweep then stops the sandbox once the lease lapses, as it does for a terminal.
  • Luke's tabs take no lease. A lease on the task's conversation would have the sweep stop Luke's sandbox ahead of its own 30-minute rule.
• The wire: @luke/features/browser-live-wire defines BROWSER_LIVE (path and query), the close reasons and their codes, and Schemas for both directions:
  • server → panel: frame, page, driver, refused, closed;
  • panel → server: take-over, hand-back, viewport, mouse, wheel, key, text.
• Privacy:
  • Frames, address and title go only to the panel's socket.
  • Nothing here logs, traces or counts them.
  • The test data is made-up strings, so no fixture holds a real page.

Where it stands

• Runtime edge: the new function stands from the existing edge apps/web/server/terminal/function.ts, which gains browserLiveFunctionServer(). Each relay is still a function of its own. Doing it this way adds no runtime edge, so neither root AGENTS.md's edge list nor effect-edges.json changes.
• Tabs after a resume: a tab whose target the browser no longer holds closes as gone. That happens, for example, after a resume restarted the browser. Reopening tabs at their last address after a resume is not done here.

Tests

• browser-live/service.test.ts:
  • refused with no credential, with a blank bearer, with a cookie but no origin, with another site's Origin, with an own Origin but no cookie, and on a deployment that owns no origin;
  • an unreadable task or tab is a 400; another path is a 404;
  • an account that doesn't resolve is a 401, a tab the account can't read is a 404, and a store outage is a 503.
• browser-live/relay.test.ts runs over fakeCdpPeer behind the real agentBrowserOver, with a fake store, on TestClock:
  • frames are relayed and acknowledged at the cap;
  • page state follows;
  • input is refused without the takeover and dispatched with it, coordinates mapped;
  • not-permitted for a member who doesn't control the agent;
  • another member's takeover is named, and a takeover that lapsed is refused;
  • a close hands the tab back;
  • a coding agent's lease is held and moved on, and Luke's tab takes none;
  • gone when the tab isn't in the browser, is destroyed, or leaves the store;
  • not-opened when the agent has no sandbox;
  • idle after 5 minutes with no message.
• functions:rewrites and functions:externals were regenerated. /api/browser/live was added to the preview probe and the function-dispatch test.

Made with Luke

---