feat: drive the in-app browser pane from the CLI + raw CDP

Merged
KI
3mo ago
+2382-40

What

Adds a superset browser command group and the plumbing behind it so agents (and the CLI) can drive a workspace's in-app browser panes, and speak raw Chrome DevTools Protocol to a pane for browser-use / Playwright-class automation.

superset browser list       --workspace <id>
superset browser open       --workspace <id> --url <url> [--target new-tab]
superset browser navigate   --workspace <id> --pane <id> --url <url>
superset browser screenshot --workspace <id> --pane <id> [--out shot.png]
superset browser eval       --workspace <id> --pane <id> --code '<js>'
superset browser console    --workspace <id> --pane <id>
superset browser cdp        --workspace <id> --pane <id>   # prints a raw CDP ws:// URL

How

Scope is the pane's guest webContents only, never the app shell.

agent / CLI  →  host-service `browser` tRPC (existing rail: local manifest PSK, or relay + JWT)
             →  loopback BrowserBridge in desktop main (endpoint+secret via child env)
             →  BrowserManager → the pane's guest webContents (+ webContents.debugger for raw CDP)
  • desktop main — new browser-bridge.ts (express + ws loopback); its endpoint/secret are passed to the host-service child via env, mirroring HOST_SERVICE_SECRET in reverse. Wraps BrowserManager, now workspace-scoped with capturePng / listPanes / requestOpen / attachCdp (single CDP session per pane, via webContents.debugger). Bridge state lives in an electron-free browser-bridge-info.ts so unit tests that mock electron don't transitively pull in shell.
  • host-service — a browser router proxying to the bridge, plus a /browser/:paneId/cdp WebSocket route that tunnels raw CDP frames (works locally and over the relay, same path as terminals). resolveHostTarget now exposes a WS base + token so the CLI can build the CDP URL.
  • renderer — browser.register now carries workspaceId; a global useBrowserOpenRequests hook consumes open requests through the existing openUrl search-param flow (pane layout stays renderer-owned).
  • cli — the browser command group.

Standalone (desktop-less) hosts return a clear "this host has no browser" error. Design + verification notes: plans/cli-browser-pane-cdp-control.md.

Verification

Driven end-to-end over CDP against the running dev app: open (renderer round-trip → pane created), list (workspace-scoped), navigate, eval, screenshot (real PNG, no clipboard clobber), console, and raw CDP (Page.navigate / Runtime.evaluate / DOM.getDocument over the tunneled WS), including the single-session-per-pane guard. Also drove a scripted click/type/form-submit flow through the CDP Input domain. Ran the real superset browser … binary, not just the tRPC layer.

CI locally: host-service (1083) + cli (99) + coordinator (35) tests pass; typecheck / biome / sherif clean.

Follow-ups (not in this PR)

  • MCP tool mirror of the command set.
  • "Agent controlling" pane badge + a consent setting (panes share the persist:superset cookie jar).
  • LRU eviction exemption so an actively-driven background pane isn't rebuilt.

Summary by cubic

Drives in‑app browser panes from the CLI and agents, with a per‑pane raw CDP WebSocket. Previously panes were renderer‑only; now superset browser can list/open/navigate/eval/screenshot/console and print an authenticated CDP URL, with workspace‑scoped access and scheme allowlisting.

  • Desktop: starts a loopback BrowserBridge (Express + WebSocket on 127.0.0.1) gated by a boot‑minted bearer secret and started before any host‑service spawn; the host‑service proxies browser.* tRPC and /browser/:paneId/cdp, redacts the bridge secret in crash tails, and standalone hosts strip inherited BROWSER_BRIDGE_* and return “this host has no browser.”
  • Renderer: browser.register accepts workspaceId and panes re‑register when they move workspaces; a global hook consumes external open requests via the existing openUrl flow.
  • BrowserManager: workspace‑scoped with listPanes, requestOpen (serialized per workspace; queue max 8 → overflow 429), capturePng, and attachCdp with a single‑session‑per‑pane guard (second attach closes 1013). Navigation enforces an allowlist (http/https/about) including CDP Page.navigate; disallowed schemes error. Eval code is capped (1MB); CDP frames are capped (4MB) with a bounded pre‑connect queue. Browser console levels map correctly (debug/log/warn/error).
  • CLI: adds superset browser {list|open|navigate|screenshot|eval|console|cdp}; cdp validates the pane exists and prints an authenticated WS URL. Host resolution returns a base WS origin and token. Dev tooling adds cli:dev (targets the dev host and honors SUPERSET_ORGANIZATION_ID), a port preflight, and dev:skills.
  • Host‑service: new browser router and a CDP WS tunnel (works locally and over the relay). Env‑driven bridge config is shared by entry points.
  • Docs: adds the superset:browser skill and CLI reference; notes that the app bundles the CLI and CDP “Enter” needs "\r".

Migration

  • None required. To make panes discoverable to the CLI/agents, pass a workspaceId in browser.register.

Summary by CodeRabbit

  • New Features
    • Added controls for listing, opening, navigating, reloading, and capturing screenshots from in-app browser panes.
    • Added browser console retrieval and JavaScript evaluation.
    • Added authenticated browser automation, including per-pane CDP WebSocket access.
    • Added workspace-aware pane handling and workspace-specific URL opening.
    • Added CLI commands for browser management, screenshots, console output, JavaScript evaluation, and CDP access.
    • Added a browser automation skill for agents.
  • Documentation
    • Added guidance covering browser access, security, CLI usage, and automation workflows.