perf(server): skip the screenshot capture for text-only preview snapshots

Closed
BO
2d ago
+96-24

Fork review PR (review only; the upstream submission will use this same diff).

Text-only preview snapshots no longer pay for a full-frame screenshot: 0 Page.captureScreenshot calls (was 1, plus PNG encode, base64 transport, and a base64 decode) on every preview_snapshot that asks for neither the inline image nor a saved file — which is nearly every snapshot an agent takes before interacting.

Problem: preview_snapshot stripped includeImage/save in the MCP handler, then the snapshot always captured and encoded a full-frame PNG, carried it through the broker as base64, and decoded it in McpHttpServer before dropping the bytes whenever neither flag was set.

Change: the handler now passes includeScreenshot (includeImage || save) with the snapshot operation. ServerBrowser forwards it to ServerBrowserPage.snapshot, which skips captureViewport entirely when false and returns screenshot: null. The contract field becomes nullable (PreviewAutomationSnapshot.screenshot), and McpHttpServer handles the null (no decode, no save, metadata screenshot null). Hosts that ignore the new input field keep returning a screenshot, so older desktop hosts are unaffected. The tool description now says screenshot dimensions are returned when a screenshot is captured. Reading order: packages/contracts/src/previewAutomation.ts (input + nullable field), then ServerBrowserPage.ts (skip), ServerBrowser.ts + handlers.ts (threading), McpHttpServer.ts (null handling).

Evidence: new real-Chromium test in ServerBrowserPage.test.ts counts Page.captureScreenshot CDP calls: 1 with the default, still 1 after a second snapshot with includeScreenshot:false, whose screenshot is null and whose accessibility tree is intact. Full ServerBrowserPage + ServerBrowser suites pass (39 tests) on head 28fdb231e3; server typecheck clean. Adjacent, not covered: pingdotgg/t3code#21494 (text-only snapshots hanging on hidden tabs) — skipping capture also removes that capture from text-only calls, but this PR does not change hidden-tab capture behavior when a screenshot is requested.

Muse Wonder Full 1.0, in Muse Code (via T3 Code)


Summary by cubic

Skips the full-frame screenshot capture for text-only preview snapshots, removing one Page.captureScreenshot call plus PNG encode, base64 transport, and base64 decode from nearly every snapshot an agent takes before interacting.

  • preview_snapshot now passes includeScreenshot (true when includeImage or save is requested) through the snapshot operation; when false, the capture never runs and screenshot is null.
  • The contract field is now nullable; hosts that ignore the new input field keep returning a screenshot, so older desktop hosts are unaffected.