cmux-next agent pane: tool calls show the images they return or write

Closed
TE
1mo ago
+259-18

Summary

An agent's image outputs now show inline in the chat, under the tool call that made them. Before this, the transcript dropped them or showed only text.

  • Returned images: a tool that returns an image, such as a browser screenshot, shows it. The ACP image content blocks used to be dropped; they are now kept as data URLs, png/jpeg/gif/webp only, up to 8 MB each.
  • Written images: a tool that writes or prints an image file shows that file. That covers a script that saves a chart, or a run that prints a screenshot path. The images come from the edited file, the call's locations, and absolute image paths in its output. They load through the host like a reply's local image (image.load), so only files inside the session's folders show.
  • Limits: at most four images per call, each capped at 360x220 and decoded lazily, so long transcripts stay light. A running call shows none yet.
  • Click to zoom: a click opens the image viewer (#18200). Local and web images in replies (ReplyImage) now open it too, through the new shared OpenableImage.

Stacked on #18200 (the image viewer). It retargets to feat-cmux-next when #18200 lands. Part of bead cx-ncc.51. Video, PDF thumbnails and artifact links follow in separate PRs.

Testing

  • Red commit 5c3dbb8ac6b, then the fix. bun test src/agent-session/acpmux/conversation/toolImages.test.tsx: 4 pass (local). Before the fix it failed because the module didn't exist.
  • bun test src/agent-session/acpmux: 1162 pass. The one failure, projectChooser "renders the attached folder, computer and branch tray without context chips", also fails on clean feat-cmux-next.
  • vp check passes, as do the gallery-coverage and ui-rules tests. The agent pane bundle is no longer committed (base dropped it); builds generate it.

Changelog

Added: Images an agent's tools return or save (screenshots, charts) show inline in the agent chat, and a click zooms them

Proof

Gallery entry agent-pane.transcript, variant tool-images, with real PNGs: a sign-in page screenshot a browser tool returned, and a build-time chart a script saved. Live preview: /wt/feat-cmux-next-tool-images/?entry=agent-pane.transcript#/agent-pane.transcript/tool-images

🤖 Generated with Claude Code

[!NOTE]
Medium Risk
UI-only in the webview, but it parses tool output for paths and persists base64 image payloads (size-capped), which affects transcript memory and rendering for untrusted tool content.

Overview
Tool calls in the agent transcript now show inline image thumbnails under each row (visible without expanding output), sourced from ACP image content blocks and from image paths the tool edited, referenced in locations, or printed in stdout.

Session merging keeps returned images as data URLs (png/jpeg/gif/webp, capped at 8 MB per block); toolImages dedupes and limits to four per call, with returned data URLs before file paths. File paths load through the existing host-mediated ReplyImage flow; data URLs render immediately.

OpenableImage centralizes click-to-zoom via ImageViewerContext for reply images (ReplyImage), markdown inline data URLs, and tool thumbnails. Gallery adds a tool-images transcript variant and extends reply-image coverage for OpenableImage.

Reviewed by Cursor Bugbot for commit 345078d52fc4ec04987a5e83bd20da409711ee93. Bugbot is set up for automated code reviews on this repo. Configure here.