fix(decopilot): show MCP image results to the model instead of base64 text
What is this contribution about?
When an MCP tool returns image content, decopilot sends it to the model as text. toModelOutput in mcp-tools.ts runs JSON.stringify on every non-text block, so the model gets a JSON string holding the base64 bytes.
Measured with the new Figma MCP (decocms/mcps#558). One get_design_context call on a 450×768 frame returned 100,341 bytes, about 25k tokens, almost all of it the PNG as base64. The model cannot read that. On larger frames the total goes over MAX_RESULT_TOKENS (32k), and truncateForModel then replaces the whole result, including the useful text outline, with a read_tool_output preview.
Studio already has a way to show tool images to the model. take_screenshot and the VM Read tool put images on pendingImages, and prepareStep attaches them as a user message. The comment in built-in-tools/index.ts explains why: images inside tool-result messages are not supported by every provider, but images in user content are. This PR routes MCP images through the same queue:
attachToolResultImagesswaps eachimageblock for a short placeholder,[image/png image, 98 KB, attached below], and queues the bytes onpendingImages.- If there is no queue, or the image is over 5 MB (Anthropic's limit), the image is only described and nothing is attached.
- The truncation check now runs on the result without the base64, so the text is no longer dropped because of the image.
- Each
toolCallIdqueues its images only once, even iftoModelOutputruns again. - Only the model-bound copy changes. The tool result streamed to the UI is untouched.
tools.tspassesextras.pendingImages, the queue the built-in tools already share, intotoolsFromMCP.
Behavior change on non-image results: none. They take the same code path as before.
Why not put the image inside the tool result?
AI SDK v7 does support { type: "content", value: [..., { type: "file", mediaType, data }] }. I checked how each provider converts it:
| Provider | Image in a tool result |
|---|---|
| Anthropic | native image block |
| OpenAI Responses | input_image |
| function-response parts | |
| OpenRouter | image part |
| OpenAI Chat Completions | JSON.stringify: back to base64 text |
The pendingImages path works the same way for all of them, and it is the convention Studio already uses.
Related PRs
- #5396 caps screenshot size at Anthropic's 8000 px limit. The Figma MCP already renders under that.
- #7021 strips inline base64 image bytes when thread parts are persisted. Images are therefore not replayed on later turns, and that stays true here.
How did you verify your code works?
- New
mcp-tools.test.ts(3 tests):- an image is queued with a data URL and label, and the result keeps a placeholder;
- with no queue, or above the size cap, the image is only described and nothing is queued;
- end to end through
toolsFromMCP: a result of about 400 KB of base64 plus text gives the model"[image/png image, 293 KB, attached below]\nFRAME Card 320x200"instead of a truncation preview. The image is queued once across twotoModelOutputcalls, and the original result keeps itsimageblock.
bun test src/harnesses/lib/decopilot: 188 pass, 0 fail.tsc --noEmitinapps/api: clean.biome formatandoxlinton the changed files: clean. The kebab-case filename warning already existed.
How to Test
- Connect an MCP that returns images, for example the Figma MCP from decocms/mcps#558, or any MCP with a screenshot tool.
- In a decopilot chat, ask it to call the tool. With Figma:
get_design_contexton a frame link. - Expected: the model describes what is in the image, the text part of the result is intact, and no base64 shows up in the model input. The tool card in the UI looks the same as before.
Review Checklist
- PR title is clear and descriptive
- Changes are tested and working
- Documentation is updated (if needed)
- No breaking changes
🤖 Generated with Claude Code
Summary by cubic
Shows MCP tool image results to the model as attached images instead of unreadable base64 text.
- MCP image content previously reached the model as a stringified JSON blob of base64 bytes, costing roughly 1 token per 4 bytes and often pushing results past the 32k-token cap so even the text was replaced by a
read_tool_outputpreview. - Routes MCP images through the existing
pendingImagesqueue used bytake_screenshotand the VMReadtool, leaving a short text placeholder in the model-bound tool result. - Images over 5 MB, or when no queue is available, are only described and not attached.
- Only the model-bound copy changes; the tool result streamed to the UI keeps its original
imageblock. - Non-image results take the same code path as before.