Chat: fit every attached image to 2000 px and 1.5 MiB instead of refusing it

Merged
TE
16d ago
+114-47

Summary

  • Every image attached to a chat prompt is now fitted to what the models use: a long edge of at most 2000 px and at most 1.5 MiB. Before, only images over 5 MiB were scaled, to 2560 px.
  • A large image is never refused for its size. It is re-encoded as PNG when that fits, else as JPEG at 0.9, then at smaller sizes (1600, 1200, 800 px).
  • An image within both bounds goes as it is, so a GIF stays animated. An image the engine cannot decode still goes as it is when the API takes it (5 MiB).

Why

On 2026-10-20 a coordinator chat lost its connection: a prompt with six Retina screenshots was about 15 MB of base64, and the next one passed acpmux's 16 MiB WebSocket frame limit. acpmux reset the socket, and the chat said "Not sent: The agent connection was interrupted". #23027 raises the frame limit and makes the chat resend a prompt it lost on reconnect. This PR keeps image prompts small in the first place (product call: never refuse a person's screenshots). Sending attachments out of band, so no prompt frame carries image bytes, is the next PR.

Tests

  • webviews/test/composer-image-fit.test.ts (committed red before the fix):
    • a 3024 px screenshot under 1 MiB is scaled to 2000 px;
    • an image within both bounds is not drawn at all;
    • a noisy image steps down to 1200 px JPEG instead of being refused;
    • an image is passed through when no engine can scale it.
  • The existing tests were updated for 2000 px.
  • I didn't run the tests locally (bun test is blocked on this Mac); CI runs them.

Agent session: cosmic-gray

๐Ÿค– Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Fits every image attached to a chat prompt to a 2000 px long edge and 1.5 MiB โ€” the size the models scale to anyway โ€” instead of only scaling images over 5 MiB to 2560 px. Keeps image prompts small so a pile of screenshots no longer exceeds the chat's 16 MiB frame limit and resets the connection (a six-screenshot prompt did just that on 2026-10-20).

Fitting

  • A large image is re-encoded as PNG when that fits, else as JPEG at decreasing sizes and quality, and is never refused for its size.
  • An image within both bounds goes as it is, so a GIF stays animated; one the engine cannot decode still goes as it is when the API accepts it.