cmux NEXT composer: a pasted screenshot shows its chip in the paste's frame

Merged
TE
16d ago
+453-18

Summary

Leo, on nightly 3a63d6d: "ME PASTING IT DOESN'T JUST PASTE RIGHT INTO THE COMPOSER IMMEDIATELY".

A pasted screenshot showed no chip until the composer had finished preparing it. useAttachmentIntake awaited intakeFiles before adding the chip, and for a full-screen Retina capture that means:

  • decoding it;
  • re-encoding it as a PNG at 2000 px (then as JPEG if that's still too big);
  • base64-encoding it.

That takes hundreds of milliseconds, in one long task on the main thread.

Now:

  • A paste or drop puts a placeholder chip in the strip at once. It's added in the paste's own task, so React draws it before the next paint. An image gets a thumbnail-sized tile with a dimmed picture glyph; any other file gets its name. The chip is marked aria-busy.
  • The file is read, scaled and saved in the background. The chip it becomes, or the refusal's note, takes the placeholder's place in the same render.
  • A prompt sent meanwhile (Return or the send button) waits. It goes once, when its chips are ready, with them. It never goes to another chat: switching chats drops the held prompt.
  • Batches count each other. A second paste during a read leaves room for the first one's chips.
  • The chat composer and the New Tab page share all of this (composer/attachmentIntake.tsx).

Paste to visible chip: before and after

Measured with the repo's interaction-latency harness (bun scripts/latency/run.ts), using a production build in headless Chromium and WebKit on Huge Blue, 7 runs each. The input is the new "paste screenshot" case in test/latency/actions.ts: a 3456×2234 PNG (window chrome, code text and a photo-like patch, a few MB) pasted on the prompt the way WebKit delivers it. The response is the chip in the strip.

enginebefore (d935e42c): work / paint, mslong tasksafter (bec4608b): work / paint, mslong tasks
Chromium248.1 / 259.27/76.3 / 9.20/7
WebKit (the app's engine)537.0 / 543.07/77.0 / 9.01/7

After the fix, both engines pass the harness's one-frame budget at 60 Hz and 120 Hz. In WebKit, one of 7 runs still had a long task (402 ms max). The CI latency-scoreboard job could not measure this: every dispatched run, base ones included (39222347431, 39218991217), is cancelled at its 20-minute limit.

Testing

webviews/test/composer-paste-instant.test.tsx uses a screenshot whose bytes arrive only when the test releases them:

  • the chip is in the strip before anything is read;
  • two screenshots show two chips at once;
  • a prompt sent during the read goes once, with the image;
  • a held prompt never goes to the chat the composer moved to.

Fleet webview suite (cmux-ci run --class light --script scripts/ci/run-webviews-tests.sh):

  • Failing test first (d935e42c): composer-paste-instant.test.tsx fails, because no chip is shown before the read.
  • With the fix (bec4608b): it passes, across 757 files.
  • Other reds: the 5 failing files (check-system-blue, gallery-coverage, hairline-rims, tooltip-rules, ui-rules) fail the same way at the red commit and on base. tooltip-rules now owns attachmentIntake.tsx, so its existing pasted-text tooltip got a tooltip-allow reason, and the file no longer appears in that test's output.

Observed / Not run

  • Observed:
    • the latency numbers above;
    • the fleet suites at d935e42c and bec4608b;
    • two independent review subagents, both SHIP. Their nits were addressed: chat switch, gallery waits that a placeholder would satisfy, React Compiler try/finally.
  • Not run:
    • the native app. Pasting in a running cmux NEXT was not tried; the harness drives the same composer in WebKit.
    • ⌘Return in the Quick composer while a paste is being read. The held prompt then goes as a plain send, without opening the window. Not covered.

Changelog

Fixed: a pasted or dropped screenshot shows in the composer the moment you paste it, instead of after it is processed.

Agent session: cosmic-gray

🤖 Generated with Claude Code

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Summary by cubic

Fixes the composer so a pasted or dropped screenshot shows its chip the moment it lands, instead of after the file is read, scaled, and re-encoded. Paste-to-visible-chip latency drops from 248/537 ms (Chromium/WebKit) in one long main-thread task to single-digit milliseconds with no long tasks.

Behavior

  • A placeholder chip (thumbnail tile for images, filename otherwise) is added synchronously on paste or drop; the real chip or a refusal note replaces it when the background read finishes.
  • A prompt sent while files are still being read waits and goes once, with the chips, when they are ready; switching chats drops the held prompt so it never goes to another chat.
  • Batches count each other, so a second paste during a read leaves room for the first batch's chips.
  • The chat composer and the New Tab page share this in composer/attachmentIntake.tsx.