feat(chat): composer shortcuts, Cmd+Enter steer, and image attachments in the transcript

Merged
HA
1mo ago
+866-94

Links

Summary

Chat v3 composer work:

  1. Composer height cap - a long draft no longer pushes the send button out of the pane.
  2. Keyboard - / and ⌘J focus the composer, ⌘U attaches a file, ↑/↓ recall sent prompts, ⌘/Ctrl+Enter sends and steers, ⇧Tab moves into the queue.
  3. Server-side steer - a prompt sent with steer: { expectedTurnId } is steered by the host in the same command that queues it.
  4. Attachments reach the agent - attached files go to the worktree's .superset/attachments and travel as <attachment path="…" type="…" /> tags in the message text, so every harness can read them. The UI parses the tags: images as thumbnails, other files as name badges.

Why / Context

  • A long draft grew the editor past the pane and clipped the send button.
  • There was no keyboard path to the composer, to past prompts, or to steering.
  • The Claude and Codex harnesses never sent attachments to the agent: claudeAdapter turned one into [name], codexAdapter dropped it. Only ACP resolved them.

How It Works

Composer (packages/chat-ui)

  • .prompt-input-editor gets max-height: 40vh; overflow-y: auto.
  • utils/historyNavigation registers ↑/↓ on the Lexical editor. ↑ recalls when the box is empty or the caret is at the very start; after that it walks history while the recalled text is unedited. Past the newest, ↓ restores the draft as EditorState, so chips survive. Once a recalled prompt is edited, the arrows stop navigating so the edit is kept.
  • The submit payload gains steer: boolean (⌘/Ctrl+Enter); IME composition never submits.

Steering (packages/chat, packages/chat-runtime)

  • promptInputSchema gains optional steer: { expectedTurnId }. LiveSession.prompt steers the queued item when the running turn still has that id; otherwise it only queues.
  • The outbox entry carries the flag, so a retry sends it again. The desktop passes the running turn id and has no client-side steer logic.

Desktop chat

  • FOCUS_CHAT_INPUT and CHAT_ADD_ATTACHMENT were registered but unhandled; the active pane's composer now handles them through useHotkey. / stays a plain listener (a registry entry holds one binding per platform) with the same IME/AltGr guards.
  • ⇧Tab from the editor focuses the newest queued row: ↑/↓ move, Enter steers, ⌫ removes, E edits, Esc returns.
  • History comes from the timeline (utils/promptHistory, bookkeeping blocks and attachment tags removed) inside useStableList, so the memoised Composer does not re-render on every item.
  • Attachments: uploadPastedFiles (moved from TerminalPane to usePaneRegistry/utils) writes the files; utils/attachmentTags formats and parses the tags; thumbnails load through filesystem.readFile, capped at 5 MB, falling back to a name badge.

Keyboard Shortcuts

ShortcutAction
/Focus the composer (active pane, focus not in a field)
⌘J / Ctrl+Shift+JFocus the composer (rebindable)
⌘U / Ctrl+Shift+UAttach a file (rebindable)
↑ / ↓Recall previous / next sent prompt
⌘/Ctrl + EnterSend and steer (sends normally when idle)
⇧Tab, then ↑ ↓ Enter ⌫ E EscQueue: move, steer, remove, edit, back

Manual QA Checklist

Verified in the dev app over CDP:

  • 60-line draft: editor 401px (40% of the window), send button inside the pane
  • Pasted PNG + "What color is this image?": file in .superset/attachments, thumbnail shown without the raw tag, claude-code agent ran Read .superset/attachments/red-square.png and answered "Red"
  • ⌘J from the page body focuses the composer
  • ↑ then ↓ restores a draft that starts with a /autoplan chip
  • (walkthrough page) / focus, ⇧Tab + ↑ + Esc, ⌘Enter steer order, history recall, layout of images and file badges

Still to check:

  • ⌘Enter with the server-side steer in a live turn
  • Enter, ⌫, E on a focused queue row
  • / with two chat panes, and inside a terminal
  • Height cap in a short split pane; light theme

Testing

  • packages/chat: 42 pass. packages/chat-runtime: 181 pass, including two new LiveSession tests (steer for the running turn interrupts it and runs next; steer for another turn only queues).
  • packages/chat-ui: 4 pass (new historyNavigation tests, headless Lexical). The package had no tests before; it now has a test script and excludes tests from tsc, like packages/chat.
  • apps/desktop (usePaneRegistry): 232 pass, including attachmentTags, promptHistory, uploadPastedFiles.
  • tsc --noEmit for chat, chat-runtime, chat-ui: clean. apps/desktop typecheck: no errors in changed files.
  • bun run lint on changed files: clean. sherif: clean.

Design Decisions

  • Steer on the host: the client-side "send, wait for the echo, then steer" needed three guards and still had a race between the check and the action.
  • Attachments as text tags: one plain path works for every harness with no adapter changes, and the files live in the workspace where agents already read.
  • ⇧Tab into the queue: ⌘↑/⌘↓ is the macOS jump-to-start/end shortcut. ⇧Tab was the brainstorm-page pick and keeps roving focus for screen readers.
  • Assistant text colour unchanged: an earlier commit set it to full foreground; it is reverted here and left to settle with Avi (#8163).

Known Limitations

  • Thumbnails crop to a square and cannot be enlarged yet.
  • A recalled slash command such as /review opens the command menu, so the next ↑ moves in that menu.
  • The 40vh cap is window-based, not pane-based.
  • Attachments stay in the worktree until it is deleted (gitignored).
  • steerInputSchema in the protocol is still unused.

Compatibility

  • prompt input gains an optional steer. Older clients never send it, so behaviour is unchanged. An older host strips the unknown key (non-strict zod object) and queues the message without steering.
  • No host-service procedure was added or changed. Older messages with structured attachment parts still render as name badges.

Follow-ups

  • Click-to-enlarge for thumbnails

Summary by CodeRabbit

  • New Features
    • Navigate previous prompts with the Up and Down arrow keys, with drafts restored when returning to the editor.
    • Steer a running chat turn with a new prompt using the keyboard.
    • Navigate, edit, remove, and select queued prompts with keyboard shortcuts.
    • Attachments appear in chat messages as image previews or file labels.
    • Focus the prompt editor and manage attachments with keyboard shortcuts when the chat pane is active.
  • Improvements
    • The prompt editor scrolls when its content exceeds the available height.