cmux mobile: newlines in the chat composers; tap any file to preview it

Merged
AZ
14d ago
+1649-183

Summary

Two gaps in the cmux-next iPhone app, plus the shared preview they need.

1. Newlines in the composers

ComposerOn-screen ReturnHardware ReturnShift- or Option-ReturnGrows to
Agent chat (GUI)new linesendsnew line8 lines, then scrolls
Chief (CmuxConversation)new line (unchanged)sends (new)new line (new)unchanged (header)
Terminal "Send to …"sends (unchanged, SPEC 10.2)sendsnew line (new)6 lines, then scrolls
  • The agent chat field was a SwiftUI TextField, which decides Return by itself. It is now ChatComposerField, a UITextView over the shared ComposerReturnTextView and ComposerReturnPolicy in CmuxMobileDesign.
  • Hardware Return only does what the send button would: send, or queue/steer while the agent runs. It never sends an empty draft and never presses Stop.
  • While text is being composed (marked text, for example Japanese), Return belongs to the input method.
  • The terminal keeps Return = send, because a shell expects it. A multi-line draft still gets a new line: Shift-Return, Option-Return, or New Line in the field's edit menu. The draft goes to the program as one paste (ComposerSubmit, unchanged).

2. Preview any file everywhere

  • One preview: FilePreviewGallery, built from ArtifactPreview pages.
    • Images: pinch, double-tap, swipe down to close.
    • Video: plays.
    • PDF, syntax-colored code and text, rendered Markdown and HTML (with a Source toggle).
    • Audio, Office/iWork, RTF and anything else Quick Look renders: an embedded QLPreviewController (new .audio/.document kinds and ArtifactContent.quickLook).
    • Types nothing on the phone can show: a share sheet ("Share or Open In…").
    • Every page has Share or Save (Save to Files, Save Image, AirDrop) and a position line ("3 of 5"). Swipe between a message's attachments.
  • Wired into:
    • agent-chat composer attachments before sending;
    • sent agent-chat attachments (files were inert; images used an image-only viewer, now removed);
    • terminal composer attachments (UIKit presentation, like Files and Links);
    • Files and Artifacts / Files and Links;
    • diffs: Open File in the diff screen and the diff card menus;
    • tool calls: long press opens the files the call touched;
    • files agents mention: links and path:line inline code in answers open the preview, web links open as before;
    • Chief: file links (onOpenLink), queued composer attachments (Quick Look), and a message's documents (Quick Look, swipeable).
  • Feed: a link in an event's markdown no longer navigates the pooled web view (which went blank). Web links open in Safari; others open the event.
  • Trust: Mac files are read only through the link's file service (LinkFileArtifactSource), so the Mac's workspace-root and Chat Folders rules (#22354, D9) still decide. Chief resolves only absolute paths. Phone-held attachments use an attachment: reference that never goes to the Mac. Nothing widens access.

CmuxConversation changes land upstream, byte-identical, in #24573 (feat-imsg-parity-gui).

Changelog

  • Added: the iPhone agent chat composer takes several lines (Return starts a new line, the send button sends; on a hardware keyboard Return sends and Shift-Return starts a new line).
  • Added: tap any file in the iPhone app (attachments before and after sending, Files and Artifacts, diffs, files agents mention) to preview it, swipe between a message's files, and share or save it.

Tests

Failing tests first: c4c894d397dc. On mini-cmux11s with verify/termqa/cmux_mobile_kit-test-ios.sh:

  • Red on the tests-only commit: the CmuxMobileAgentChatTests build fails (ArtifactContent has no member quickLook, no ComposerReturnPolicy, and so on).
  • Green on head 32251815e7b3: CmuxMobileAgentChatTests 132/132, CmuxMobileLiveTests 262/262, CmuxMobileChiefTests 79/79. ComposerSubmitTests 5/5 and CmuxMobileShellTests passed on earlier heads.
  • Sent attachments: red on 3475760ebbbc (no recordedImage/recordedImageID), green on the fix.
  • New suites in ComposerFilesTests.swift:
    • Return policy, chat and terminal;
    • the text view's Return and New Line;
    • kinds for every common type;
    • local attachments decoded like Mac files (image, text, docx through Quick Look, video, binary, PDF);
    • missing bytes;
    • a Mac file through the fallback source;
    • local copies named and stable;
    • FileLink (web, app and anchor links refused; path:line, file://#L);
    • MarkdownFilePath (console.log is not a file).

Live check (real usage, real link, no shortcuts)

  • Setup:
    • Phone: headless iPhone 17 Pro simulator cmux-composer-files (iOS 26.5) on mini-cmux8s, Demo build of this branch, signed in with the dev credential.
    • Mac: the hidden rig Mac app on mini-cmux11s (fleet build cmux-mobile-e2e-e196c421, open -g -j -n), workspace live-files in a real project folder.
    • Link: WireGuard (path=Direct, then path=Turn).
    • Agent: a real Claude Code chat (Opus 5.5), started from the phone's New Agent Chat With….
    • Driving: axe HID input. Recording: simctl io recordVideo. Simulator.app was never opened, and nothing on the laptop took focus.
  • Newlines (agent chat): the on-screen Return typed "Reply with the line count of this prompt.\nline two\nline three". Claude answered "Your prompt has 3 lines of text" and described the attached photo.
  • Newlines (Chief): hardware Shift-Return and Option-Return, then Return, sent one three-line message to the Mac's Chief.
  • Composer attachments (photo, notes.txt, report.pdf, clip.mp4): each tap opened the full-screen preview (image, syntax-colored text, PDF, playing video), paging "1 of 5" to "5 of 5", with Share or Save. Claude received all of them; it ran a command on …/.cmux/attachments/phone-…/clip.mp4.
  • Sent message:
    • The live check found a bug: the sent bubble showed no attachments, because acpmux's user_message names images only. Fixed in 05f0ad0b0987/32251815e7b3 (tests first in 3475760ebbbc).
    • After the fix, the new turn's bubble shows the photo plus notes.txt, report.pdf and clip.mp4. Tapping each opens it, paging "1 of 4" to "4 of 4".
    • After an app relaunch, the earlier turn's photo, which now comes from the Mac's record through _acpmux/attachment, shows in the bubble and opens full screen.
  • Files agents mention: tapping report.pdf in Claude's answer opened the PDF read from the Mac over the link (Copy Path shown).
  • Mac files (terminal Files and Links):
    • At first the Mac refused with "isn't sharing any folders with this phone yet" (forbidden, "no workspace roots are shared"), which is the trust rule holding.
    • With the project folder set as a Chat Folder, these opened from the Mac: report.pdf, photo.png, page.html (rendered, with Source), notes.md (rendered), brief.docx (Quick Look) and main.swift (syntax colored).
  • Evidence: git-ignored, on the laptop under ~/cmux-mobile-builds/20261023-composer-files/evidence/ (videos 01–10, shots/).

Observed / Not run

  • Not observed: Chief's queued-attachment Quick Look (the agent-chat equivalent was observed); the Feed link change (no Feed event with a link on the rig); the diff "Open File" menu (unit-wired only).
  • Not finished: Claude's tool permission in the second turn asks for device authentication, which a simulator without a passcode can't answer, so that turn was declined.
  • No real iPhone (user direction: simulators only).

Review

A fresh correctness review subagent reviewed both rounds. Round 1 found no blockers and 6 should-fix items, fixed in 0909d699c39b. Round 2, on the sent-attachment fix, found no blockers and 2 should-fix items, fixed in 32251815e7b3. Both are in PR comments.

🤖 Generated with Claude Code