NEXT: dictation sends long recordings to a remote chat's machine

Merged
TE
7d ago
+915-18

Leo, 2026-10-25: "arbitarrily large file sizes"; 2026-10-26: "I ALREADY SAID WE NEED ARBITARRILY LARGE FILES OF ANY TYPE".

What

#26559 keeps dictation recordings of any length, but a recording too long to go inline (over 4 MiB) only reached a chat on this Mac. For a chat on another computer (an ssh peer's acpmux, or a Cloud chat), the page showed dictation.recordingElsewhere and the prompt got only the transcript. With this PR, the host sends that WAV to the chat's machine, and the chip links the copy there, so the remote agent can open it.

How

  • Page: remoteRecordingAttachment (attachments.ts) sends dictation.recording.upload {path} with the path the host gave it in the dictation update. It gets back {path} on the chat's machine and attaches savedAttachment(remotePath). The page never reads the recording's bytes. If the send fails, or the host is older and has no such method, it falls back to the old notice and the transcript goes alone. If the pane moved to another chat before the answer came, the answer is dropped.
  • Host: AgentPaneDictationUpload (CmuxNextAgentPane) streams the file from disk through the chat's own upload stager (AgentPaneModel.onUploadStage). That is the stager a dropped file already uses: _acpmux/file_put for a peer, cmux link access upload for Cloud. It sends 512 KiB chunks, reading each one off the main thread (FileManager.checked, blockingRead), and only one chunk is in memory at a time. If a chunk fails it cancels the stage, so the stager drops what it staged. It reads the answer as one path, either a peer's {path} or Cloud's {uploads: [{path}]}.
  • No CmuxNextApp source lines change. The new code lives in CmuxNextAgentPane and reuses the App's existing stagers.

Security

  • The host never accepts an arbitrary path from the page.
  • AgentPaneDictation keeps the long recordings it handed the page: the last 8, each with the chat it was recorded in.
  • takeRecording(path, session:) returns a file only when both of these hold:
    • the path is one of those recordings;
    • the chat is the same one.
  • Each recording can be taken once.
  • Any other path is refused (dictation.not_a_recording) before anything is read or staged.
  • A symlink or any file that is not a regular file is refused too (attributesOfItem's .type must be .typeRegular).
  • No page gesture is needed because the bytes are the host's own recording, which the person started in this pane. The page cannot choose them.
  • A chat with no stager answers unsupported, as before. A chat on this Mac still attaches its local file directly and never calls this.

Limits

  • Peer chats take up to AgentPaneUpload.maximumPeerFileBytes (1 GiB, about 9 hours of 16 kHz mono).
  • Cloud chats keep CloudUploads.maximumFileBytes (32 MiB, about 17 minutes). This PR does not change that policy: a longer recording for a Cloud chat gets the notice and the transcript. #27073 removes both stager caps.
  • The chip appears when the upload answers. A prompt sent before then goes without the recording. Holding the send the way a streaming drop does is a follow-up.

Tests

  • Red: 0b84b5983f473f3a6682305062ee05b9fca33641. Tests plus stubs that compile, so the tests fail on their assertions, not on the build.

    • Stubs: Swift AgentPaneRequest.dictationRecordingUpload(path:) (never decoded; the model answers it unsupported), AgentPaneModel.onDictationRecording, AgentPaneDictation.chatSession, .issuedRecordingLimit and takeRecording(_:session:) (always nil), with deliver(_:) made internal; web remoteRecordingAttachment (sends nothing).
    • bun node_modules/.bin/tsc --noEmit -p .: clean.
    • cd webviews && CMUX_WEB_TEST_FILES="test/dictation-audio.test.tsx" ../scripts/ci/run-webviews-tests.sh: 10 pass, 1 fail, on expect(calls).toEqual([["dictation.recording.upload", ...]]) (received []).
    • Swift (Mac run needed): swift test --filter AgentPaneDictationUploadTests in Packages/macOS/CmuxNext should compile and fail on its #expects.
  • Green: 7606b9e4692f7f36ce7cc1bead59c6bb941c9da0 (the fix; the tree equals the old head f4844fbfc67 plus #26559's new head a31b4ac4c62).

    • bun node_modules/.bin/tsc --noEmit -p .: clean.
    • cd webviews && CMUX_WEB_TEST_FILES="test/dictation-audio.test.tsx" ../scripts/ci/run-webviews-tests.sh: 11 pass.
    • Swift (Mac run needed): the same swift test --filter AgentPaneDictationUploadTests should pass.
  • AgentPaneDictationUploadTests covers:

    • decoding and the op allowlist;
    • only an issued recording, for its own chat, is taken, and only once;
    • the pane holds at most 8 recordings;
    • the file streams through the stager in chunks of 512 KiB or less, and the reassembled bytes match;
    • a Cloud answer becomes one path;
    • a path that was not issued is refused before the stager is called;
    • a chunk that fails cancels the stage.
  • The green commit also has two small fixes to the test file: it uses description (there is no safeDescription), and the chunk counting is written so it compiles.

  • Review fix (P1, a remote recording dropped from the prompt). The cross-machine upload ran asynchronously, but auto-send (and a Send click) went on the next task, so the transcript went without the recording. Decision: #26966's first pair already made the remote recording a pending chip that a send waits for (useAttachmentIntake().attachLater, ComposerHandle.attachLater, App.tsx). After the restack that pair sat directly on this PR, so this PR's head moved to it as a fast-forward. Nothing was rewritten, and #26966 keeps only its review fixes. The swift give-back for Retry (AgentPaneDictation.giveBackRecording, AgentPaneModel.onDictationRecordingUnsent) came down with it.

    • Red: 3476359694c35977659414b3068f3e77c14db80a, with stubs (an attachLater that shows and makes nothing; a giveBackRecording that does nothing). bun test test/dictation-recording-hold.test.tsx gives 0 pass, 3 fail. One of the failures is "a send made while a long recording uploads waits for it and carries the remote copy": dictation's auto-send in the same task as the hand-off, plus a second Send press while it waits. It is the same send() the Send button and Return take.
    • Green: d8011ea9a4d51530b1045244cd0c07177d2d79af (head). Per file: dictation-recording-hold 3/3, dictation-audio, and composer-send-hold pass. tsc and prepush-gate are clean.
    • Fleet: swift test --filter AgentPaneDictationUploadTests should fail its give-back cases at red and pass at green.
  • Review fix (page side of the recording-chat fix). If the pane moved from chat A to a local chat B while A's recording closed, the last update attached A's file to B's composer (a local file needs no upload, so no host check ran), and auto-send could send it in B. Now deliver stamps the recording with its chat (DictationAudio.chat, audio.session on the wire), and useDictation attaches it only while the pane shows that chat. Otherwise App.tsx shows a notice, dictation.recordingOtherChat (21 locales): "This voice recording was made in another chat, so it wasn't attached here." It also keeps a short, inline recording in A's kept chips (keepRecordingWithChat). A long recording's file is not kept there, because A may run on another computer. Auto-send never sends it in B.

    • Red: 935ed86d6ec8f11d5cc52667a2dcad65f710a2f5, with stubs (DictationAudio.chat, never set; the page's session, chat and elsewhere, never read). Web dictation-audio: 12 pass, 1 fail ("a recording made in another chat is not attached to the one the pane shows now"). Fleet: swift test --filter AgentPaneDictationAudioTests should fail theLastUpdateNamesTheChatItsRecordingWasMadeIn.
    • Green: 0a080079dc1b4392216fbb2d786474c3ed0f9e4f. Web dictation-audio and composer-kept-attachments pass. tsc, oxlint, oxfmt, the xcstrings lint and prepush-gate are clean. Fleet: the same filter should pass.
  • Review fix (#26966's P1, introduced here): a finished long recording was tagged with chatSession() when its async close ended, so switching from chat A to B in the meantime issued A's recording to B. AgentPaneDictation now reads the chat when the session starts (recordingChat, set only when the session is startable), and deliver(_:recordedIn:) takes that chat explicitly. Tests that deliver by hand now name it.

    • Red: 3d589f2ce83f406304c401ee69268ffe954f16ee adds AgentPaneDictationAudioTests.aRecordingIsIssuedToTheChatItWasRecordedIn. It holds the close on the recording's writer, moves the pane to chat B and releases the close. It then checks that B's dictation.recording.upload is refused (dictation.not_a_recording) with nothing staged, that B can't take the recording, and that A can. No stubs. Fleet: swift test --filter AgentPaneDictationAudioTests should compile and fail on those expects.
    • Green: 50cd05879731ac815ad99b57d92c6c60e2e35cb4. Fleet: swift test --filter 'AgentPaneDictationAudioTests|AgentPaneDictationUploadTests' should pass in full.
    • The page side is the fix above.
  • Restacked onto #26559's 5366f70396fae2094ccbce4bcef20e617f5d49f6 (write failures). This PR's diff over the earlier stack (red 68963acd6ff, green 4476aeaeb16) is the same as before, apart from context lines.

Changelog

  • Changed: A long dictation recording in a chat running on another computer (an ssh peer or a Cloud machine) is now sent to that machine and attached, instead of staying on this Mac.

Observed / Not run

  • Observed on the lane VM (Linux, Node 18):
    • bun node_modules/.bin/tsc --noEmit -p . is clean.
    • bun node_modules/.bin/oxlint on the changed files is clean.
    • oxfmt (printWidth 120) gives no diff.
    • The webviews check:* scripts and acpmux-protocol:check pass.
    • python3 scripts/cmux-next/prepush-gate.py finds no new violations. CmuxNextApp source is unchanged by this PR.
    • The push used CMUX_WEB_LINT_SKIP_REASON, because vp needs a newer Node than this VM has.
  • Not run:
    • Swift is not compiled here. The CmuxNextAgentPane changes and AgentPaneDictationUploadTests need a Mac compile and test run (swift test --filter AgentPaneDictationUploadTests on the fleet).
    • No end-to-end dictation to a real peer or Cloud machine.
    • No capture, because nothing changes visually: the chip is the existing saved-file chip.

No capture: no visual change (the existing saved-file chip, now with a remote path).

Agent session: agent session

šŸ¤– Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Long dictation recordings now reach the machine a remote chat runs on, instead of staying on this Mac with only a notice. A recording over 4 MiB for an ssh peer or Cloud chat previously left the WAV on the local Mac; now the page asks the host to stream the file through the chat's own upload stager, and the chip links the copy on that machine. Older hosts and failed uploads fall back to the old notice.

Security and limits

  • The host only sends recordings it made in this pane, for the chat they were issued to, and each only once; any other path, symlink, or non-regular file is refused before any bytes are read.
  • The file is opened once and every chunk is read through that descriptor, and only when its device and inode still match the file the writer closed, so a link or file swapped in mid-upload can't go out.
  • A recording is bound to the chat it was made in (captured when the session starts): one that closes after a pane switch is never issued to the new chat, and a pending recording chip is never carried to another chat by a switch that takes the prompt.
  • Until the upload answers, the chip is a pending tile a send waits for; its Ɨ stops the host and drops the tile, and a failed upload leaves Retry (the host gives the recording back). A failed or answerless finish cancels the stage.
  • Recordings stream to disk as heard, live in the chat's upload folder, and are removed by the 30-day launch sweep or with the chat.
  • Peer chats take up to 1 GiB (about 9 hours of 16 kHz mono); Cloud chats keep the existing 32 MiB upload limit, so longer recordings for Cloud still get the notice and transcript.