perf(service): send tool output by reference in transcripts

Merged
CH
4d ago
+1225-97

PR 4a of the performance rollout. The brief: a transcript page should weigh kilobytes, not megabytes. Per agent conversation, stored parts run to 12–14 MB at the top, and most of that is tool-bash output, reasoning, read_file, and MCP outputs.

What changed

The read cuts outputs (packages/service/src/agents/coder-host/part-cut.ts, messages-procedures.ts)

  • A transcript read asked with ?parts=cut cuts some parts down: a tool output larger than CODING_AGENT_BOUNDS.MAX_PART_CHARS (4096 characters) and a reasoning part's text larger than the same bound. Each cut part keeps only its first and last 2048 characters (head / tail), plus its full size in UTF-8 bytes.

  • The cut form is explicit in the wire schema (coding-agent-wire.ts). The message gets a new cutParts: [{ index, field, bytes, head, tail }], and the part at that index is sent without its output / text field. Nothing in the payload passes off the cut text as the real output.

  • The reference to the full part is the message id plus the part index.

  • Parts under the bound are sent unchanged.

  • Only tools whose output the window just draws as text are cut:

    • bash, read_file, grep, glob, run_on_mac, and the web reads
    • connection_* and the integrations' remote tools
    • the Claude Code and Codex equivalents of these

    Every other tool's output is sent whole, because the panel reads meaning from it: the agent a start made, a worker's result, a page id, read_agent's agent. A tool added later is also sent whole until someone names it in that list.

  • Cutting is opt-in through the query. A released desktop still gets whole parts. Copy transcript (CODING_AGENTS_MESSAGES) also still reads whole pages. Only the followed transcript the window draws asks for the cut.

Fetching one part whole

  • New routes: GET /api/agents/{id}/messages/{message}/parts/{index} and GET /api/agents/{id}/subagents/{call}/messages/{message}/parts/{index} (messages.agentPart / agentWorkerPart).
  • They check access the way the transcript read does (reachedAgent: a member of the agent's workspace), then read the message by id under the conversation's owner. That read is the new readMessageById in message-reads.ts.
  • Another account, an id that is not a UUID, or an index with no part all answer not-found.

Host and panel

  • New pieces from the client to the act port:
    • CodingAgentClient.agentPart
    • GATEWAY_METHOD.CODING_AGENTS_PART
    • HostOperator.codingAgentPart
    • ACT_KIND.CODING_AGENTS_PART
  • In the panel, agentParts marks cut parts, and useWholePart (whole-parts.tsx) asks for the whole part once the fold is actually open. A find that only searches inside the fold does not trigger it.
  • While the request runs, the fold shows "Reading the whole output…" / "Reading the whole thought…". It then draws the whole part through the same row, so the fold stays open.
  • If the request fails, the fold shows the two ends and says the middle was not read.
  • If a part grows after it was read (for example, a thought still streaming), it is cut again and read again.

Validation on write, not on read (packages/wire/src/ui-messages/validate.ts)

  • I checked writer.ts. Every write path that stores parts goes through admitted() → readStoredUIMessages with CODER_TOOL_SET, and stores the validator's output. There are two exceptions, and both are valid by construction:
    • the empty-parts journal insert
    • turnEnded's finishMessage, which keeps the stored inputs and adds the unknown-outcome output that storeWriter checks every tool against when it is built
  • Reading a stored row now runs the SDK's structural pass without tools. That is the shallow shape check. The checks that matter on read stay: dropping retired tool parts, the registry and dynamic-tool check, the tool-state check, and the metadata check by role. The write door still validates each tool's schemas.

Not done here

  • The host's double decode (host-operator.ts's onTranscriptChanged / codingAgentAnswer on top of the hosted client's own decode) is left as a follow-up. The decode happens on each side of the gateway's process boundary, so removing one is not a small, clean change. With the cut in place, each decode walks kilobytes, not megabytes.
  • Attachments and inline base64 (PR 4b) and the newest-first read (PR 2a) are out of scope. 2a's wireMessages callers will need the new cut argument when one of the two branches is rebased onto the other. We agreed in the chat that whoever lands second rebases.

Tests

  • coding-agents-app.test.ts covers:
    • A page with a 2 MB bash output and a long thought carries the cut output and cut text, their sizes, and their indexes. A long read_agent answer is sent whole.
    • A read without parts=cut gets the whole parts.
    • The part fetch returns the whole part to its owner and not-found to another account, for an index past the end, and for an id that is not a UUID.
  • message-reads.test.ts: readMessageById, and that it reads nothing from another conversation or another account.
  • validate.test.ts: a stored-row read accepts an output that the tool's schema refuses (the write door still refuses it), and a structurally broken row is still refused.
  • whole-parts.test.ts, which renders AgentTab through its public export:
    • a cut bash output is fetched only when its fold opens, shows the reading state, then the whole output, with the fold still open
    • a cut thought is shown whole after opening
    • a failed read shows the two ends
  • Host gateway and act-port tests cover the new method and act.

Made with Luke

Devin Review