feat(web): a coding agent's message and task carry attached files to the model and the transcript (attachments 1/2)

Merged
DA
7d ago
+831-41

What

Part 1 of 2 of Attach files in the agent composer: the service side. A coding agent's message, or the task it starts on, can now carry files. The model sees them, and the developer's message in the transcript keeps them after a restart. Part 2 is the desktop UI, stacked on this PR.

What's supported

  • Images (PNG, JPEG, WebP), up to 1.5 MB each. The desktop re-encodes larger images to WebP and turns GIF/HEIC into WebP. eve gets them as AI SDK file parts over data: URLs.
  • PDFs, up to 3 MB. They go as file parts. Every model the coder runs on is Anthropic or OpenAI (coder-host/model.ts), and both providers take a native PDF part (@ai-sdk/anthropic document, OpenAI Responses input_file), so nothing is extracted. Both providers also take images, so there is no per-model vision check.
  • Text and code files, any extension, strict UTF-8 (the wire checks it), up to 100 KB. Each one is inlined for the model as its own text part: Attached file: <name> followed by the file in a fence longer than any backtick run inside it. No provider path takes text/plain as a file (OpenAI throws), so the file is inlined instead.
  • Per message: at most 10 files and 3 MB together, which is about 4 MB of base64. That stays under Vercel's 4.5 MB function body limit. The route body caps rise to fit.
  • A message must still include words. The desktop keeps Send disabled for files with no words; see PR 2.

Wire (coding-agent-wire.ts)

  • codingAgentMessageRequestSchema and codingAgentStartRequestSchema take an optional attachments: [{ name, mediaType, data }].
  • mediaType is one of image/png | image/jpeg | image/webp | application/pdf | text/plain. data is bare base64, bounded by its type.
  • The array is bounded by count and by total bytes. A Start may carry files only when it also has a task.
  • Four JSON Schema goldens were re-recorded under LUKE_UPDATE_FIXTURES=1. The only change in each is the new optional attachments property, purely additive. These are request goldens and no model is shown them.

Model path

  • steer.ts and start.ts hand eve AI SDK UserContent (attached-files.ts's eveContent): the words, then the inlined text files, then the image and PDF parts.
  • A message with no files still goes as a plain string, exactly as before.
  • eve 0.74 stages file bytes in the sandbox when one is up.
  • The coder channel now sets an eve uploadPolicy: only those four media types, at most 3 MB each. eve's default was 25 MB of anything.

Journal and transcript

  • The developer's row keeps the words as part 0. Each file follows as an AI SDK file part with a data: URL (text files under text/plain).
  • The transcript route already returns parts verbatim, so the desktop reads files back with no new read path.
  • Receipt matching: eve's message.received.data.message flattens files into [file: …] placeholders, so the relay can no longer match a line by that string. It now matches on data.parts' text parts joined by \n\n. That equals what each row's words plus inlined text files join to, whether or not eve coalesced several steers. Without this, a line with files would be journaled twice.
  • A receipt the relay writes as its own row, such as a task's first message, keeps its words in part 0 and its files as file parts. Text files are read back from their heading. A file eve staged in the sandbox carries no URL, so it keeps its name with url: "".

Storage: inline, not blob storage

  • Files live in messages.parts jsonb as data: URLs. There is no blob store in apps/web today; adding one means a new dependency and secret. The board PNG is already stored inline as base64 up to 3 MB.
  • How big it can get: one message holds at most about 4 MB of base64. TOAST compresses base64 back by roughly a quarter. A conversation with 20 screenshot-bearing messages at a typical 0.3–1 MB each holds about 6–20 MB in its messages column, and up to about 80 MB if every message carried 3 MB.
  • The first transcript read of such an agent downloads those rows once. Later reads are past the cursor, so they don't re-download them.
  • Follow-up if volume grows: move the bytes to blob storage and keep a reference in the part.

Secrets and telemetry

  • Nothing new reaches traces, analytics or fixtures. Neither eve project has instrumentation, product events are an allowlist with no free text, and no log line carries content. The desktop tab is ph-no-capture.

Tests

  • coding-agents-app.test.ts:
    • A message's files reach eve as the exact UserContent above, and the transcript reads the row back with every file.
    • Each refusal: unknown type, non-UTF-8 text, non-bare base64, nameless file, over its per-type bound, 11 files, over the total. Files at the total bound alongside 16k chars at JSON's widest escape are accepted, which proves the body cap math.
    • A task's files open the session, and files without a task are refused.
  • coder-host.test.ts: a line with files and a plain line before it are both taken by one coalesced receipt, with no second row. An unmatched receipt is written with its words and its files: an inlined text file (with a fence inside), a PDF with a URL, and a staged image without one.

./scripts/check.sh exits 0. This PR does not change the Mac app; part 2 does.

šŸ¤– Generated with Claude Code


Devin Review