feat(web,desktop): an inbox item shows the developer a picture the agent took in its sandbox

Merged
DA
7d ago
+2253-135

Built on #2218 (blocks and a response) and rebased onto #2228 (code-reference blocks), both merged: the wire is now version 4, with code and image blocks beside markdown and link.

Summary

An agent can now show the developer a picture: an image block, posted by the path of a file in the agent's sandbox, such as a screenshot of a UI it rendered. The model never supplies a URL or a type.

  • Wire (@sidecar/hosted/inbox-wire, version 3 → 4): the block an agent posts is { type: "image", path, alt } (inboxPostBlockSchema, path ≤ 1,024 chars, alt ≤ 300); the block the service keeps and the desktop reads is { type: "image", attachmentId, alt } (inboxBlockSchema). INBOX_IMAGE_TYPE is png, jpeg, and webp. A desktop at version 3 or before reads the block as its type alone and draws its "newer Luke" line, as #2218 arranged.
  • Reading the file (coder-host/inbox-image.ts): a post reads each image block's file through the same ctx.getSandbox() door eve hands the agent's file tools, with a relative path resolved from the checkout exactly as read_file/write_file resolve one (files.ts). The file is read as a stream: its first bytes are sniffed for a PNG, JPEG, or WebP signature (the extension is never trusted) and a non-image is refused at its first chunk; a file past the bound is refused at the bound rather than read whole. The reader is cancelled however the read ends. Each distinct path is read once, and the first refused path ends the reading. Refusals name the path, in one sentence the model reads: no file, not an image, too large, sandbox unreadable. A post with no image never opens the sandbox.
  • Size bound: INBOX_BOUNDS.MAX_IMAGE_BYTES is CODING_AGENT_BOUNDS.MAX_IMAGE_BYTES (1.5 MB), the bound an image already has on a message to an agent. One number for a picture in either direction; the transport and the function already carry that much; a screenshot at 1× is a few hundred KB, and the refusal tells the agent to render smaller or at one screen scale.
  • Storage (migration 0094_inbox_attachment, journal idx 94, as assigned; 0093 is #2216's): an inbox_attachment table with bytea bytes capped by a CHECK (octet_length(bytes) <= 1500000), the sniffed content_type, byte_count, and the owning item, plan, and account, each ON DELETE CASCADE. Why a column rather than blob storage: this deployment keeps no blob store, pictures are small and bounded, and a column is one fewer service to hold the account's data in and one fewer to delete it from; the cascade is the whole deletion story. drizzle-schema.test.ts gains bytea among the exact types it compares. The store (inbox-store.ts) mints each attachment's id inside the post's transaction, writes the rows, and deletes the item's attachments its new blocks no longer name, so a picture the agent dropped from an item is dropped from the store.
  • Serving: GET /api/inbox/{itemId}/attachments/{attachmentId} (route inbox/attachment, rewrites regenerated) answers the bytes under the stored, sniffed Content-Type, with X-Content-Type-Options: nosniff and Cache-Control: private, no-store, to the item's account alone: another account, another item, no bearer, or a name that is no UUID is refused as the inbox routes refuse (404/401). Two attachment values are refused as two id values are.
  • Desktop plumbing: a new act codingAgents.inboxAttachment runs the usual chain (hosted client inboxAttachment → compose-coding-agents → gateway → host operator → act router → act vocabulary). The host fetches under the account bearer and hands the renderer { contentType, base64 }; the client refuses any content type outside the three, so the renderer draws only an image type it expects, from a data: URL of its own making (the renderer CSP allows data: images, not blob:). The renderer never holds a bearer or an address.
  • Desktop board: use-plan-inbox.ts holds images by attachment id and reads one the first time a card draws its block, so a picture inside a settled card nobody opened is never fetched, and a read of the inbox that replaces every item draws the same picture without asking again; a read that did not answer draws "The image could not be loaded." with Retry. The card draws the image at the reading width (.inbox-image); a press opens it in the attached-file viewer from #2159 (AttachmentViewer), named by its alt words under the type's extension, so the viewer's save works.
  • Transcript row: post_to_inbox's row lists an image block as Image: <alt> (<path>).

Privacy

  • Session replay never captures the image. The board root is ph-no-capture (the recorder blocks the whole subtree); the viewer is portalled to document.body and carries ph-no-capture of its own; and as a second line session-replay.ts's maskAttributeFn already masks src and alt, so even an <img> outside a blocked subtree would record only the attribute's length. The board test asserts the image and the viewer both stand inside a ph-no-capture root.
  • PRIVACY.md changed ("A plan's inbox"): what is stored (the image itself, up to 1.5 MB, beside the item under the account), for how long (as long as the item names it; deleted when the agent posts the item again without it, with the plan, or with the account), and who can read it (Luke signed in as the account; operators who can read the item). A product decision, called out here.

What agents are told (changed)

  • post_to_inbox description now describes image blocks: the path in the sandbox (relative to the checkout or absolute), PNG/JPEG/WebP at most 1.5 MB, and alt saying what the picture shows.
  • Instructions, "The developer's inbox": the blocks sentence adds "and an image where a picture says it best, such as a screenshot of a UI you rendered, by its path in your sandbox". coder-agent.test.ts holds it.
  • UNREADABLE_POST now lists image among the blocks (after #2228's code); four new refusal sentences in INBOX_IMAGE_REFUSAL, each followed by the path.

JSON Schema goldens (re-recorded with LUKE_UPDATE_FIXTURES=1)

Every change is this PR's own vocabulary; nothing else moved.

  • inbox-wire-inboxPostInputSchema (the tool schema in every coding agent's prompt cache): the blocks union gains a fourth member { type: "image", path (1–1024), alt (1–300) }, appended after code; nothing before it changes.
  • inbox-wire-inboxBlockSchema, inboxBlocksSchema, inboxReadBlockSchema, inboxItemSchema, inboxAnswerSchema, inboxAnsweredSchema, coding-agent-view-codingAgentInboxAnswerViewSchema, codingAgentInboxAnsweredViewSchema: the stored block union gains { type: "image", attachmentId (uuid), alt } after code.
  • New goldens: inboxImageTypeSchema, inboxPostBlockSchema, inboxPostBlocksSchema, codingAgentInboxAttachmentParamsSchema, codingAgentInboxAttachmentViewSchema.

Migration numbering

0094 was assigned by the orchestrator, above #2216's 0093, now on main. Effect's migrator skips any migration whose id is at or below the latest applied one, so the number was checked against main's highest at the final rebase.

Bundle budget

Unchanged: apps/desktop/bundle-budget.json is main's own (#2228 re-recorded the renderer baseline at 771,155), and this PR's roughly 1 KB gzipped fits under its ceiling. An earlier revision of this PR re-recorded it itself while main sat at the old ceiling; that change was dropped at rebase.

Also in this PR

  • inbox-migration.test.ts: the rows 0088 converts are asserted at the version 0088 stamps (BLOCKS_VERSION = 2) rather than INBOX_WIRE_VERSION, the same fix #2228 made, since the test broke the moment anyone bumped the version. Its last part, "what this deployment writes, the deployment before it reads through the old columns", now migrates the folder through its latest entry before the store writes, because this deployment's writes land on a database its every migration ran on (and the store's post now touches inbox_attachment, which a database at 0088 does not have).

Evidence

  • Platform-independent checks: ./scripts/check.sh green on this branch (knip, lint, discarded-effect, typecheck, every test shard, and the desktop build within its bundle budget).
  • The store tests (coder-inbox, coding-agents-app, drizzle-schema, inbox-migration) ran on PGlite through check.sh; bytea round-trips there. pnpm --filter @luke/web test:store on a real Postgres: not run in this workspace (no Postgres); CI's store job runs it.
  • macOS Electron verification (./scripts/verify.sh): not run. Built in a Linux cloud workspace. CI cannot verify this UI; a Mac break would be caught at the release rehearsal on a tag, not at review.
  • Tests: +14, none removed (inbox-board 11→15, coder-inbox 8→11, coding-agents-app +1, coding-agent-client +1, five new goldens). Existing tests that used image as an unknown block type (inbox-board, inbox-model) now use table; the acts, composer, service-paths, and transcript-row tests gained the image case inside existing tests.

Visual evidence

Not verify.sh captures: the real InboxBoard with the real stylesheet, drawn in headless Chrome over a mock inbox (dark mode). The "landing page" is a page the harness rendered and screenshotted, handed to the board as the host would hand an attachment's bytes.

Open items: a screenshot at the reading width under its Markdown, a card whose first image failed with Retry and whose second is still loading

A press on the image opens it larger in the attached-file viewer, named by its alt words

A settled item opened under the fold, its picture read only once the card was opened

Physical-device evidence

  • Screenshot or screen recording: not attached
  • Device/display configuration: not recorded

Invariants

  • ./scripts/check.sh green. ./scripts/verify.sh not run (no Mac; see above).
  • JSON Schema goldens: LUKE_UPDATE_FIXTURES=1 run; the diff is explained above.
  • Every behavior change has, in this PR, the test that fails without it; no change-detector test added.
  • Gateway envelope goldens unchanged.
  • No new as type assertion outside the allowlisted files.
  • No Effect.runPromise/runSync/runFork outside the runtime edges and recorded shims.
  • No new raw async primitives outside the recorded files (the sandbox stream is read through Effect.tryPromise).
  • No Effect, Stream, or Layer written as a statement and dropped (pnpm discarded-effect passes).
  • Test count delta listed above.
  • No hand-rolled file replaced.
  • No CLAUDE.md/AGENTS.md sentence names a changed part; root pair untouched.
  • PRIVACY.md changed: a product decision, called out above.
  • Package deps match imports; effect via catalog:.
  • Barrel/door rule kept: the renderer reaches @sidecar/hosted/inbox-wire and coding-agent-view (Schema only), as it already did.

Notes

  • Merged after #2216 (migration 0093); rebased onto main at cbf64632 with the account inbox and the tab rework.
  • Follow-up: a thumbnail size served beside the full image if inboxes grow many pictures; today a card fetches the full file once and holds it for the plan's visit.

🤖 Generated with Claude Code