fix(desktop,host-service): send attachments through storage, not host-service bodies

SA
16d ago
+841-636

Fixes SUPER-2658.

Why

Pasting or dropping an image into a cloud workspace failed with "Failed to send the file to the remote workspace" (terminal) or "Couldn't attach files / value$1 is not iterable" (chat). The desktop shipped the bytes as a base64 filesystem.writeFile body over the streaming tRPC link (httpBatchStreamLink). tRPC v11 returns a streamed batch's response head before it has read the request body (getRawInput is lazy), and a Vercel sandbox's ingress (sb-*.vercel.run, which the gate forwards to) treats that early response as the end of the exchange: the rest of the upload is dropped, the procedure never runs, and the client sees a stream that ends after the head line.

It is a race, not a size limit, which is why it looked intermittent: measured against a real box with Expect: disabled (browsers never send it), a 500 KB body at 1 MB/s failed 2/2, 2 MB at full datacenter speed passed, 6 MB failed, and the same payloads in plain (non-jsonl) mode passed 8/8. Against the host over localhost it never fails. Sandboxes moved to Vercel on Sep 11 (#7408); the Aug 20 paste fix (#6692) was verified on Blaxel's ingress.

What

Desktop — the chat composer, the terminal rich input, and the terminal paste/drop override now take the route the mobile composer and the new-workspace modal already take: attachment.createUpload → presigned PUT to storage → cloudWorkspace.attachFiles (cloud only) → attachments.materializeIntoWorkspace on the host. The host-service request carries ids, never bytes. uploadPastedFiles (base64 over filesystem.writeFile) is deleted; sendAttachmentsToWorkspace + useSendAttachments replace it. Limits come from @superset/shared/attachment-limits like everywhere else.

host-service — /trpc/* buffers the request body before handing it to tRPC, so no response starts before the request is complete. materializeIntoWorkspace and the chat-v3 image writer share one ensureWorkspaceAttachmentsDir helper, so the directory always gets the .gitignore the old desktop helper used to write (materialize never wrote one).

Docs — docs/cloud-sandbox-mismatches.md describes the ingress race, the curl Expect: 100-continue trap, and the attachment route.

Mobile already does exactly this; nothing to change there.

Verified

  • packages/host-service/test/integration/trpc-request-body.integration.test.ts: a streamed batch with a body that is still open does not get a response until the body closes. Fails without the app.ts change (checked by stashing it), passes with it.
  • sendAttachmentsToWorkspace.test.ts: upload order, cloud attach-before-materialize, naming of nameless blobs, size/count limits refuse before any upload, a rejected PUT never materializes.
  • bun run typecheck in apps/desktop and packages/host-service; Biome on changed files; host-service attachment and chat-image suites; bun run check:i18n (only the removed "Workspace path is not available yet" message changes in the catalogs).
  • Not exercised end to end in the desktop app from this sandbox (no Electron here). The underlying failure and the plain-mode fix were reproduced against a live box with curl, and the storage route is what mobile ships on.

Open

A relay-reached personal machine now also round-trips attachments through cloud storage, where before the bytes went straight to that host. The attachment rows stay pending and the file sweep reclaims them a day later, so nothing is kept, but if we want "never stored" for personal hosts, that's a createUpload retention flag plus a delete after materialize — separate change.

Also in this PR: attachment.resolve scope and reap (from #8712, SUPER-2874)

#8712 is folded in as its own commit (10a2b744e5) and closed. It is API only (packages/trpc), with no file overlap with the changes above:

  • attachment.resolve is scoped (router/attachment/resolvable.ts). A box resolves only files anchored to its own workspace (unchanged). A kept (ready) file resolves only through a cloud workspace the caller can see. An upload still in flight (pending) stays org-wide, because a host resolves with its owner's credential and the sender may be a teammate on that host. materializeIntoWorkspace above depends on this.
  • New attachment.resolveEach: same input and scope, one entry per id, { fileId, missing: true } for a file that is gone or not visible. resolve keeps its output and its NOT_FOUND / BAD_REQUEST errors for released hosts.
  • Cloud workspace files are reaped with the box: the archive's final delete stage, and the cloudWorkspace.delete fallback, run detachAll + reapOrphanFiles.

tRPC compatibility: resolve has the same input, output, errors and procedure type. Auth is stricter only for ready files, and no released non-box caller resolves a ready file. resolveEach is additive. The full analysis is in #8712's description.

Re-checked on this branch: packages/trpc bun run typecheck passes, bun test passes 660 tests, and Biome is clean on the changed files. resolvable.integration.ts is a manual DB test. It passed against a Neon branch in #8712 and was not re-run here.

The #8712 commit conflicted with current main on one import line in cloud-workspace.ts. Merge commit 05165d4479 merges main into this branch and keeps both imports. After the merge, packages/trpc typecheck passes and bun test passes 700 tests, and packages/host-service and apps/desktop typecheck.

Summary by cubic

Fixes SUPER-2658. Pasting or dropping an image into a cloud workspace failed with "Failed to send the file to the remote workspace" or "value$1 is not iterable" because the desktop shipped file bytes as a base64 filesystem.writeFile body over the streaming tRPC link, and a Vercel sandbox's ingress drops the upload when tRPC answers the batch before the body is fully read. It's a race, not a size limit.

What changed

  • Desktop chat, terminal rich input, and terminal paste/drop now send attachments through storage (attachment.createUpload → presigned PUT → cloudWorkspace.attachFiles → attachments.materializeIntoWorkspace), the same route mobile and the new-workspace modal use; the host-service request carries ids, never bytes.
  • uploadPastedFiles is replaced by sendAttachmentsToWorkspace and useSendAttachments; limits come from @superset/shared/attachment-limits.
  • host-service now buffers /trpc/* request bodies before handing them to tRPC, so no response starts before the request is complete; both attachment paths share ensureWorkspaceAttachmentsDir, so the attachments .gitignore is always written.

One side effect: a relay-reached personal machine now also round-trips attachments through cloud storage where bytes previously went straight to that host; attachment rows stay pending and the sweep reclaims them a day later.

Summary by CodeRabbit

  • New Features

    • Upload files and images to workspace chat and terminal sessions, including remote workspaces, without needing a locally available worktree path.
    • Attachment-limit errors now explain file count and size restrictions.
    • Attachment resolution reports missing files individually while preserving request order.

  • Bug Fixes

    • Improved reliability for remote uploads and streamed requests.
    • Preserved existing workspace attachment ignore settings.
    • Cleaned up associated files when archived cloud workspaces are deleted.