#26673 NEXT: the self-hosted Home bucket streams multipart uploads of any size

Part 1 of 2 of #26432 (R2 multipart transport for Home attachments; its session is gone, the attachments lane owns it now), split to keep each PR under ~800 lines. Part 2 is the Worker transport, stacked on this.

Leo's standing order: "Attachments take arbitrarily large files of any type; there is no size cap" (Leo, 2026-10-26: "WHY THE FUCK SHOULD THERE BE A FUCKING SIZE CAP ANYWAYS ... I ALREADY SAID WE NEED ARBITARRILY LARGE FILES OF ANY TYPE").

What changes

The self-hosted server's R2 stand-in (backend/apps/home-server/src/fs-bucket.ts) supports multipart uploads of any size:

• Each part streams to its own file, never held in memory.
• Complete assembles the object.
• Listing cursors stay valid across deletes.
• Part numbers outside 1..10,000 (R2's range) are refused. That was #26432's review finding 2 on this side.

Risk: normal (self-hosted server only; no caller changes in this PR).

Red / green

cd backend/apps/home-server && bun test: red at 7639d99d17b (the multipart streaming test fails on base's bucket), green at bd964c05143 (11 pass); bun run typecheck clean.

Review finding (Codex Astra, [P2] fs-bucket.ts:38, a key such as ./.multipart/<id>.json normalized into the internal upload directory, so a put could rewrite an open upload's record and the upload could then be completed as another object): red at 62a2397d9d4, green at f062fa26b4f. Every key-to-path route now goes through one helper, keyPath. It refuses empty, . and .. segments, backslashes, NUL and a leading /. It also checks the normalized path, which must resolve strictly inside the root and not under its top-level .multipart. An inner segment such as home/v1/.multipart/b is still an ordinary key. Tests in test/fs-bucket.test.ts:

• "refuses keys with '.', '..' or empty segments, backslashes or absolute paths, so no key reaches outside the root or into .multipart": fails at red (put("./.multipart/<id>.json") resolved), passes at green.
• "refuses to overwrite an open upload's record through a key that normalizes into .multipart, so the upload can't be redirected": fails at red (the forged record put resolved), passes at green.
• "lists only stored objects and never walks the internal .multipart directory" (extended): list skips a file no key could have written.

At green, bun test gives 18 pass, 0 fail, and bun run typecheck is clean.

Review findings (Codex Astra, at f062fa26b4f): (1) containment was lexical, so a symlinked directory under the root let put, get and delete reach outside the bucket, and list() followed links or threw on a dangling one; (2) list({ limit: 0 }) crashed on page.at(-1)!.key whenever objects existed. Red at eef51a3f099, green at 810fbee7cf5.

• The root is resolved once at construction, so a configured root that is a link still works. Below it, each parent directory is checked with lstat (one call per segment, no walk). Put and completion make missing parents one at a time, checking each one. Object, metadata, record and part files are opened with O_NOFOLLOW. New files are created O_EXCL|O_NOFOLLOW and renamed into place, so a link at the destination is replaced rather than written through. .multipart and a parts directory that are links are refused, and list() skips every link.
• list() refuses a limit that is not an integer from 1 to 1,000 with InvalidMaxKeys, matching R2 (miniflare: "MaxKeys params must be positive integer <= 1000.").

Tests:

• "never follows a symbolic link out of the bucket: a linked directory, a linked file or a dangling link": fails at red (put("linked/new") through a linked directory resolved), passes at green.
• "refuses a list limit outside 1 to 1,000 as R2 does (InvalidMaxKeys), and pages with any limit inside it": fails at red (limit 0 was not refused; with an object stored it threw TypeError: undefined is not an object (evaluating 'page.at(-1).key')), passes at green.
• "accepts a root that is itself a symbolic link, and stores in the directory it names": a guard that passes at both red and green.

At green, bun test gives 21 pass, 0 fail, and bun run typecheck is clean.

Review findings (Codex Astra, at 810fbee7cf5): (1) the reserved .multipart check was case-sensitive, so on a case-insensitive disk (APFS, NTFS) .MULTIPART/<id>.json reached an open upload's record; (2) parents were checked with lstat and then used by name, so a directory swapped for a link between check and use led outside the bucket. Red at eac200e1420, green at e606b2e5b81.

• The first segment is reserved when it folds to .multipart. Folding drops default-ignorable code points, applies NFKC, folds case (upper, then lower), applies NFKC again, and drops trailing dots and spaces. I chose NFKC over NFC so that compatibility lookalikes such as fullwidth ．ｍｕｌｔｉｐａｒｔ fold too. This folds more than any of those filesystems does, so it refuses at least every name that aliases. list() also skips a folded .multipart.
• Node and Bun have no openat, so this uses the fallback:
  • A file opened O_NOFOLLOW must have a realpath inside the realpath'd root. It must also be in the area the caller expects: the key space, or .multipart for the bucket's own records and parts. Its inode, from stat of that realpath, must match fstat of the descriptor.
  • A directory is realpath-checked after put creates it, and again right before a file is created in it, renamed into it or removed from it.
  • A test-only constructor option, afterPathCheck, runs between the lstat walk and the use, so the test swaps a directory for a link at each checkpoint deterministically, with no real race.
• Residual window (also documented on FsBucket): a swap after the last realpath check and before the rename, unlink or mkdir call itself. Closing it needs renameat, unlinkat and mkdirat on a held directory descriptor, which Node lacks.
• Tests:
  • "refuses any key whose first segment folds to .multipart, so a case-insensitive disk can't reach an upload record": fails at red (put(".MULTIPART/<id>.json") resolved), passes at green. It covers .MULTIPART/<id>.json, .Multipart/x, fullwidth, zero-width-joiner, soft-hyphen and trailing dot or space forms, and the forged-record attack through the upper-case key.
  • "refuses an operation when a checked directory is swapped for a link before it is used, at every point between check and use": fails at red (the bucket had no seam), passes at green. It runs get, head, delete, put, put creating directories, complete, upload part and complete from .multipart once for each checkpoint, with the swap at that checkpoint.
  • Mutation check: with the directory re-check disabled, the swap test fails at delete @1. With the file re-check disabled, it fails at get @1. With an exact (unfolded) reserved check, the fold test fails.
  • 22d17abf00a hardens the swap test: the swapped .multipart copy now holds different part bytes. With the upload-area file re-check disabled it fails at "complete from .multipart @4", because the forged part is read.

Carried over from lane/attach-multipart-fs-bucket-gc (#26674's fs-bucket base)

#26674's fixes depend on four commits, cherry-picked in order on top of the fix above:

• Custom metadata: put and createMultipartUpload keep customMetadata, and a completed object carries its upload's metadata, as R2 does. Red 9142756d7b4 (from 1dcd44f05df), green 24c4c759e77 (from 987846a6bc3). The writes now go through the checked writer.
• abortMultipartUploads(prefix): aborts every open upload under a key prefix. Red 5fb33b085fe (from 8c91a58ac96), green 1eaf8bcab77 (from 19323c2ce45).
  • The fix is adapted to the checks above. .multipart is lstat-checked, opened, then realpath-checked as the upload area. Each record is read with the no-follow, realpath- and inode-checked reader. Keys go through isKey, which includes the folded reserved check.
  • My own red test 3b27c6e7a51, "aborts uploads by prefix only through the bucket's own .multipart: a linked directory, a linked record or a swap after the check is refused", fails at red and passes at green.

At head 22d17abf00a, bun test gives 26 pass, 0 fail (603 expect() calls, 4 files), bun run typecheck is clean and prepush-gate.py reports no new violations.

Review findings (Codex Astra, at 22d17abf00a): (1) [P1] nothing called abortMultipartUploads, so abandoned uploads stayed on self-hosted disks; (2) [P1] put and completion renamed the data and then wrote the .meta.json sidecar after an await, so concurrent writes to one key could pair one write's bytes with another's size and etag. Red at 9c8b37b2a15, green at 59d95fb3905.

• deletePrefix lives in backend/apps/api/src/home-attachment-gc.ts (it loops list and delete; FsBucket has no prefix delete). It now calls the bucket's abortMultipartUploads(prefix) when the bucket has one. That is four lines; R2 has no such method, so the Worker is unchanged. Conversation deletion and its later purge both go through it, so #26674 and #26701 only add slot bookkeeping on top.
• put and completion stage the data and the metadata as temp files, then commit both in one synchronous step: metadata renamed first, data last.
  • The home-server process is the bucket's only writer, and a step with no await runs whole in it. So no other put, completion, delete or read of that key runs in between, and that step is the per-key lock; no Map of promises is needed.
  • A conditional put (etagDoesNotMatch: "*") is decided inside that step, so only one of several racing puts to a new key wins.
  • A crash between the two renames leaves metadata whose size the data doesn't match. Reads answer that as no object until the next write. Residual: with a same-size overwrite, the new etag would show on the old bytes.
• Tests:
  • home-server test/attachments.test.ts, "deleting a conversation's storage also ends its open multipart uploads, so no parts stay on disk": fails at red (the abandoned upload's record and parts remained), passes at green.
  • test/fs-bucket.test.ts, "commits each write to a key whole: concurrent puts and completions never leave one write's bytes with another's metadata": fails at red in 10 of 10 runs, and the mismatch and conditional-put checks each trip on their own. Passes at green in 5 of 5 runs.
• Checks at green: home-server bun test gives 28 pass, 0 fail, and bun run typecheck is clean. backend/apps/api tsc -p . is clean.

Changelog

none

Agent session: agent session

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

---

Summary by cubic

Opens the self-hosted Home bucket to multipart uploads of any size: the R2 stand-in streams each part to its own per-upload file instead of holding it in memory, and assembles the object on completion.

• Operations on one upload (part upload, complete, abort) are serialized, so a completion never reads a half-replaced part.
• Completion validates the whole parts list — ascending numbers, no repeats, matching etags — before writing anything; part numbers outside 1–10,000 are refused, and the upload id is checked before the part number, so forged, cross-key or closed uploads answer NoSuchUpload. resumeMultipartUpload verifies the persisted record when the handle is created.
• Keys are validated by their normalized path: none can resolve outside the root, into the top-level .multipart directory, or through a symbolic link. Directories swapped for links after being checked are caught by re-checking every path where it lands, and renames and deletes run through held directory descriptors.
• Listing skips .multipart, refuses a limit outside 1–1,000 (InvalidMaxKeys), and cursors reference keys, so they stay valid across deletes.
• put and multipart uploads keep customMetadata, and a completed object carries its upload's metadata.
• Each write to a key commits whole: data and metadata are staged as temp files and renamed in one synchronous step, and the metadata carries a content digest every read recomputes, so a torn commit never serves bytes under a stale etag. Only one of several conditional puts to a new key wins.
• Metadata written before the digest existed is still readable (its etag counts as the digest), and listing removes orphaned data files a crash leaves between the two renames.
• Deleting a conversation's storage aborts its open multipart uploads, so abandoned records and parts do not stay on disk.

This is part 1 of 2; the Worker multipart transport lands after it.