Home: stream arbitrarily large originals in multipart pieces

Closed
TE
8d ago
+627-51

Native Home originals above the Worker streaming threshold previously used one presigned object PUT, leaving them subject to its single-request size limit. The Swift wire client now opts into bounded multipart pieces; completion streams them into an R2 multipart object and verifies the original size and SHA-256 before committing it.

  • Preserve participant checks, derived-image ordering, rate policy and existing clients' presigned protocol. Admission and byte-quota changes remain in #26409.
  • Make part writes conditional, reject invalid bounds and changed etags, batch cleanup, renew an idle lease only as pieces or assembly make progress, and abort interrupted assembly on slot expiry. Existing presigned compatibility tests stay intact.
  • Support the same protocol in the self-hosted filesystem bucket, with streaming writes, assembly and downloads; deletion-safe listing cursors keep expiry cleanup complete.
  • This PR covers the Swift wire client, API Worker and self-hosted server. The Rust cloud proxy remains on its existing transport; it needs a separate file/stream upload API before it can remove whole-file buffering.

Relevant standing order from the audit:

“No attachment size cap ("arbitrarily large files of any type"). ... Remove them (stream large files instead of reading them whole), invert the test, and add a lint that rejects MAX_*BYTES in attachment and upload paths.”

Validation:

Each streaming loop ends on EOF, bounds failure or cancellation; file handles/readers close on exit and expiry aborts incomplete assembly.

HOLD: exact-head independent review is pending.

Agent session: agent session