Throw DataCloneError when structured-cloning an S3File

RO
22d ago
+252-71

Problem

  • structuredClone(s3file) throws TypeError: Unable to deserialize data. postMessage(s3file) returns, and the receiver gets messageerror. A 5-byte slice clones into 5 wrong bytes with no error.
  • Each attempt also sets size of the source to 0. Then file.text() reads 1 byte of 10 and client.write(key, file) uploads 1 byte. No user report exists.
  • Cause: _on_structured_clone_serialize (src/runtime/webcore/Blob.rs:726) tags an S3 store as Bytes, writes a File body, and calls resolve_size(), which sets the size to 0.

Fix

  • structured_clone_store_tag() returns None for an S3 store. The serialize hook then throws a DataCloneError before it writes a byte or calls resolve_size(). A Blob built from an S3File throws too.
  • Store::serialize becomes File::serialize, without the S3 and Bytes arms. The deserializer passes check_s3 = false: a File record with an s3:// path builds a local File store, never an S3File with the environment credentials.
  • Verified: test/js/web/structured-clone-blob-file.test.ts (18 of 19 new tests fail without the fix). Notes list the other suites.
  • Self-reviewed: 15 concerns raised, 10 addressed. The other 5 concern Blob routes outside this PR.

Background

  • A Blob store is Bytes, File or S3. The clone record has no S3 tag and no field for credentials.
  • Considered: open the key with the receiver's credentials (possibly another bucket), or put the credentials in the record (a secret key in each buffer).

Downsides

  • No maintainer has confirmed the new contract in the docs: an S3File cannot be cloned. postMessage(s3file) now throws at the sender.
  • expect(s3file).toHaveLength() and toBeEmpty() still set size to 0 through resolve_size(). #43934 changes that function.
  • Cost for other blobs: none found. Serialize swaps two store checks for one match. Deserialize drops one prefix compare.
Notes

Questions for a maintainer.

  1. Is "an S3File cannot be structured-cloned" the wanted contract? The two other designs are under Background. A throw is the least binding choice: a later change can still make the clone work.
  2. The deserializer still rebuilds a local File store from any File record that comes from raw bytes. #31329 covers that class.

Measured on a release build of main (1.4.3 canary, 367d939d9) and on a debug build of this branch. The S3 endpoint is a Bun.serve on loopback that serves a 10-byte object.

Act on file = client.file("x.txt")mainmain, after the actthis branch
none (control)size is NaN, text() reads 10 bytes, client.write("copy.txt", file) uploads 10 bytessame
structuredClone(file)TypeErrorsize is 0, text() reads 1 byte (Range: bytes=0-0), write() uploads 1 byteDataCloneError, then the same as the control
port.postMessage(file)returns, the receiver gets messageerrorthe same as the row aboveDataCloneError, then the same as the control

Other results on main, with no I/O:

  • structuredClone(file.slice(0, 5)) returns a Blob of the bytes 00 00 00 00 01, and size of the slice goes from 5 to 0. The reader takes the low 32 bits of the u64 size as the byte length. The other windows that were tried throw the TypeError, except slice(2, 7), which returns 3 wrong bytes.
  • serialize(file) returns 49 bytes that deserialize() rejects.
  • new Blob([file]), new File([file], "n.txt") and formData.get() of an appended file are not S3File objects. They share the S3 store, throw the TypeError, and get size 0.
  • A File record with the path s3://bucket/key, written by hand, deserializes to [object S3File] signed with the environment credentials.

On this branch each of these throws, and size does not change. The hand-built record deserializes to a plain [object Blob] over a local File store.

Why the clone is refused and not made to work. An S3 store is a key plus refcounted S3Credentials (endpoint, bucket, region, keys) and the upload options (ACL, storage class, requester pays, multipart settings). The old record carried only the key as the user typed it. A receiver that opens that key again uses the credentials of its own environment, and a bare key (client.file("key")) would open a local Bun.file("key"). A DataCloneError at serialize time is what the structured clone algorithm specifies for a platform object that cannot be serialized. Bun already throws it for a Response and, since #32799, for a detached ArrayBuffer.

A third design keeps the same behavior and makes the generated serialize hook return JsResult<()> (branch robobun/22c566ef/s3file-structured-clone). It changes src/codegen/generate-classes.ts and BlockList.rs too. This PR throws from the existing hook, as 52 other let _ = global.throw_*() sites in src/runtime and src/jsc do. CloneSerializer checks for a pending exception after each hook.

The two arms of Store::serialize that are gone. The S3 arm wrote a path record under the Bytes tag, which is the wrong record of this bug. The Bytes arm had no caller: the Blob serializer writes the bytes of its own window inline.

Self-review, the 5 concerns with no change here. Each one is about a route that this PR does not touch.

  • Other entry points also take an S3 store down the in-memory arm of a two-way store check. #43934, #38364 and #39082 each cover one route. A change to the shared check is a separate PR.
  • resolve_size() still sets an unknown S3 size to 0 for its other callers (#43934).
  • Other Blob states that the clone record cannot describe. They are not S3 stores.
  • Two concerns are about object URLs of an S3File.

Related PRs.

  • #43934 changes the S3 arm of resolve_size() and removes the size change for every caller. The size assertions here pass with or without it. Without this PR the rows still fail, because they expect the DataCloneError.
  • #38364 covers the FormData route. #39082 covers the WebSocket client.

Not changed on purpose.

  • Serialization for cross-process transfer (process.send) turns each Blob into {} before it reaches the Blob serializer. An S3File still arrives as {}, the same as Bun.file(). A test asserts this.
  • The S3 arm of Blob::resolve_size() (see Downsides).
  • new Blob(["x", s3file]) with more than one part. It is a Bytes store that holds only x, and it clones as before.

Receiver side. The deserializer passes check_s3 = false to find_or_create_file_from_path. A File record with an s3:// path builds a local File store with that path. That is what a Bun from before S3 support wrote for Bun.file("s3://..."), so such a record still deserializes. An earlier revision rejected the record with a hard error. A review pointed out that this broke old persisted records, so the reject is gone.

Tests. The block "S3File structured clone" has 19 tests. It covers five ways to make an S3File, two slices (one of 5 bytes), three plain wrappers over an S3 store (new Blob([s3file]), new File([s3file], name), formData.get()), each entry point (structuredClone, serialize of bun:jsc and node:v8, postMessage of MessagePort and Worker, the for-storage flag), an S3File nested in a larger graph, the {} result for cross-process transfer, a matrix of the other stores that still clone, and the hand-built s3:// record (a local File store, never an S3File). The rows assert that size of the source does not change. No test does S3 I/O.

Suites run with the debug build of this branch (merged with main at 7c96a1212f).

  • test/js/web/structured-clone-blob-file.test.ts: 61 pass. 18 fail on the release build of main.
  • test/js/web/workers/structured-clone.test.ts: 244 pass (with --timeout 180000, the 2 GiB ArrayBuffer tests need more than 5 s on a debug build).
  • test/js/web/workers/structuredClone-classes.test.ts: 57 pass.
  • test/js/web/structured-clone-fastpath.test.ts: 93 pass.
  • test/js/bun/jsc/bun-jsc.test.ts: 41 pass.
  • 16 files of test/internal/source-lints/: 103 pass.
  • The repro under BUN_JSC_validateExceptionChecks=1: no report.
  • cargo clippy -p bun_runtime --no-deps: clean. bun run rust:check-all x86_64-pc-windows-msvc: ok. tsc --noEmit in test/: no error in the test file.