formData(): read a file- or S3-backed Blob body before parsing it

RO
24d ago
+483-7

Problem

  • formData() on a Response or Request over Bun.file() resolves an empty FormData and never reads the file. Node parses the same body from fs.openAsBlob().
  • A multipart file rejects with FormData parse error missing final boundary. Bun.file(p).stream() as the body parsed in 1.3 and is empty since 1.4.0.
  • Cause: get_form_data (src/runtime/webcore/Body.rs:2130) parses blob.slice() at once. That slice is empty when the bytes are on disk or in S3.

Fix

  • After the Content-Type check, FormDataBodyRead takes a Blob whose bytes are on disk or in S3. It reads them off the JS thread, then parses.
  • A read failure rejects as text() does (ENOENT, EISDIR). A parse failure keeps ERR_FORMDATA_PARSE_ERROR.
  • Verified: test/js/web/html/FormData.test.ts (33 new cases, all fail on a debug build of main). Also the body, image and s3 suites.
  • Self-reviewed: 17 concerns raised, 17 addressed.

Background

  • A Bun.file() is a Blob with a lazy store: a path, a file descriptor, or an S3 key.
  • read_bytes_to_handler (Blob.rs) reads any store and hands over an owned buffer. Bun.Image was its only user.
  • Considered Blob::to_form_data (8da2c60b57 in #31674): it rejects with a plain Error. Considered a stream: it reads on the JS thread.

Downsides

  • Behavior change: formData() on these bodies waits for the read and can reject.
  • Bodies in memory: -2 to +3 instructions per call, no added allocations or syscalls. Release .text: +3,840 B.
  • Still wrong: FormData.from(Bun.file(p)) is empty. An S3 read failure has name: "Error", not S3Error.
Notes

Where this came from. No user reported it. It was found during the work on #44999. There is no issue for it.

Repro. The file holds a=1&b=2.

const type = "application/x-www-form-urlencoded";
await new Response(Bun.file(p, { type })).formData(); // main: {}   here: { a: "1", b: "2" }
await new Response(Bun.file(p), { headers: { "content-type": type } }).formData(); // main: {}
await new Request("http://a/", { method: "POST", body: Bun.file(p, { type }) }).formData(); // main: {}
await new Response(await fs.openAsBlob(p, { type })).formData(); // main: {}   node v26.3.0: { a: "1", b: "2" }

History, from released binaries ([] is an empty FormData):

formData() of1.1.01.2.01.3.01.4.01.4.2mainhere
new Response(Bun.file(p), { headers })[][][][][][]parsed
new Response(Bun.file(p), { headers }) after .body was readparsed[][][][][]parsed
new Response(Bun.file(p).stream(), { headers })parsed[]parsed[][][]parsed
a clone() of thatone side[]parsedparsedparsed[]parsed
new Response(Bun.stdin.stream(), { headers }), stdin a pipeparsed[][][]parsed
  • The direct Bun.file() body never parsed. The parse of blob.slice() dates from aa0762e466 (#2051).
  • The stream forms changed with the lift of an unread file stream back into its Blob. 289f5321e6 (#36624) is first in 1.4.0 and b5ba14b612 (#42053) is first in 1.4.3. The lift stays: the other readers need it, and they check where the bytes are. formData() was the one reader that did not.

Workaround on a released version. new Response(await r.arrayBuffer(), { headers: r.headers }).formData().

The change. Body.rs only, +83 -6. The gate is the if let AnyBlob::Blob(source) = &blob && (source.needs_to_read_file() || source.is_s3()) after use_as_any_blob(). The Content-Type decision and the stream (Locked) branch above it are as before, so a wrong type still rejects before anything is read. FormDataBodyRead owns the promise and the encoding. read_bytes_to_handler gives it the bytes, or the error, on the JS thread.

Why not Blob::to_form_data. 8da2c60b57 in #31674 put the same gate here and sent the body to the Blob method. That PR has conflicts and no push since 2026-06-10. This PR replaces that commit. The Blob method takes the encoding from the Blob's type, so the commit wrote the header's type onto the Blob first. A Blob type is lower case, and a boundary is not. The Blob method also rejects a parse failure with a plain Error, and on main it does not free its read buffer (#44888 and 675d347bc1 in #31674 each fix that, neither is merged).

What formData() reads now. One gate covers each way to a file or S3 Blob body:

  • Bun.file(path), Bun.file(fd), Bun.stdin, a slice, fs.openAsBlob(), new Blob([file]).
  • A Request whose body is s3.file(key). new Response(s3File) is a redirect and has no body.
  • .body read first, Bun.file(p).stream() as the body, and a clone() of each.
  • The Response of fetch("file://...") and of fetch(URL.createObjectURL(file)). The second one is also how a Response gets an S3 body.

Not fixed here.

  • The class. Blob::shared_view() and Any::slice() are empty for a file or S3 store. Other APIs read such a Blob as zero bytes: Bun.hash(file) equals Bun.hash(""), and new Blob([file, "x"]).size is 1. formData() lands alone because it already returns a promise and its four sibling readers already read. The others are synchronous and need a decision: read or throw. #33600, #36836, #37905, #38364, #39082 and #41193 each take one site.
  • FormData.from(Bun.file(p)) returns an empty FormData with no error (FormData.rs:137). It is synchronous, so its fix is a throw. That is a behavior change on another public function.
  • Value::resolve's GetFormData arm (Body.rs:1146) has the same synchronous parse. Its four callers pass bytes in memory, so it cannot see a file.
  • .body read first when only Bun.file(p, { type }) carries the form type still rejects with incorrect MIME type/boundary. The type check runs before the read that this PR adds. That is a separate bug. #42015 and #42217 are open for the Content-Type of a body that was read first.
  • A parse failure still has three other shapes elsewhere: Error "FormData X" for a pending fetch or Bun.serve body, Error "X while parsing FormData" for a JS stream body, and Error "FormData encoding failed: X" for Blob.formData(). This PR matches the shape of a body in memory.
  • An S3 read failure rejects with { name: "Error", code: "NoSuchKey", message, path, syscall: "fetch" }. text() rejects with { name: "S3Error", code, message, path }. Code, message and path are equal. read_bytes_to_handler flattens the failure for each of its users (Blob.rs:574), and Bun.Image ships the same shape. A new ReadBytesResult arm would keep the name, but #33312 matches that enum with two arms, so the arm would break its build.

Self-review. The review asked for these changes, and each one is in:

  • Keep the source change in one file. An S3 failure keeps the shape that read_bytes_to_handler gives it, and ReadBytesResult gets no new arm (see the last bullet above).
  • No JSPromise::wrap around the start of the read, and no bytes.is_empty() clause in the gate. No test needed either one.
  • Tests for a pipe on stdin, fetch() of a blob: URL (file and S3), a clone() of a stream body, the async stack of a read failure, and Bun.gc(true) while the read is pending. A later case covers bytes over the string limit: a file body and the same bytes in memory both reject with ERR_STRING_TOO_LONG.
  • A faster Worker case (about 1 s on a debug build). It stays in FormData.test.ts and is not a row in worker-late-completion.test.ts. On a 12-core machine the rows of that file time out under bun bd test at the default 5 s.
  • Name each exclusion, and put the history in this text.

Measurements. Release builds of base 7c96a1212f and of the first revision of this branch, linux x64. The later commits are a rebase, test changes, and one statement moved in the cold start.

  • In-memory formData(): -2 to +3 self instructions per call in get_form_data (Response string body 113 -> 114, Request string body 136 -> 137, Blob and URLSearchParams bodies +3. gdb nexti, pc inside the function). A body behind a stream and a pending Bun.serve body: +0 (131 -> 131, 154 -> 154, 123 -> 123).
  • In-memory formData(): +0 mi_malloc, +0 mi_free, +0 promises, +0 syscalls per call (13 / 7 for a string body on both builds).
  • Release binary: +3,840 B .text (65,818,341 -> 65,822,181). get_form_data 1,376 / 1,732 B -> 1,347 / 1,710 B. New cold symbols 3,061 B. read_bytes_to_handler instantiations 1 -> 2 (1,005 B).
  • Host functions +0, codegen rows +0, .d.ts lines +0.
  • File-body formData(): 1 openat / 1 fstat / 2 read / 1 close per call, 0 on the JS thread (text(): 1 / 1 / 2 / 1, 0). A ptrace counter over 1,000 calls, strace is not installed here.
  • File-body formData(): 1 Promise, 0 ReadableStream per pending call (text(): 1, 0). 3 mi_malloc in the call (text(): 2).
  • S3 Request body: 1 GET per formData() call (0 on the base). Range header on an unsliced body: none. A slice sends bytes=5-20.
  • LeakSanitizer on FormData.test.ts: 0 leak reports, exit 0. RSS over 40 x 8 MiB: formData() 43 -> 44 MB, text() 33 -> 34 MB.
  • Fields that differ from text(): ENOENT 0, EISDIR 0, NoSuchKey 2, S3 sign failure 2 (name and syscall, see above). From the in-memory parse error: 0.
  • Bun.Image S3 read failure: 0 of 5 fields changed. This PR does not touch that code.
  • Worker teardown with a pending formData() read: exit 0, sanitizer reports 0.
  • +3 lines with unsafe, +0 cfg arms. bun run rust:check-all: 0 errors across 12 targets. The tests ran on linux x64 only.

Suites, debug build:

  • FormData.test.ts: 182 pass. On main 149 pass and the 33 new cases fail. A release build skips the Worker case, which needs a debug or ASAN build.
  • The same file with BUN_JSC_validateExceptionChecks=1 and LeakSanitizer: 182 pass.
  • body.test.ts: 774 pass. Two "string body consumption does not leak" json cases time out at 5 s, on main too. The release build passes all 776.
  • body-clone 100, body-mixin-errors 23, client-fetch 34, image 103, s3 307, body-stream 9086.
  • streams.test.js: 604 to 624 of 624. The Bun.spawn({ stdin }) cases of "direct stream contract" time out at 5 s in some runs, on main too (616 to 623 there).

Open PRs on the same lines. #44999 and #44969 change get_form_data above the gate, and #44999 adds tests to the same file. #40221 changes ReadBytesHandler::on_read_bytes to take self: Box<Self>. The second PR to land needs a small rebase.


[human-review] gate passed · iteration 0 · 2 files touched

fails on main (without fix)
ASAN without fix: 33 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/web/html/FormData.test.ts
bun test v1.4.4 (367d939d9)

test/js/web/html/FormData.test.ts:
(pass) FormData > should be able to append a string [3.68ms]
(pass) FormData > should be able to append a Blob [14.45ms]
(pass) FormData > should be able to set a Blob [6.32ms]
(pass) FormData > should be able to set a string [3.08ms]
(pass) FormData > should get filename from file [3.73ms]
(pass) FormData > should use the correct filenames [4.46ms]
(pass) FormData > should parse multipart/form-data (simple) with Response [13.56ms]
(pass) FormData > should roundtrip multipart/form-data (simple) with Response [33.10ms]
(pass) FormData > should parse multipart/form-data (simple) with Request [4.15ms]
(pass) FormData > should roundtrip multipart/form-data (simple) with Request [9.85ms]
(pass) FormData > should parse multipart/form-data (simple with trailing CRLF) with Response [1.46ms]
(pass) FormData > should roundtrip multipart/form-data (simple with trailing CRLF) with Response [2.55ms]
(pass) FormData > should parse multipart/form-data (si
... (truncated)

release without fix: 32 failed, 1 skipped
bun test v1.4.3-canary.1 (367d939d9)

test/js/web/html/FormData.test.ts:
(pass) FormData > should be able to append a string [0.13ms]
(pass) FormData > should be able to append a Blob [0.46ms]
(pass) FormData > should be able to set a Blob [0.12ms]
(pass) FormData > should be able to set a string [0.07ms]
(pass) FormData > should get filename from file [4.31ms]
(pass) FormData > should use the correct filenames [0.13ms]
(pass) FormData > should parse multipart/form-data (simple) with Response [5.96ms]
(pass) FormData > should roundtrip multipart/form-data (simple) with Response [13.72ms]
(pass) FormData > should parse multipart/form-data (simple) with Request [0.13ms]
(pass) FormData > should roundtrip multipart/form-data (simple) with Request [0.06ms]
(pass) FormData > should parse multipart/form-data (simple with trailing CRLF) with Response [0.01ms]
(pass) FormData > should roundtrip multipart/form-data (simple with trailing CRLF) with Response [0.02ms]
(pass) FormData > should parse multipart/form-data (simple with trailing CRLF) with Request
(pass) FormData > should roundtrip multipart/form-data (simple with trailing CRLF) with Request [0.01ms]
(pass) FormData 
... (truncated)
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/web/html/FormData.test.ts
bun test v1.4.4 (367d939d9)

test/js/web/html/FormData.test.ts:
(pass) FormData > should be able to append a string [22.97ms]
(pass) FormData > should be able to append a Blob [12.72ms]
(pass) FormData > should be able to set a Blob [6.27ms]
(pass) FormData > should be able to set a string [2.51ms]
(pass) FormData > should get filename from file [5.39ms]
(pass) FormData > should use the correct filenames [5.00ms]
(pass) FormData > should parse multipart/form-data (simple) with Response [15.97ms]
(pass) FormData > should roundtrip multipart/form-data (simple) with Response [33.06ms]
(pass) FormData > should parse multipart/form-data (simple) with Request [4.35ms]
(pass) FormData > should roundtrip multipart/form-data (simple) with Request [4.48ms]
(pass) FormData > should parse multipart/form-data (simple with trailing CRLF) with Response [1.46ms]
(pass) FormData > should roundtrip multipart/form-data (simple with trailing CRLF) with Response [2.28ms]
(pass) FormData > should parse multipart/form-data (s
... (truncated)

release with fix: 1 skipped
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped)
  target       linux-x64-gnu
  build type   Release
  build dir    ./build/release
  revision     f260d7b834
  features     lto, baseline

23 deps, 146 codegen, 1179 objects in 5646ms

ninja: Entering directory `/workspace/bun/build/release'
[1/4] fetch lolhtml
[lolhtml] up to date
[2/4] fetch rust-argon2
[rust-argon2] up to date
[2/4] cargo plan → /workspace/bun/build/release/rust-target/plan.json
264 units: 192 lib, 16 proc-macro (host), 19 custom-build (host), 15 run custom-build, 17 lib (host), 4 run custom-build (host), 1 rlib
[3/4] reconfigure
[1/1531] mkdir stamps
[2/1531] mkdir codegen
[3/1531] gen bake.{client,server,error}.js
-> bake.client.js, bake.server.js, bake.error.js
[4/1531] install /workspace/bun
bun install v1.4.3-canary.1 (367d939d9)

Checked 22 installs across 44 packages (no changes) [1032.00ms]
[5/1531] install /workspace/bun/packages/bun-error
bun install v1.4.3-canary.1 (367d939d9)

Checked 1 install across 2 packages (no changes) [46.00ms]
[6/1531] install /workspace/bun/src/node-fallbacks
bun install v1.4.3-canary.1 (367d939d9)

Checked 111 installs acr
... (truncated)
diff hotspot
src/runtime/webcore/Body.rs       |  89 ++++++++-
 test/js/web/html/FormData.test.ts | 401 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 483 insertions(+), 7 deletions(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                               reads  edits  tests
src/runtime/webcore/Body.rs            3      4     50
test/js/web/html/FormData.test.ts      4      1     49