Bun.serve: do not re-derive a Content-Type the handler deleted from the Response

RO
2mo ago
+327-30

Problem

  • Bun.serve sends a Content-Type the handler deleted. const r = new Response(new URLSearchParams("q=1")); r.headers.delete("content-type"); return r; has no Content-Type in-process, but the client gets application/x-www-form-urlencoded;charset=UTF-8. Same for FormData, a typed Blob, Bun.file(), a deleted init Content-Type, and routes.
  • The constructor and the headers getter copy the body's type into the header list (Response.rs). The send paths (get_content_type in RequestContext.rs, StaticRoute::from_js, FileRoute::from_js) and formData() then derive it from the body again when the list has none.

Fix

  • Response Init gets headers_own_content_type: the header list has held a Content-Type (from the init, copied from the body, or Response.json()). clone() copies it. new Response(body, otherResponse) recomputes it.
  • Every Content-Type reader asks the Response. With the flag set and no Content-Type in the list, the send paths send none and formData() rejects, as in undici and Deno. The header list is then the single source.
  • A Response whose headers were never materialized keeps the allocation-free send-time derivation (new Response("str") is unchanged). String and untyped bodies keep their defaults.
  • Verified: test/js/bun/http/bun-serve-headers.test.ts (36 new raw-socket cases, 15 fail on 1.4.3), test/js/web/html/FormData.test.ts (1 new, fails on 1.4.3). Other suites in Notes.

Background

  • Per Fetch, new Response(body) appends the body's type to the header list at construction, and undici and Deno serialize that list.
  • Bun defers it: with no headers init it allocates no FetchHeaders and derives the Content-Type at send time. The first response.headers access materializes the list and copies the type in.
  • routes: { "/x": response } snapshots the headers in StaticRoute::from_js / FileRoute::from_js with the same fallback.
Notes
  • Why a flag and not "a header list exists, so never derive": new Response(Bun.file(p).stream(), { headers: {...} }) has a header list from the start, but its body is a stream with no type. At send time to_blob_if_possible() turns the lazy file stream back into a file Blob whose type comes from the extension, and the wire gets text/javascript etc. today. A stateless rule drops that. The flag records whether the Content-Type decision actually reached the header list. The stateless rule becomes possible if a maintainer decides that a string body should put text/plain;charset=utf-8 into the header list the way undici does (the question #8530 asks). That is a separate, visible change (new Response("x").headers.get("content-type") is null today), so it is not made here. Question for a maintainer: is that materialization wanted? If yes, this flag can go.
  • render_metadata still computes the body's MimeType when the flag suppresses the header, because the Content-Disposition filename rule (autoset_filename) keys on its category. Only the content-type write is skipped.
  • The send-time fallback order in get_content_type is unchanged: header list, else blob.content_type(), else image sniff, else text/plain for strings, else application/octet-stream. The flag only stops it after step one.
  • Self-reviewed: 5 concerns raised, 2 addressed in code (formData()/compileStreaming read the same rule, the send paths ask the Response instead of taking a bare flag), 3 answered here:
    • String bodies: new Response("str", { headers: { "content-type": x } }) plus delete now sends none, but new Response("str") plus headers.set plus headers.delete still sends text/plain, because Bun never records a string body's default type in the header list (see the first note). Only a Content-Type that was in the header list is tracked.
    • new Response(r.body, r) after deleting from r gets the type again when r's body was an in-memory typed Blob: Bun turns a blob-backed stream body back into the typed Blob at construction (Body.rs, kept for #14988), so the new Response pairs a typed body with a header list that has no Content-Type, and copies it in. In-process and wire agree there. Left as is.
    • #32913 and #42015 (both open) touch the same functions for the other face of this gap (Content-Type lost when the body is read before the headers) and add their own Init field. They do not fix the delete case: after the header list is materialized they fall back to the body's type at send time like main does. Whichever lands second rebases, the fields compose. #30997 rewrites the detached-body condition 50 lines below the call changed here, no textual overlap.
  • Unchanged on purpose: untyped Blob/ArrayBuffer bodies keep the image sniff and application/octet-stream fallback. A fetch()-created Response whose own headers are edited is not flagged (new Response(res.body, res) is, it goes through the constructor). The client side (fetch(request) after request.headers.delete("content-type")) is a separate path in fetch.rs.
  • HEAD through the fetch handler already sends no Content-Type for these bodies (it renders metadata from an empty blob, #41585 and #33427 are about that). The test pins GET dynamic, GET static and HEAD static.
  • Cost: one bool in Init, one extra fast_has(ContentType) (a scan of the common-header vector) in new Response(body, { headers }). No new FetchHeaders allocation anywhere.
  • Suites run with the debug build: serve.test.ts (2 pre-existing environment failures, same on stock bun: root port, loopback dev mode), bun-serve-static, bun-serve-file, bun-serve-routes, bun-serve-html, bun-server, web/fetch/body, body-clone, response, wasm-streaming, fetch.test.ts (environment-only failures: ipv6, public internet, root permissions, ASAN timeouts), web/html/FormData.

[human-review] gate passed · iteration 2 · 6 files touched

fails on main (without fix)
ASAN without fix: 17 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/bun/http/bun-serve-headers.test.ts test/js/web/html/FormData.test.ts
bun test v1.4.3 (f42e98025)

test/js/bun/http/bun-serve-headers.test.ts:
(pass) weird headers [505.72ms]
(pass) response header values are isomorphic-encoded on the wire > Response headers [613.20ms]
(pass) response header values are isomorphic-encoded on the wire > Set-Cookie headers [103.94ms]
(pass) response header values are isomorphic-encoded on the wire > long values and values that start with a non-ASCII char [251.93ms]
(pass) response Connection: close closes the socket > string body [83.66ms]
(pass) response Connection: close closes the socket > case-insensitive value [49.89ms]
(pass) response Connection: close closes the socket > token list [54.97ms]
(pass) response Connection: close closes the socket > streaming body [65.72ms]
(pass) response Connection: close closes the socket > keep-alive still the default [76.55ms]
479 |       expected["GET static"] = c.wire;
480 |       expected["HEAD static"] = c.wire;
481 |       actual["GET static"] = await co
... (truncated)

release without fix: all passed
bun test v1.4.3-canary.1 (8de054b85)

test/js/bun/http/bun-serve-headers.test.ts:
(pass) weird headers [9.15ms]
(pass) response header values are isomorphic-encoded on the wire > Response headers [8.68ms]
(pass) response header values are isomorphic-encoded on the wire > Set-Cookie headers [3.25ms]
(pass) response header values are isomorphic-encoded on the wire > long values and values that start with a non-ASCII char [7.44ms]
(pass) response Connection: close closes the socket > string body [3.02ms]
(pass) response Connection: close closes the socket > case-insensitive value [1.89ms]
(pass) response Connection: close closes the socket > token list [1.99ms]
(pass) response Connection: close closes the socket > streaming body [1.81ms]
(pass) response Connection: close closes the socket > keep-alive still the default [2.25ms]
(pass) Bun.serve does not re-derive a Content-Type the handler deleted > URLSearchParams: body type deleted [3.95ms]
(pass) Bun.serve does not re-derive a Content-Type the handler deleted > URLSearchParams: init content-type deleted [1.57ms]
(pass) Bun.serve does not re-derive a Content-Type the handler deleted > URLSearchParams: content-type se
... (truncated)
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/bun/http/bun-serve-headers.test.ts test/js/web/html/FormData.test.ts
bun test v1.4.3 (f42e98025)

test/js/bun/http/bun-serve-headers.test.ts:
(pass) weird headers [452.66ms]
(pass) response header values are isomorphic-encoded on the wire > Response headers [501.47ms]
(pass) response header values are isomorphic-encoded on the wire > Set-Cookie headers [82.44ms]
(pass) response header values are isomorphic-encoded on the wire > long values and values that start with a non-ASCII char [286.05ms]
(pass) response Connection: close closes the socket > string body [86.73ms]
(pass) response Connection: close closes the socket > case-insensitive value [52.91ms]
(pass) response Connection: close closes the socket > token list [41.80ms]
(pass) response Connection: close closes the socket > streaming body [56.40ms]
(pass) response Connection: close closes the socket > keep-alive still the default [45.69ms]
(pass) Bun.serve does not re-derive a Content-Type the handler deleted > URLSearchParams: body type deleted [104.85ms]
(pass) Bun.serve
... (truncated)

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 823ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/5] gen generated_host_exports.rs
generated_host_exports.rs: 122 exports (host=5, lazy=10, generic=107, rust=0); 242 extern-C blocks audited
[1/5] cargo bun_runtime → libbun_runtime.a
   Compiling bun_runtime v0.0.0 (/workspace/bun/src/runtime)
    Finished `release` profile [optimized + debuginfo] target(s) in 5m 41s
[2/5] link bun-profile
[4/5] strip bun
[4/5] bun-profile --revision
1.4.3-canary.1+c31617e1e
[build] done
bun test v1.4.3-canary.1 (c31617e1e)

test/js/bun/http/bun-serve-headers.test.ts:
(pass) weird headers [10.90ms]
(pass) response header values are isomorphic-encoded on the wire > Response headers [8.53ms]
(pass) response header values are isomorphic-encoded on the wire > Set-Cookie headers [3.34ms]
(pass) response header values are isomorphic-encoded on the wire > long values and values that start with a non-ASCII char [8.16ms]
(pass) response Connection: close closes the socket > string body [2.68ms]
(pass) response Connection: close closes the socket > cas
... (truncated)
diff hotspot
src/runtime/server/FileRoute.rs            |   5 +-
 src/runtime/server/RequestContext.rs       |  10 +-
 src/runtime/server/StaticRoute.rs          |  10 +-
 src/runtime/webcore/Response.rs            |  74 +++++++---
 test/js/bun/http/bun-serve-headers.test.ts | 221 ++++++++++++++++++++++++++++-
 test/js/web/html/FormData.test.ts          |  37 +++++
 6 files changed, 327 insertions(+), 30 deletions(-)

gate history · 2 passed · 0 rejected · iteration 2

evidence per changed file
file                                        reads  edits  tests
src/runtime/server/FileRoute.rs                 3      2     16
src/runtime/server/RequestContext.rs            5      5     16
src/runtime/server/StaticRoute.rs               3      3     16
src/runtime/webcore/Response.rs                 9     14     16
test/js/bun/http/bun-serve-headers.test.ts      3      2     12
test/js/web/html/FormData.test.ts               2      2      9