node:http: make res._send() the stored head plus _writeRaw(), as in Node.js

RO
25d ago
+888-68

Problem

  • res._send(data) on a response queued behind a pipelined one loses data and the head of writeHead(). bun 1.4.2 threw an uncaught ERR_HTTP_HEADERS_SENT.
  • ServerResponse.prototype._send (src/js/node/_http_server.ts:3886) writes to the handle at once. Only write(), end() and _writeRaw() know the queue.
  • http.OutgoingMessage.prototype.end.call(res, chunk) never ends a response.

Fix

  • _send() is Node's: the stored head, then _writeRaw(), which keeps bytes for a queued response. The data goes out unframed.
  • OutgoingMessage.prototype.write() and end() call the ServerResponse method for a response with a handle.
  • res.flushHeaders() on a queued response keeps the head through _send("").
  • Verified: test/js/node/http/node-http.test.ts (199 new tests, 174 fail on main), 10 more node-http suites. Self-reviewed: 45 concerns raised, 30 survived, 26 addressed, 4 rejected (Notes).

Background

  • A response behind an open one on its connection is queued (pipelining): its calls are kept and run at its turn.
  • The handle writes the response: the native NodeHTTPResponse, or a JavaScript object for a duplex given to 'connection'. The native one batches small writes (the cork).
  • Considered a send op in the queue with main's framed write: two writers and two framings stay.

Downsides

  • res._send(data) on a chunked response sends data unframed, as on Node.js. A caller that relied on main's frame breaks the body.
  • A 1xx block is a send() of its own (2 sends, was 1). write() and end() of a ClientRequest run 4 more bytecode instructions. .text grows by 1,792 bytes, the builtins by 2,693.
  • After end(), a raw write sends nothing and fails its callback. Node.js sends it behind the message.
Notes

Reproduction. Two requests on one connection. The response to the first one stays open, so the second one is queued.

const http = require("node:http");
const net = require("node:net");
let first;
const server = http.createServer((req, res) => {
  res.sendDate = false;
  if (req.url === "/first") return void (first = res);
  res.statusCode = 404;
  res.setHeader("X-Custom", "yes");
  res._send("RAW");
  res.end("b");
  first.end("1");
});
server.listen(0, "127.0.0.1", () => {
  const client = net.connect(server.address().port, "127.0.0.1");
  let wire = "";
  client.on("data", chunk => (wire += chunk));
  client.on("close", () => (console.log(JSON.stringify(wire)), server.close()));
  client.write("GET /first HTTP/1.1\r\nHost: x\r\n\r\nGET /second HTTP/1.1\r\nHost: x\r\nConnection: close\r\n\r\n");
});

Behind the response to /first, the client receives:

second response
Node v26.3.0RAW, then HTTP/1.1 404 Not Found, X-Custom: yes, Connection: close, Content-Length: 1, b
mainHTTP/1.1 200 OK, Date, Content-Length: 1, b. The status, the header and RAW are lost
bun 1.4.2uncaught ERR_HTTP_HEADERS_SENT (Stream already started) from the end() of the first response: the second one wrote out of turn
this PRas Node

What a raw write is now. One helper, writeRawToHandle(), writes for _writeRaw() and for the replay of a kept op. It serves _send(), writeProcessing(), writeEarlyHints() and writeInformation().

  • The native handle closes an open header block, sends the held tail of an earlier large write(), writes the bytes, and sends what the cork buffer holds. It returns the count of bytes that the socket did not take. One native call for each raw write, as on main.
  • Count 0: the callback runs on the next tick, behind the callbacks of earlier writes. Count above 0: the callback waits for the drain, and the call returns false.
  • A socket that is gone: the call returns false and the callback never runs, as in Node's _writeRaw(). Main returned true and ran the callback.
  • After end(): nothing is written. The callback runs behind 'finish', with ERR_STREAM_WRITE_AFTER_END when the write had bytes. The response emits no 'error'.
  • The JavaScript handle gives the callback to its stream, and returns what socket.write() returns.
  • A string in an encoding other than utf8, latin1 or ascii becomes a Buffer first. The native table of names has no utf-16le.
  • _send(data, cb) calls cb. Main never called it on the native handle.

The cork. uWS copies a write of up to 16 KiB into a cork buffer while a socket is corked. The request dispatch corks the socket, and so does the native writeHead(). The count of bytes that wait does not see that buffer, and a close drops it unless a body write happened. So a raw write now ends with AsyncSocket::sendCorked(). Without it, res.writeHead(); res._send(data); res.destroy() in a later turn sent nothing (main sent all of it), and a callback ran before its bytes left the process. #44468 adds the same call for the same reason, for 1xx blocks.

Other changes that the raw _send() needs.

  • write() on a response with no body (HEAD, 204, 304) no longer runs the callbacks that wait for a drain. Only a raw write can park one there.
  • With rejectNonStandardBodyWrites, a write() or end(chunk) that throws ERR_HTTP_BODY_NOT_ALLOWED stores the implicit head first, as Node's write_() does. headersSent is true behind the throw, and a later _send() puts the head first.
  • OutgoingMessage.prototype.write() and end() go to ServerResponse only for an object that the ServerResponse constructor made (kPendingCallbacks). The JavaScript handle is also given to a response class that extends OutgoingMessage directly. They keep the chunk checks of write_().
  • OutgoingMessage.prototype.flushHeaders() needs no change: Node's port calls _send(""), which now works.

Tests. 199 tests in one describe: 4 transports (TCP, TLS, a duplex given to 'connection', http2 allowHTTP1) times a queued and a current response.

  • main (canary 47ba1c8d85): 174 fail, 52 of them by timeout. The 25 that pass guard behaviour that main has: the chunk checks through OutgoingMessage.prototype (8), end(chunk) after end() through the prototype (8), a response class that extends OutgoingMessage (1), writeHead(); _send(); destroy() in a later turn (2), a pipe() behind _send() (2), and 4 cases on the JavaScript handle.
  • Node v26.3.0: 195 of the 199 test bodies pass. The other 4 are the unknown encoding on a queued response: Node throws at the turn of the response, where nothing can catch it. One test has a second expected result for Node: the raw write after end().
  • Debug build: all of node-http.test.ts (491 pass, 1 skip), node-http-displaced-response, -backpressure, -transfer-encoding, -pinned-write, -nested-cork, -server-socket-end-drain, -server-abort-events, -with-ws, -req-complete, early-hints-crlf-injection (411 pass). Vendored test-http-* and test-https-*: 450 of 456 pass. The other 6 are client tests with short timers. They also failed on a debug build of main (9ed8d11b96).
  • A local debug build needs about 4 s to start a child process that loads node:http. The tests in these files that spawn one need --timeout above the default 5 s locally, on main too.

Measurements. Bytecode from BUN_JSC_dumpGeneratedBytecodes=1 on debug builds of main and of this PR. Sends from an LD_PRELOAD shim that logs send, write and writev on the server socket. Sizes from release builds.

functionmainthis PR
OutgoingMessage.prototype.write2261 (4 run for a ClientRequest: resolve_scope, get_from_scope, get_by_val, jfalse)
OutgoingMessage.prototype.end188226 (the same 4)
ServerResponse.prototype.write248250
ServerResponse.prototype.end288289
onNodeHTTPRequest, ServerResponse constructor749, 98749, 98
ServerResponse.prototype._send54 and a closure of 5854
ServerResponse.prototype._writeRaw68139
ServerResponse.prototype.flushHeaders8791
writeRawToHandle48
sends of the serverNodemainthis PR
end("x")111
write("a"); end("b")111
writeEarlyHints(); end("x")212
writeProcessing(); end("x")212
writeHead(); _send("RAW"); end("b")212
queued: writeEarlyHints(); end("x") (with the first response)233
queued: writeHead(); _send("RAW"); end("b") (with the first response)212, data lost4

.text: 65,658,709 to 65,660,501 (+1,792 bytes), from release builds of e655c58032 and of this PR on one machine. Builtins: +2,693 bytes, from release bundles of src/js on 47ba1c8d85 (_http_server.js +1,663, _http_outgoing.js +772, internal/http.js +53, http1_server_fallback.js +205).

Self-review. Two passes, the second one on the reworked diff. 30 distinct concerns survived a skeptic, and 26 are addressed. The 4 that are not:

  • OutgoingMessage.prototype._send.call(res) and ._writeRaw.call(res) still run Node's ports for a response with a handle. A test of this[kHandle] there is on the write path of every ClientRequest, and no caller is known.
  • _writeRaw() checks the encoding before it sees that the socket is gone. Node checks the socket first. The other order costs one more native call for each raw write, and the input is an error in both orders.
  • For a queued response, Node moves the head in front of raw bytes kept before it when the call that sends the head has a Buffer or a string in another encoding (outputData.unshift(header)). Bun keeps the call order, as main does for a 1xx block.
  • res.socket.bytesWritten does not count the bytes of _send(). It counts neither the head nor a 1xx block on main.

Other open PRs on these lines.

  • #33871 frames the body by Transfer-Encoding. It lists "res._send() writes with the framing of the body. Node writes raw bytes" as not decided. This PR decides it.
  • #44468 sends a 1xx block when it is written. It adds the same AnyResponse::send_corked() and the same call in write_informational, and it edits the same arm of _writeRaw(). Both PRs return false with no callback for a socket that is gone. Whichever lands second drops its copy.
  • #44338 (trailers travel with the end call) names res._send() on a queued response as not fixed there. This PR is that fix.
  • #44505 has a test that expects res.statusCode = 99; res._send() to throw from the native check. With this PR _send() writes no head there, as in Node, so that test changes.
  • #44676 edits OutgoingMessage.prototype.end. This PR adds lines at its top.
  • The branch robobun/7f2a07bf/http-queued-response-sync-throw moves the kept ops of a queued response into the native handle. The two changes fit together: _send() stays raw, and the one that lands second drops the two queued.ops pushes of _send().

Differences from Node that stay.

  • After end(), a raw write sends nothing, and its callback gets ERR_STREAM_WRITE_AFTER_END. Node sends the bytes behind the message while the response has the socket.
  • After a raw write that returned false, the response emits 'drain'. Node emits it only after a write() that returned false. writableNeedDrain is true while bytes wait, so a pipe would stall without the event.
  • An unknown encoding on a queued response throws at the call. Node throws at the turn of the response.
  • The JavaScript handle gives the head and the data of _send() to its stream as two writes. A destroy() right behind them drops the second one. Node writes a string with the head as one write. Main does the same as this PR.
  • On the JavaScript handle, the callback of a raw write can run before the callback of an earlier write() that waits for 'drain'.
  • A queued response that called writeHead() and _send() is replayed as 3 sends. Node makes 1.

Found, not fixed. All of them are on main.

  • res.write("AB", "utf-16le") throws ERR_UNKNOWN_ENCODING. The table in src/runtime/node/types.rs has utf16-le where Node has utf-16le.
  • rejectNonStandardBodyWrites has no effect on the JavaScript handle.
  • res.write(123) on a queued response returns true, and the throw comes at the replay, uncaught.
  • res.destroy() outside the request handler, with no head written, sends HTTP/1.1 200 OK and Connection: close.
  • A 'socket' listener on a queued response that writes: its bytes go out ahead of the kept ones.
  • On a duplex, 'finish' is emitted before the callback of an earlier write().

no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/http/node-http.test.ts