node:http: dispatch bit 0 and the response mark read the parser's connection verdict

Closed
RO
1mo ago
+18-12

Behaviour change: none

Problem

  • The native HTTP parser decides one time for each request if the connection closes after the response (sawConnectionClose, packages/bun-uws/src/HttpParser.h:1286). Two more places compute the same answer again.
  • src/jsc/bindings/NodeHTTP.cpp:105 scans the Connection fields a second time for the dispatch bitfield. src/js/node/_http_server.ts:789 forces HTTP/1.0 to close with a version test of its own.
  • The next PR changes the rule for HTTP/1.0. With three copies of the rule, that PR must change three places.

Fix

  • Bit 0 of the dispatch bitfield is now the parser's verdict. The JS dispatcher reads that bit and has no version test.
  • For node:http, the response mark in HttpContext.h takes HTTP_ANCIENT_REQUEST from the version and HTTP_CONNECTION_CLOSE from the verdict. The Bun.serve lines do not change.
  • Correct because the parser's rule is the same as before (HTTP/1.0, or a close item). Each reader gets the value that it computed itself before.
  • Verified: no new test. A raw-socket probe of 40 exchanges gives byte-identical output on main and on this branch. The tests that cover this code are in Notes.

Background

  • sawConnectionClose is a latch of the parser, one for each connection. After a request sets it, the parser refuses the next request on that connection.
  • The dispatch bitfield is one integer that native code gives to the node:http dispatcher. It holds the facts about the request headers that the dispatcher needs, so that JS does not build req.headers for them.
  • HTTP_ANCIENT_REQUEST tells the native writer to send no chunks to an HTTP/1.0 request. HTTP_CONNECTION_CLOSE closes the socket after the response.

Downsides

  • None found. Each HTTP/1.1 request makes 2 scans of the Connection fields, not 3. The Bun.serve code has 0 different instructions in a release build. The numbers are in Notes.
Notes

Measurements (linux x64, main 1313ca61d4 against this change)

Whatmainthis PRHow
Calls of HttpRequest::hasConnectionToken for 101 HTTP/1.1 keep-alive requests303202gdb breakpoint with an ignore count, debug build
Calls of HttpRequest::hasConnectionClose for the same requests202101same
NodeHTTPServer__onRequest_http1500 instructions, 196 conditional jumps1504, 196llvm-objdump of the release build
Request handler of HttpContext<SSL>::onData<true> (node:http)266 (TCP), 296 (TLS) instructions266, 298same
HttpContext<SSL>::onData<false>, its request handler and HttpParser::consumePostPadded<false> (Bun.serve)0 different instructionsnormalized disassembly, release build
.text of the release binary80,661,994 bytes80,661,973 bytessize
Release binary file80,827,976 bytes80,827,976 bytesls -l

Existing tests that cover this code. All pass with this change, except tests that start a second debug process and reach the 5 s limit on the loaded test host. Those tests fail on main on that host too.

  • test/js/node/http/node-http.test.ts (responses that close the connection: HTTP/1.0, Connection: close)
  • test/js/node/http/node-http-transfer-encoding.test.ts
  • test/js/node/http/node-http-req-complete.test.ts, node-http-server-timeouts.test.ts, node-http-server-close-drain.test.ts
  • test/regression/issue/34415.test.ts
  • test/js/bun/http/serve.test.ts, request-smuggling.test.ts, serve-close-delimited-framing.test.ts
  • test/js/node/test/parallel/test-http-1.0.js, test-http-1.0-keep-alive.js, test-http-should-keep-alive.js, test-http-keep-alive-close-on-header.js, test-http-keep-alive.js, test-http-keep-alive-max-requests.js, test-http-pipeline-flood.js

The probe. 40 raw-socket exchanges with an http.Server: HTTP/1.0 and HTTP/1.1, with and without Connection: keep-alive and close, compound Connection values, Proxy-Connection, responses with and without Content-Length, HEAD, 204, 304, a Connection header that the listener set or removed, res.shouldKeepAlive = false, pipelined requests. For each exchange it records the URLs that the listener saw, res.shouldKeepAlive, the bytes on the wire and whether the server closed the connection.

Why the parser's latch is the verdict of the request in dispatch. The latch is set while the parser reads the request head, before it calls the request handler. After a request sets the latch, the parser refuses every later request on the connection. Thus the value of the latch in the dispatch of a request is the verdict on that same request.