node:http: dispatch bit 0 and the response mark read the parser's connection verdict
Behaviour change: none
Problem
- The native HTTP parser decides one time for each request if the connection closes after the response (
sawConnectionClose,packages/bun-uws/src/HttpParser.h:1286). Two more places compute the same answer again. src/jsc/bindings/NodeHTTP.cpp:105scans theConnectionfields a second time for the dispatch bitfield.src/js/node/_http_server.ts:789forces HTTP/1.0 to close with a version test of its own.- The next PR changes the rule for HTTP/1.0. With three copies of the rule, that PR must change three places.
Fix
- Bit 0 of the dispatch bitfield is now the parser's verdict. The JS dispatcher reads that bit and has no version test.
- For
node:http, the response mark inHttpContext.htakesHTTP_ANCIENT_REQUESTfrom the version andHTTP_CONNECTION_CLOSEfrom the verdict. TheBun.servelines do not change. - Correct because the parser's rule is the same as before (HTTP/1.0, or a
closeitem). Each reader gets the value that it computed itself before. - Verified: no new test. A raw-socket probe of 40 exchanges gives byte-identical output on
mainand on this branch. The tests that cover this code are in Notes.
Background
sawConnectionCloseis a latch of the parser, one for each connection. After a request sets it, the parser refuses the next request on that connection.- The dispatch bitfield is one integer that native code gives to the
node:httpdispatcher. It holds the facts about the request headers that the dispatcher needs, so that JS does not buildreq.headersfor them. HTTP_ANCIENT_REQUESTtells the native writer to send no chunks to an HTTP/1.0 request.HTTP_CONNECTION_CLOSEcloses the socket after the response.
Downsides
- None found. Each HTTP/1.1 request makes 2 scans of the
Connectionfields, not 3. TheBun.servecode has 0 different instructions in a release build. The numbers are in Notes.
Notes
Measurements (linux x64, main 1313ca61d4 against this change)
| What | main | this PR | How |
|---|---|---|---|
Calls of HttpRequest::hasConnectionToken for 101 HTTP/1.1 keep-alive requests | 303 | 202 | gdb breakpoint with an ignore count, debug build |
Calls of HttpRequest::hasConnectionClose for the same requests | 202 | 101 | same |
NodeHTTPServer__onRequest_http | 1500 instructions, 196 conditional jumps | 1504, 196 | llvm-objdump of the release build |
Request handler of HttpContext<SSL>::onData<true> (node:http) | 266 (TCP), 296 (TLS) instructions | 266, 298 | same |
HttpContext<SSL>::onData<false>, its request handler and HttpParser::consumePostPadded<false> (Bun.serve) | 0 different instructions | normalized disassembly, release build | |
.text of the release binary | 80,661,994 bytes | 80,661,973 bytes | size |
| Release binary file | 80,827,976 bytes | 80,827,976 bytes | ls -l |
Existing tests that cover this code. All pass with this change, except tests that start a second debug process and reach the 5 s limit on the loaded test host. Those tests fail on main on that host too.
test/js/node/http/node-http.test.ts(responses that close the connection: HTTP/1.0,Connection: close)test/js/node/http/node-http-transfer-encoding.test.tstest/js/node/http/node-http-req-complete.test.ts,node-http-server-timeouts.test.ts,node-http-server-close-drain.test.tstest/regression/issue/34415.test.tstest/js/bun/http/serve.test.ts,request-smuggling.test.ts,serve-close-delimited-framing.test.tstest/js/node/test/parallel/test-http-1.0.js,test-http-1.0-keep-alive.js,test-http-should-keep-alive.js,test-http-keep-alive-close-on-header.js,test-http-keep-alive.js,test-http-keep-alive-max-requests.js,test-http-pipeline-flood.js
The probe. 40 raw-socket exchanges with an http.Server: HTTP/1.0 and HTTP/1.1, with and without Connection: keep-alive and close, compound Connection values, Proxy-Connection, responses with and without Content-Length, HEAD, 204, 304, a Connection header that the listener set or removed, res.shouldKeepAlive = false, pipelined requests. For each exchange it records the URLs that the listener saw, res.shouldKeepAlive, the bytes on the wire and whether the server closed the connection.
Why the parser's latch is the verdict of the request in dispatch. The latch is set while the parser reads the request head, before it calls the request handler. After a request sets the latch, the parser refuses every later request on the connection. Thus the value of the latch in the dispatch of a request is the verdict on that same request.