node:net: complete a write that waited for the socket in the async context of the write

RO
17d ago
+1934-25

Problem

  • A socket.write(chunk, cb) that waits for the socket runs its callback, 'drain' and 'finish' with the AsyncLocalStorage store of connect() or listen(). Node 26.3 uses the write's store. On a keep-alive connection that is another request's.
  • The write slot in src/js/node/net.ts (kwriteCallback) kept the bare callback. Socket events complete it, in the context that opened the socket (src/runtime/socket/Handlers.rs:173).

Fix

  • _write binds the waiting callback to its own async context (waitedWrite). What is queued behind that write completes there too, as in Node.
  • The same rule covers TLS end() before the handshake, 'upgrade' and CONNECT sockets, and ServerResponse.
  • 'close' stays where the socket's events run.
  • Verified: nine fixtures under Node and Bun (66 rows fail on main), net-syscall-fault.test.ts, module-graph-net-write.test.ts. Self-reviewed: 36 concerns raised, 33 addressed.

Background

  • The write slot holds the callback of a write that the kernel took part of. Socket events call it.
  • runInSameGraph never installs a context of another Bun.ModuleGraph than the socket's. Those cases behave as on main.
  • Considered a { callback, frame } record: every reader of the slot changes. A native write request adds a branch per drain on every socket.

Downsides

  • A waiting write costs one bound function (96 B). A socket whose write completed in another context keeps 160 B. Binary: +4.6 KB.
  • A reset or a bad TLS record that fails a waiting write reports 'error' in the write's store. Main used the socket's.
  • ClientRequest 'drain' runs in the write's store. 1.4.0 to 1.4.3 used the request's.
Notes

The rule. A write that waits for its handle completes in the async context in which _write handed it to the handle. The stream issues a queued write from the completion of the write before it, so a queued or corked write, an end() callback and 'finish' complete in the context of the write they are behind. A write issued while the socket connects is issued from 'connect'. This is what Node's write request does (lib/internal/stream_base_commons.js, createWriteWrap and onWriteComplete). All of it was measured against Node v26.3.0. The fixtures also pass on Node 24.22.0 and 26.12.0.

Rows against Node 26.3 (Linux x64, debug build, main's builtins against this branch's).

ProbesRowsDiffer on mainDiffer on this branch
node:net and node:tls sockets (12 scripts)147789
ServerResponse, 12 write and end orders, listen()36290
ServerResponse, same orders, emit("connection")36340
ClientRequest on a reused keep-alive connection36230
Teardown with a write waiting, 8 orders, client reconnects from 'close'7540
The same over TLS, 4 orders3220

No row of these that equals Node on main differs on this branch. The 9 rows that remain are the same on main: 'close' after an explicit destroy() runs where destroy() was called (4 rows, Node: where the socket was opened), the events of an accepted socket run in the store of listen() (2), a TLS handshake that fails under a write reports 'error' before the write callback and where the socket was opened (2), and 'close' after a small end() behind the peer's FIN (1).

Two cases outside these probes move off main's value. Both are the 'error' of a waiting write that fails for a reason the socket read, and both follow from the write callback being what destroys the socket:

  • A peer's reset with no 'error' listener: the uncaught exception carries the store of the write. Node gives the store of the socket when the socket was reading, and the store of the write when it was not.
  • A TLS record that cannot be decrypted: 'error' runs in the store of the write. Node reports it where the socket's events run and does not fail the write.

What changes, by file.

  • net.ts: waitedWrite at the three places where _write leaves the callback in the slot. The slot still holds a function, and every reader of the slot calls it as before, in the same order. A write that a TLS wrap holds until its handle is attached goes to the handle in the context of the write. When a write that waited completes in another context than the socket's events, the socket keeps the context of its events (kEventFrame), and a private AsyncLocalStorage holds that record while the completion runs. A destroy() of that socket that the completion leads to (a failed write, an end() behind it) schedules 'close' in the context of the events (scheduleClose). Another socket that the callback destroys, and a destroy() that comes after the write, emit 'close' where destroy() was called, as on main. The socket keeps no context of a write. SocketHandlers.error returns after a failed write destroyed the socket, so an adopted fd emits one 'error' (the repro of #43038).
  • tls.ts: a _final that waits for the handshake runs in the context of end().
  • _http_server.ts: the tunnel socket of 'upgrade' and CONNECT binds its waiting callback. A ServerResponse keeps the context of the first write that reported backpressure (kWaitingWrite). 'drain', the write callbacks, the 'finish' of an end() behind it, and their failure at close run there. Node corks a response for the rest of the tick, so an end() in the tick of that write takes its place. On the 36 rows of the default path, main had 7 right and this branch has 36.
  • _http_client.ts, http1_server_fallback.ts: the re-entry of a saved context for 'drain' is removed, because the socket's 'drain' now carries the write. It stays for a socket of another Bun.ModuleGraph. The fallback fails the callbacks that wait, at close, through the same kWaitingWrite rule as the native path.

Bun.ModuleGraph. A context of another graph than the socket's is never installed by this change. The stream's own work (the next write, the shutdown, 'close') would go with a graph that only wrote, and a disposed graph drops it: a host socket then never closes, and a host http.Agent never serves its next request. So those cases run exactly as on main. Where main entered the request's or the response's context for 'drain', it still does when the socket is another graph's. This rule (runInSameGraph, isInAnotherGraph in internal/async_context_frame.ts) is new and needs a yes from the owner of Bun.ModuleGraph.

Measurements.

  • A write that completes in the call: the text of _write up to the park is the same, apart from the branch for a send that the kernel rejects and the branch for a write held by a TLS wrap. 80,000 echo writes park 0 times.
  • Per socket that never parks, or whose writes complete in the context of its events: no new property. Out-of-line slots are 16/16 (client), 17/32 (accepted), 46/64 (TLS client) on both builds.
  • Per socket whose write completed in another context: one record of 32 B and one property (a client socket goes from 16/16 to 17/32 out-of-line slots, +128 B). The record holds the context of the socket's events, not of the write.
  • Per parked write: +1 bound function while it waits (heap +96 B to +100 B per parked write over 1,000 parked writes) and nothing after it completes. Main: +0.
  • Per completion in another context than the event's: one AsyncLocalStorage.run() and 2 host calls of the graph check. 0 host calls without AsyncLocalStorage or when the write and the socket have one context. internal/async_context_frame is not loaded by a program that uses node:net without AsyncLocalStorage.
  • Per socket close: one read of a module variable, which is unset until a write has completed in another context. After that, one getStore().
  • How often a write parks: 16 of 1,024 writes (64 MiB in 64 KiB chunks), 0 of 600 after connect, the same on both builds.
  • Socket calls (LD_PRELOAD counter): 4,000 send for 2,000 echo round trips and 400 send for 200 parked writes, on both builds.
  • Bundled builtins (release): six modules, +4,574 B, measured before main removed the http2 allowHTTP1 proxy (a seventh module, +162 B more then). Release binaries (--lto=off) of the first revision: text 101,811,313 B to 101,815,686 B (+4,373 B) for +4,357 B of bundle, file 102,463,288 B to 102,467,384 B.
  • Wall clock, release builds of the first revision, 24 interleaved runs each: 200,000 echo round trips, median 1,658.6 ms on main and 1,669.8 ms here (+0.7%, minimum 1,487.0 and 1,464.1 ms). 256 MiB in 64 KiB chunks, median 263.4 ms and 259.3 ms. Both are inside the run-to-run spread.
  • perf, valgrind and strace are not installed in the build container, so there is no instruction count.

Tests.

  • Fixtures in test/js/node/async_hooks/async-context/, which AsyncLocalStorage-tracking.test.ts runs under Node and Bun on every lane: net-socket-write-waits, tls-socket-write, tls-wrap-socket-write (STARTTLS), http-upgrade-socket-write, http-connect-socket-write, http-clientrequest-write, http-serverresponse-write (both listen() and emit("connection")), http2-allowhttp1-socket-write. Each checks that its write waited, and fails by name if its socket reports an error or closes early. net-socket-write is as on main.
  • net-syscall-fault.test.ts (debug and ASAN builds): the cases that need an injected short or failed send, and the Bun-only rows of 'close' (another socket, a later destroy(), a peer's reset on an accepted socket).
  • test/js/bun/module-graph/module-graph-net-write.test.ts: the Bun.ModuleGraph cases.
  • 23 mutations, each of which reverts one clause of the change in the bundled builtins: the fixtures and tests catch all 23 (run before the last two rebases).
  • Also ran, with no failure that main's builtins do not show: node-http-backpressure, node-net, node-net-server, node-tls-server, node-tls-connect, tls-syscall-fault, node-http, node-https-server, node-http-connect, node-http-upgrade-body, node-http-with-ws, node-http-server-close-drain, node-http-server-socket-end-drain, node-http-agent-free-socket, node-http-syscall-fault, node-http2-upgrade, module-graph-io, module-graph-callbacks, AsyncLocalStorage, async_hooks.node, and 25 of Node's test-net-*, test-tls-* and test-http-* tests.
  • Windows takes a write of any size at once, so no write waits for its socket there. The two fixtures that only test such writes (net-socket-write-waits after its first scenario, http-serverresponse-write) do nothing on Windows. macOS was not run locally.

Self-review, not changed.

  • A TLS handshake that fails under a waiting write: Node fails the write first, so 'error' runs in the context of the write. Here 'error' still comes first and runs where the socket was opened, as on main. The write callback now runs in the context of the write, and the TLS fixture pins that row. The order is a separate defect.
  • The TLS cases that need an injected fault stay in net-syscall-fault.test.ts, next to the slot they test. The cases that need no fault moved to the fixtures.
  • A callback that throws: the exception of a parked write's callback still leaves through the socket's error handler, as on main. Node reports it as an uncaught exception and keeps the socket open.

Still different from Node, and not part of this change.

  • node:http2 stream writes: the write callback and 'drain' see no store.
  • child.stdin (an fs.WriteStream, no write slot), a dgram send that the kernel refused, session.settings(cb), and an IPC send(cb) that did not fit the pipe.
  • Listeners on req.socket of a reused http.Agent connection run in the store of the first request.
  • request.socket.end() with a response write waiting, on the default server path: the connection never closes (Bun 1.4.3 too). The fixture runs that scenario for emit("connection") only.
  • The tunnel socket of 'upgrade' and CONNECT emits a second 'drain' from its native drain event, as on main. The first one, from the stream, now runs in the context of the write. The draft #46014 removes the second one.
  • A response whose first write of a tick is taken at once and whose next write in that tick waits: Node uses the context of the first write (it corks both), this branch and main use the context of the write that waited.
  • A response write that takes the backlog of an earlier write with it, before the drain event: the callback of the earlier write runs in the context of the later write, as on main. Node runs it in the context of the earlier write.

Open PRs that touch the same lines. #43250, #43392, #43026 and #43877 read or write the write slot: the slot still holds a function, so their calls stay correct. #43038 is covered by the one-line guard in SocketHandlers.error. The draft #46014 rewrites #onDrain of the tunnel socket, which this change leaves as on main.


no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/net/net-syscall-fault.test.ts, test/js/bun/module-graph/module-graph-net-write.test.ts