node:http2: emit the SETTINGS ACK before DATA the new settings unblocked

Merged
RO
2mo ago
+143-4

Problem

  • When a peer raises SETTINGS_INITIAL_WINDOW_SIZE while a stream is stalled on flow control, the node:http2 client flushes the unblocked DATA frames before it sends the SETTINGS ACK. nghttp2 and grpc-go enforce the old receive window until they process the ACK, so they reset the stream with NGHTTP2_FLOW_CONTROL_ERROR. This breaks gRPC clients against Go servers (grpc-go raises the window under load). Fixes #40358.
  • The cause is handle_settings in src/runtime/api/bun/h2/connection.rs:762. It ran sink.on_remote_settings before send_settings_ack. The sink callback applies the new window and calls flush(), which drains the queued DATA into the same outbound buffer, ahead of the ACK.

Fix

  • Reorder handle_settings: queue the SETTINGS ACK (and run the outbound ACK flood check) first, then run the on_remote_settings callback. Both paths write through the same corked buffer, so call order is wire order.
  • Gate the flush() in on_remote_settings on a stream window that actually grew. With the ACK queued first, an unconditional flush pushed it through a detached transport during the post-close frame drain. The EBADF surfaced as the session error before the pending GOAWAY, which broke test/js/node/test/parallel/test-http2-too-many-settings.js.
  • RFC 9113 6.5.3 requires the ACK immediately after the SETTINGS values are processed. Node, nghttp2, and grpc-go all emit the ACK before any DATA that uses the new window.
  • Verified: test/js/node/http2/node-http2-settings-ack-ordering.test.ts. On the unfixed build the client sends 196609 bytes of DATA before the ACK, with the fix it sends 0. Also ran all of test/js/node/http2/ (479 pass), test-http2-too-many-settings.js (20 runs), fetch-http2-client, fetch-http2-adversarial, and test/js/third_party/grpc-js/.

Background

  • HTTP/2 flow control gives each stream a send window, initialized to the peer's SETTINGS_INITIAL_WINDOW_SIZE. A sender that exhausts the window queues further DATA until the window grows.
  • A SETTINGS frame that changes SETTINGS_INITIAL_WINDOW_SIZE adjusts every open stream's send window by the delta (RFC 9113 6.9.2). Bun applies that delta correctly since #30342. This PR only fixes the frame ordering.
  • Frames on one connection are ordered, so the peer sees everything written before the ACK measured against its old window.
Notes
  • Repro: the issue's self-contained script (raw TCP HTTP/2 server, no dependencies). On bun 1.4.0 the client sent 458753 bytes in 29 DATA frames between the SETTINGS and the ACK. Node 20 and 26 send the ACK first.
  • The unfixed client also ACKs the server's initial SETTINGS late, after the first 65535 bytes of DATA. The test therefore matches ACKs by count (the raise is the second ACK) instead of a flag set by the first ACK seen.
  • The test replaces the repro's 500ms stall timer with an exact trigger: the server raises the window when the received DATA total reaches 65535, the point where the client cannot send another byte. Any DATA between that SETTINGS and the second ACK used the unacknowledged window.
  • The test is a standalone file because it needs a raw net server that drives Bun's client, and no existing http2 test file has that harness (h2-conformance.test.ts is the inverse: a raw client against Bun's server). The directory already has per-fix files, for example node-http2-invalid-padding.test.ts.
  • The too-many-settings regression mechanism, from debug logs: the engine stops mid-batch after the first SETTINGS, the socket closes, and the remaining frames drain post-close. The old code corked the ACK and dispatched the GOAWAY before the cork drained, so the GOAWAY error won. The reorder plus unconditional flush pushed the ACK through the dead transport first, and the EBADF won. Flushing only when a window grew restores the old timing for that path. Verified deterministic: 20/20 fail before the gate, 20/20 pass after.
  • note_outbound_ack (the outbound control-frame flood check) now runs before the sink callback. If it trips, the session is torn down and the callback is skipped, which matches the other teardown paths in handle_settings.
  • grpc-js suite: 5 DNS tests, one outlier-detection timeout, and the tonic test fail in this environment without the change too (public DNS and external tooling), all other grpc tests pass. test-server-deadlines.test.ts passes 5/5 locally.
  • The issue also reports a secondary bug: write() callbacks report success for bytes that never reached the socket after the stream died. That is a separate defect and is not addressed here.

[review] gate passed · iteration 2 · 3 files touched

fails on main (without fix)
ASAN without fix: 1 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" "test/js/node/http2/node-http2-settings-ack-ordering.test.ts"
bun test v1.4.1 (4448a2e21)

test/js/node/http2/node-http2-settings-ack-ordering.test.ts:
121 |     req.on("error", failed);
122 |     req.write(Buffer.alloc(BODY, 0x61));
123 | 
124 |     await done;
125 | 
126 |     expect({ dataBeforeAck, framesBeforeAck }).toEqual({ dataBeforeAck: 0, framesBeforeAck: 0 });
                                                     ^
error: expect(received).toEqual(expected)

  {
-   "dataBeforeAck": 0,
-   "framesBeforeAck": 0,
+   "dataBeforeAck": 196609,
+   "framesBeforeAck": 13,
  }

- Expected  - 2
+ Received  + 2

      at <anonymous> (/workspace/bun/test/js/node/http2/node-http2-settings-ack-ordering.test.ts:126:48)
(fail) client sends the SETTINGS ACK before DATA unblocked by a larger INITIAL_WINDOW_SIZE [1069.73ms]

 0 pass
 1 fail
 1 expect() calls
Ran 1 test across 1 file. [5.60s]
error: script "bd" exited with code 1
__F:1:S:0

release without fix: all passed
bun test v1.4.1-canary.1 (048e73692)

test/js/node/http2/node-http2-settings-ack-ordering.test.ts:
(pass) client sends the SETTINGS ACK before DATA unblocked by a larger INITIAL_WINDOW_SIZE [20.05ms]

 1 pass
 0 fail
 2 expect() calls
Ran 1 test across 1 file. [150.00ms]
__F:0:S:0
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" "test/js/node/http2/node-http2-settings-ack-ordering.test.ts"
bun test v1.4.1 (4448a2e21)

test/js/node/http2/node-http2-settings-ack-ordering.test.ts:
(pass) client sends the SETTINGS ACK before DATA unblocked by a larger INITIAL_WINDOW_SIZE [769.30ms]

 1 pass
 0 fail
 2 expect() calls
Ran 1 test across 1 file. [3.89s]
__F:0:S:0

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 764ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/126] gen JSBuffer.lut.h
Generating /workspace/bun/build/release/codegen/JSBuffer.lut.h from /workspace/bun/src/jsc/bindings/JSBuffer.cpp
[2/126] gen generated_host_exports.rs
generated_host_exports.rs: 120 exports (host=5, lazy=10, generic=105, rust=0); 244 extern-C blocks audited
[3/126] gen cpp.rs (cppbind)
[4/126] gen JS modules (bundle-modules)
Preprocess modules (13342ms)
Bundle modules (61ms)
Postprocesss modules (168ms)
Bundle Functions (669ms)
Generate Code (57ms)

[14.30s] Bundled "src/js" for production
  2594 kb
  197 internal modules
  13 native modules
  50 internal functions across 16 files
[4/125] cargo bun_runtime → libbun_runtime.a (--target x86_64-unknown-linux-gnu)

  nightly-2026-07-20-x86_64-unknown-linux-gnu unchanged - rustc 1.99.0-nightly (9f36de775 2026-07-19)

   Compiling bun_runtime v0.0.0 (/workspace/bun/src/runtime)
    Finished `release` profile [optimized + debuginfo] target(s) in 4m 46s
[121/125] cxx obj/codegen/GeneratedFakeTimersConfig.cpp.o
... (truncated)
diff hotspot
src/runtime/api/bun/h2/connection.rs               |   6 +-
 src/runtime/api/bun/h2_frame_parser.rs             |   9 +-
 .../http2/node-http2-settings-ack-ordering.test.ts | 132 +++++++++++++++++++++
 3 files changed, 143 insertions(+), 4 deletions(-)

gate history · 3 passed · 0 rejected · iteration 2

evidence per changed file
file                                                      reads  edits  tests
src/runtime/api/bun/h2/connection.rs                          2      3      0
src/runtime/api/bun/h2_frame_parser.rs                        4      5      0
…/js/node/http2/node-http2-settings-ack-ordering.test.ts      2      7      0

root cause · written by the author bot

The bug was in the HTTP/2 client's SETTINGS handling, where the callback that applied a peer's raised SETTINGS_INITIAL_WINDOW_SIZE to open streams and flushed their queued DATA ran before the SETTINGS ACK was queued, so stalled DATA reached the wire ahead of the ACK and peers like nghttp2 and grpc-go measured it against the old window and reset the stream. The fix reorders handle_settings to queue the ACK before on_remote_settings runs, ensuring the acknowledgment always precedes any DATA sent under the enlarged window. It also gates the post-settings flush on an actual window increase, whi…