node:quic: use one default stream window per endpoint
Problem
- With default
transportParams, a node:quic server opens a bidirectional stream and sends more than 16 KiB. The connection dies:QUIC transport error 1: flow control violation on stream 1. - A unidirectional stream that a client opens in 0-RTT ends the same way after 12288 bytes.
us_nq_settings_init(packages/bun-usockets/src/node_quic_shim.c:424) keeps lsquic's HTTP/3 windows: 0 for a server-opened bidirectional stream, 12 or 32 KiB for a unidirectional one. lsquic also reads the wrong window as the send limit.
Fix
- An endpoint now has one default stream window: 1 MiB on a server, 6 MiB on a client.
- With equal windows a sender reads the right number whichever window it takes. Explicit
transportParamsstill win. - Verified:
test/js/node/quic/quic-stream.test.ts, 4 new tests, main fails 3 in every run. Also the 235test-quic-*.mjsfiles. - Self-reviewed with #46322: 35 concerns raised, 29 addressed. Its Notes list the rest.
Background
- A QUIC endpoint advertises a receive window per kind of stream. The kinds: bidirectional that it opens (
bidi_local), bidirectional that its peer opens (bidi_remote), unidirectional. - Only node:quic calls this function.
Bun.serveandfetchdo not change. - Considered Node's 256 KiB for the two zero windows only. A bun 1.4.2 server then killed 5 of 6 connections at 4 MB.
Downsides
- A node:quic handshake carries 6 (client) or 8 (server) more transport parameter bytes.
- The default unidirectional window was 12 KiB or 32 KiB. The connection window still bounds what a peer sends unread.
- Explicit windows that differ still fail. #46322 corrects that.
Notes
How it was found. The failure is in the "Found, not in this PR" list of #46054. No user reported it.
Why equal windows are enough for the default case. lsquic keeps one send limit for bidirectional streams and picks it by the role of the connection (apply_trans_params, lsquic_full_conn_ietf.c). A server takes the client's bidi_local, a client takes the server's bidi_remote. That is right only for a stream the client opens. After 0-RTT the same function gives a unidirectional stream the server's bidi_remote. When an endpoint's three windows are equal, every one of these reads returns the right number. The same values set by hand through transportParams carry 4 MB each way on main and on bun 1.4.2.
Measurements. Release builds of main 71bc8185c8 and of this PR. A server-opened bidirectional stream, two processes on loopback, 6 runs each. "1.4.2" is the released bun.
| Case | main | this PR |
|---|---|---|
| this build on both ends, 4 MB each way | 6 of 6 connections die | 6 of 6 complete |
| 1.4.2 server sends 4 MB to this build | 6 of 6 die | 6 of 6 complete |
| this build as server opens the stream, a 1.4.2 client sends 4 MB on it | 6 of 6 die | 6 of 6 complete |
| this build as server sends 1000 bytes to 1.4.2 | complete | complete |
| this build as server sends 262144 bytes to 1.4.2 | 5 of 6 die | 5 of 6 die |
| 1.4.2 server, this build as client sends 1000 bytes | complete | complete |
| a resumed 1.4.2 client sends 400000 bytes on a 0-RTT unidirectional stream | 0 of 4 complete | 4 of 4 complete |
No pair is worse than before. A new server that sends to a 1.4.x client still overruns that client's window, because the client advertises 0 and accepts 16 KiB.
- Transport parameters of a node:quic handshake: client 72 to 78 bytes, server 90 to 98 bytes plus the connection ID.
Bun.serveandfetch: 72 and 88 bytes before and after. - Session ticket: 1510 to 1522 bytes before, 1534 to 1546 after (default ALPN). With
quic-test: 1533 to 1545 before, 1541 to 1553 after. 80 tickets each. The range is the connection ID length. - Release binary: +256 bytes of text (
size). One symbol changed,QuicEndpoint::build_engine, +219 bytes.
Tests. On a release build of main, "are one value per endpoint", "256 KiB to the client" and the 0-RTT test fail in 6 of 6 runs. "256 KiB from the client" depends on timing there: it fails in 12 of 12 runs as the only test of a process and passes after the other tests. No body size makes that direction fail in every run, because the server reads as fast as the data arrives and its window moves.
Also run on the debug build: all 235 test/js/node/test/parallel/test-quic-*.mjs files. 229 pass. The 6 others fail on main as well: 2 import node:stream/iter, 1 asserts an RTT below 10 ms, 3 test-quic-callback-error-* are timing dependent (1 to 2 of 6 runs on both builds).
Not in this PR. lsquic still picks the send limit by role. With explicit windows that differ from each other, a server-opened stream still fails. #46322 is based on this PR and changes lsquic.