child_process: apply kernel backpressure to stdout/stderr pipes

Merged
RO
4mo ago
+135-17

Problem

node:child_process stdout/stderr pipes had no backpressure: a child writing 100 MB into a pipe nobody reads was never blocked (completes in <4 s; Node blocks at the ~64 KiB kernel pipe buffer). The parent buffered the whole output in RAM with no ceiling, and if 'exit' fired before a 'data' consumer attached the entire buffered output was discarded, so a late reader got 0 of 104,857,600 bytes.

import { spawn } from 'node:child_process';
const c = spawn('sh', ['-c', 'dd if=/dev/zero bs=1M count=100 2>/dev/null'], { stdio: ['ignore', 'pipe', 'ignore'] });
await new Promise(r => setTimeout(r, 4000));
console.log('exitedBeforeAttach', c.exitCode !== null); // node: false  bun: true
let got = 0; for await (const d of c.stdout) got += d.length;
console.log('bytes', got, 'of', 100 * 1048576);         // node: 104857600  bun: 0

Cause

PipeReader::start() registered the fd with epoll/IOCP immediately, so the event loop drained the child's output into PosixBufferedReader._buffer (which has no HWM for non-streaming parents) before any JS consumer attached. When the child exited, #handleOnExit called resume() on a stream with no 'data' listener, dropping every buffered chunk.

Fix

When Bun.spawn is called with lazy: true (the documented "don't start reading stdout until accessed" option, which node:child_process uses internally), start the pipe reader paused: set IS_PAUSED before PosixBufferedReader::start() so it skips register_poll(), and on Windows wire the source without calling start_reading(). The kernel pipe buffer then provides backpressure and the child blocks until JS first reads, matching Node. FileReader::on_start() clears IS_PAUSED before its first read() so the transferred reader wakes up.

Also:

  • #handleOnExit now matches Node's flushStdio: only resume() stdout/stderr when readableFlowing === null (the stream was never touched), so an explicitly paused stream is not drained behind the user's back.
  • Fixed a debug-build $debug template literal in native-readable.ts that interpolated the resolved Uint8Array directly. Uint8Array.prototype.toString() comma-joins every byte, so a 220 KB chunk spent ~600 ms stringifying under ASAN. This was pre-existing but the chunkier delivery made it visible.
  • Dropped lazy: true from the huge() fixture in spawn.test.ts; its after exit subtest relied on the eager buffering this removes (with correct lazy semantics a 250 KB write cannot finish before stdout is read).

Verification

$ bun-debug repro.mjs
exitedBeforeAttach false
bytes 104857600 of 104857600

The new stdout pipe backpressure > blocks the child until a reader attaches and delivers every byte test in test/js/node/child_process/child_process.test.ts fails on main (child exits with exitCode: 0 within 10 ms) and passes with the fix.

no test proof · iteration 3 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/spawn/spawn.test.ts test/js/node/child_process/child_process.test.ts