process.stdin: do not reset the ended state when resume() follows EOF
Problem
process.stdinnever emits'end'when EOF arrives while it is paused with bytes buffered. The program exits 0 and never runs its'end'handler.pipeline(process.stdin, process.stdout, cb)inbun copy.mjs < in.bin | catnever calls back for a 1,048,577-byte file: 3 of 3 runs. Anode:readlineconsumer that pauses in'line'never gets a last line that has no line feed: 5 of 5 runs. Node.js ends each run.- Regression from #33123 (Bun 1.4.0), which made EOF a
push(null). The'resume'listener ingetStdinStream(src/js/builtins/ProcessObjectInternals.ts:308) calls_undestroy()on each resume, which clearsended. Nothing pushes EOF again.
Fix
- The listener returns once stdin has reached EOF. #33123 added this check to
own(), but not to the_undestroy()call. - Correct because after EOF there is nothing to revive: the native reader is released for good.
- Verified:
test/js/node/process/process-stdin.test.ts(5 new cases, all fail on main). Also the stdin, stdio, readline and console suites.
Background
process.stdinis a NodeReadableoverBun.stdin.stream(). At EOF it releases the native reader and callspush(null).'end'fires when the buffer is empty._undestroy()is the Node helper that resetsdestroyedandended. Bun calls it on'resume'to revive a destroyed stdin.- Considered removing the listener, as the first version of this PR did. That breaks programs that work on main. #48174 proposes it.
Downsides
destroy()before EOF, thenresume(), still revives stdin.- One more boolean read per
'resume'event. 41 more bytes of builtin source.
Notes
State of this version. Main is merged in at 2596ac5d39. The source change is the same two lines. One case is new: a file on stdin piped to a stdout under backpressure. On that merge a debug build passes process-stdin.test.ts 28 of 28. With the guard taken out, the 5 new cases fail and the other 23 pass.
A second report: stdin from a file, and resume() a moment after pause(). Debug builds of main (7ca0042d3b) and of this branch. Node.js v26.3.0 ends every run of every row.
Stdin is a file and stdout is a shell pipe into cat. The rows are pipeline(process.stdin, process.stdout, cb) and process.stdin.pipe(process.stdout, { end: false }) at 8 sizes, 3 runs a cell:
| Build | Cells with a missing 'end' or callback |
|---|---|
| main | 4 of 16: both rows at 1,048,577 and at 1,048,578 bytes, 3 of 3 runs each |
| this branch | 0 of 16 |
Stdin is a pipe from dd bs=777 with 1,048,577 bytes. Both processes run on one CPU. Runs of 5 in which 'end' came:
| Row | main | this branch |
|---|---|---|
stdin.pipe() into a Writable with highWaterMark: 1 that calls back in a setImmediate | 0 | 5 |
pipeline(stdin, Transform, Writable), highWaterMark: 1, the Transform calls back in a setImmediate | 0 | 5 |
pause() in 'data', resume() in a setImmediate | 0 | 5 |
the same with setTimeout of 0 ms, and of 2 ms | 0, 0 | 5, 5 |
the same with process.nextTick | 5 | 5 |
7 more rows (queueMicrotask, pause() and resume() at once, no pause, for await, 'readable', Bun.stdin.stream(), a Transform with the default highWaterMark) | not run | 5 each |
Bun 1.4.3 canary (release), the same rows, runs of 10 in which 'end' came: 3, 2, 0, 0, 0.
In each failed run the trace shows the same order. push(null) sets ended while stdin is paused and readableLength is not 0. The next 'resume' event clears ended.
node:readline loses the last line. readline gives a last line that has no line feed only when its input ends. in.txt holds 20,000 lines of 50 bytes and then LAST with no line feed.
// rl-last.mjs dd if=in.txt bs=777 status=none | bun rl-last.mjs
import readline from "node:readline";
const rl = readline.createInterface({ input: process.stdin });
let n = 0, last = "", closed = false;
rl.on("close", () => { closed = true; });
rl.on("line", l => { n++; last = l; rl.pause(); setImmediate(() => { if (!closed) rl.resume(); }); });
process.on("exit", () => console.log(`${n} lines, last=${last.length > 10 ? "(a line)" : last}, close=${closed}`));
Runs of 5 in which all 20,001 lines came. The second row pauses at each 100th line and resumes in a setTimeout of 0 ms:
| Row | main, debug | this branch, debug | Bun 1.4.3 canary | Node.js v26.3.0 |
|---|---|---|---|---|
pause() in each 'line', resume() in a setImmediate | 0 | 5 | 1 | 5 |
pause() at each 100th line, resume() in a setTimeout | 0 | 5 | 4 | 5 |
In each run that is not whole, exactly LAST is missing, 'close' does not come, and the exit code is 0.
The for await (const line of rl) loop over cat l100k.txt | bun rl.mjs hangs in 15 of 400 runs on main (the table under Measurements).
The new case. pipeline(process.stdin, process.stdout, cb) with a 1,048,577-byte file on stdin and a socket pair on stdout (POSIX).
- The parent reads the socket only after the child reports a
write()that returnedfalse, and then exactly the bytes written up to that point. So each chunk that is larger than the socket buffer pauses stdin. - Bun 1.4.3 canary and a debug build with the
src/of main fail 30 of 30 runs each: all bytes arrive, the exit code is 0,callbackisnullandendisfalse. A debug build of this branch passes 30 of 30. - With a reader that does not wait, the same program ends on main in 7 to 9 of 10 runs. The test does not use one.
- The case does not assert that EOF was buffered behind data. The chunk sizes come from the native file reader, and I measured them on Linux x64 only (262,144, then 524,288, then 262,144, then 1).
- Debug build of this branch:
process-stdin.test.tspasses 28 of 28. With thesrc/of main, the 5 new cases fail and the other 23 pass.
Scope of this version. The first pushed version of this PR (c75c3a7) removed the 'resume' listener and made own() return for a destroyed stream. This version is the regression fix only. A review of the follow-up work showed that the removal alone breaks programs that work on main:
- A raw-mode prompt reader that turns Ctrl-D into
process.stdin.emit('end')and callsresume()for each prompt. Main and Node.js answer the second prompt. With the listener removed, the internal'end'listener destroys stdin and nothing revives it, so the second prompt gets no input. - An ESM entry that calls
process.stdin.destroy()and later runs a top-levelfor awaitover readline. Main reads the input. With the listener removed, the program waits forever. Node.js exits with code 13.
The removal also did not make destroy() final, as the earlier body said. The native reader is released by a 'close' listener. That listener does not run after an unhandled 'error' or after removeAllListeners(). With the listener removed, 62 of 90 measured destroy() orders still kept the process alive where Node.js exits (77 on main). The sentence "The _destroy override of #35621 is not needed" in the earlier body was wrong. #48174 proposes the destroy() work with that override.
Measurements. Linux x64, release builds of main (25ef6936bd) and of this change on top of it. Input: seq 1 100000 > l100k.txt (588,895 bytes).
// rl.mjs
import readline from "node:readline";
const rl = readline.createInterface({ input: process.stdin, crlfDelay: Infinity });
let got = 0;
for await (const line of rl) got++;
console.log(`ok ${got}`);
cat l100k.txt | bun rl.mjs, one run at a time, killed after 6 s:
| Build | Runs that hang |
|---|---|
| main | 15 of 400 |
| Bun 1.4.3 canary (367d939d9) | 6 of 100 |
| this branch | 0 of 800 |
| Bun 1.3.14 | 1 of 500 |
| Node.js v26.3.0 | 0 of 100 |
The one timeout on Bun 1.3.14 did not occur again in 400 more runs. I did not find its cause.
A consumer that calls pause() in 'data' and resume() from a microtask or from process.nextTick, over the same cat: this branch ends 40 of 40 runs in each mode. Main ended 40 of 40 and 39 of 40. The hung run reported flowing=true ended=false reading=true length=0.
How the state is reached. _undestroy() also clears reading. After a 'resume' event, _read() runs again while the first reader.read() is still in flight, so two reads are in flight. When the pipe closes, one read resolves with the last chunk and the next with done in the same microtask drain. If 'data' pauses the stream on that chunk and data stays buffered, push(null) sets ended with a buffer that is not empty. The 'resume' event that follows clears ended. own() does nothing after EOF, so _read() cannot push EOF again. Bun 1.3.14 emitted 'end' by hand at EOF and never read ended.
Tests.
pause() in 'data', EOF, then resume()(2 cases, POSIX): the regression in flowing mode, with no race. Apause()andresume()pair leaves two reads in flight. The child then blocks until the parent has written one chunk to a socket pair and shut down its end. The parent usesshutdown(), because aclose()is not EOF while the child of another test holds a copy of the descriptor. The'data'listener pauses and puts the tail of the chunk back. The case asserts its precondition:endedis already set whenresume()runs (endedAtResume).- Bun 1.4.3 canary reports
end: falsein 32 of 32 runs. A debug build with thesrc/of main does the same. - A debug build of this branch passes 20 of 20 runs.
- Bun 1.3.14 and Node.js v26.3.0 emit
'end'in 20 of 20 runs each. They reportendedAtResume: false, because they have one read in flight. - The first version of this case used a FIFO for stdin. It timed out on both macOS lanes in CI. I cannot run macOS, and I did not find the cause. A lead that I did not verify:
Bun.spawnclearsO_NONBLOCKon an fd that becomes the stdin of the child (#43868), andsrc/io/PipeReader.rsnotes thatpoll(2)is unreliable on FIFOs on macOS. The other cases in this file use a socket pair for stdin and pass on macOS, so this case now uses one too.
- Bun 1.4.3 canary reports
resuming after EOF is buffered behind unread data still delivers the data and 'end': the same state through a'readable'listener. It asserts the precondition (buffered: 8). Bun 1.3.14 fails it at the precondition, because 1.3.x never had EOF buffered behind data.resume() after 'end' does not reset readableEnded: Node.js parity for the same_undestroy()call. Bun 1.3.14 fails it too. This is the case of #33556.
Suites.
- Release build of this branch: 231 pass and 0 fail across
process-stdin.test.ts,process/stdin/,process-stdio.test.ts,readline/andtest/js/bun/console/. Main fails only the 4 new cases. - Node's
test-stdin-*,test-readline-*andtest-tty-*(33 files): all exit 0 on both builds. - Debug build:
process-stdin.test.tspasses (26 of 26). With thesrc/of main, the 4 new cases fail. - Debug build only, and the same with the
src/of main:stdin-fixtures.test.ts,stdin-pause-pty.test.tsand 3 cases ofprocess-stdio.test.tstime out. - The release runs are older than the rewrite of the
pause() in 'data'case. The debug runs include it. - I cannot run macOS or Windows. The socket pair cases are POSIX only.
Related PRs. #34613 (re-own in _read, closed as stale) does not fix this hang: own() returns early after EOF. #33556 was closed in favor of this PR, and its case is a test here. #35621 was closed in favor of the first version of this PR. This version does not cover its destroy() cases.
no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/process/process-stdin.test.ts