PR #37799 - oven-sh/bun

Problem

await $`cmd < ${buffer}` , where the child exits without reading a redirect bigger than the pipe, sometimes never resolves (2 of 8 runs on bun 1.4.0), and the process does not exit either, since the interpreter still counts as pending activity. It hangs every time if something holds the child's stdin open for a moment after the child exits.
A command running a subprocess is complete once it has an exit code and stdin, stdout and stderr have all closed. Those four events arrive as separate event-loop callbacks in no fixed order.
Three of the four moved the command to Done when they completed it. The stdin close only recorded itself, so whenever it landed last, the command was complete and nothing finished it.
It lands last often: the pending stdin write only fails with EPIPE once the read end of the pipe is gone, which is at the same time as, or after, the exit and the stdout/stderr EOFs.

Fix

The stdin close now does what the stdout/stderr closes and the exit already did: all three end in one shared tail that marks the command Done when it is complete and returns the resumption for the caller to run.
Correct because completion is a conjunction of four independently delivered events, so each must be able to finish the command; exactly one does, since the transition tears the subprocess down synchronously and the other events have already been delivered by then.
Running that transition can free the subprocess, so the stdin-close and exit callbacks take a raw pointer instead of &mut self (the shape the stdout/stderr readers already had; the exit path had the same latent problem), and the shared pipe writer nulls its process backref before its single close notification.
Verification: six new tests (Buffer and Blob redirects, a helper that exits or drains, the command inside a pipeline and on the left of ||) force the stdin close to be the last event. All six time out without the src/ changes and pass with them on bun 1.4.0 and a debug ASAN build; the existing shell, spawn and install-scanner tests still pass.

Background

A < ${buffer} (or Blob or Response) redirect in the Bun shell is pumped into the child's stdin over a pipe by a StaticPipeWriter, a writer Bun.spawn and the install security scanner also use. It reports its close to its owning process through a raw backref.
A shell Cmd that spawned a subprocess tracks the exit code plus a closed flag per piped stdio and is finished only when all are in, whoever holds the pipes: a helper that inherited the child's stdin keeps the stdin side open after the child is gone, and the shell keeps pumping to it.
Shell callbacks do not advance the interpreter directly. They return a Yield (resume this node, or nothing) that the caller runs; running it can complete the parent node and free the Cmd and its subprocess, so nothing may still borrow either at that point.

no test proof · iteration 2 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/shell/bunshell.test.ts

Original description

Repro

import { $ } from "bun";
// The child exits without reading its stdin; the buffer is bigger than the pipe,
// so the shell's stdin write is still pending when it does.
await $`sh -c 'exit 0' < ${Buffer.alloc(1 << 20, "a")}`.quiet();

This never resolves on some runs (2 of 8 with bun 1.4.0 here, about 2 of 3 on a debug build). Whether it hangs depends on the order in which the event loop happens to deliver four events that all become ready when the child exits. This variant forces the problematic order and hangs every time: a helper keeps the child's stdin open (without reading it) for a moment after the child itself has exited.

await $`sh -c 'exec 3<&0; sleep 0.5 <&3 >/dev/null 2>&1 & exit 0' < ${Buffer.alloc(1 << 20, "a")}`.quiet();

Cause

A Cmd running a subprocess is complete once it has an exit code and every piped stdio has closed (Cmd::has_finished). For a Buffer/Blob/Response redirect, stdin is piped: the bytes are pumped into the child by a StaticPipeWriter, and BufferedIoClosed::stdin has to be marked closed too. Three of the four events that can complete the command checked has_finished() and moved the Cmd to Done (Cmd::on_exit, and Cmd::buffered_output_close for stdout and stderr). The fourth, Cmd::buffered_input_close (reached from the writer's close through ShellSubprocess::on_close_io and on_static_pipe_writer_done), only set the flag.

When the child exits without draining its stdin, the pending write fails with EPIPE only once the pipe's read end is gone, so the stdin close is processed at the same time as, or after, the exit and the stdout/stderr EOFs. Whenever it lands last, the command is complete and nothing transitions it. The promise never settles, and since the interpreter still counts as pending activity the process does not exit either (the script above has to be killed), even though the child and all of its pipes are gone by then.

Debug trace of a hanging run (`BUN_DEBUG_SHELL=1 BUN_DEBUG_SHELL_SUBPROC=1 BUN_DEBUG_StaticPipeWriter=1`)

[shell_subproc] onProcessExit(...)
[shell] cmd exit code=0 has_finished=false
[shell] cmd close buffered stderr
[shell] BufferedIOClosed all_closed=false stdin=false stdout=false stderr=true
[shell] cmd close buffered stdout
[shell] BufferedIOClosed all_closed=false stdin=false stdout=true stderr=true
[staticpipewriter] StaticPipeWriter(...) onError(err=EPIPE: Broken pipe (send()))
[staticpipewriter] StaticPipeWriter(...) onClose()
[shell_subproc] Subproc(...) onStaticPipeWriterDone(cmd=Node#2)
(nothing further)

Fix

Cmd::buffered_input_close, buffered_output_close and on_exit all end in the same finish_if_done tail (has_finished() -> state = Done -> Yield::Next(this), including the existing "spawn has not returned yet" gate that transition_to_exec resumes from) and return the Yield to their caller. Previously only the output closes and the exit did this, and on_exit ran the Yield itself. buffered_input_close stays a no-op once deinit has taken exec, which is the only way it is reached during teardown (deinit_in_flight_io).
The two ShellSubprocess callbacks that run those Yields, on_stdin_writer_close (replaces on_static_pipe_writer_done plus the stdin arm of on_close_io) and on_process_exit, take this: *mut Self and are forwarded raw by the StaticPipeWriterProcess shim and the link_impl_ProcessExit! thunk. Running the Yield can reach Cmd::deinit, which frees the subprocess and recycles the Cmd's arena slot, so no &mut to either may be on the stack at that point; this is the shape PipeReader::on_reader_done / finish_after_state_set already use for stdout/stderr, and it also removes the pre-existing instance of the problem in the exit path. on_stdin_writer_close empties the Writable::Buffer slot before signalling (the subprocess may be gone afterwards); dropping create()'s ref there cannot free the writer the callback is running inside of, because every path into StaticPipeWriter::on_close holds start()'s ref (or the one deinit_in_flight_io claims) until after the callback returns.
on_close_io is left with the stdout/stderr reader path, its only remaining caller. Its old Writable::Pipe arm was unreachable: the shell never creates a FileSink stdin, and a FileSink reports its close through Writable::on_close, not this function.
StaticPipeWriter (src/spawn/static_pipe_writer.rs, shared with Bun.spawn and the install security scanner) so far documented its process backref as outliving the writer. The shell's callback can now free the process from inside on_close_io and run the rest of the script while doing so; nothing in the writer used the backref after that call before either, but the trait doc now states the contract, and on_close nulls the field before the (single) dispatch so a future use after it faults in every impl instead of dangling only for the shell. The other two impls are unaffected (they get the same pointer they got before).

Why this shape: the completion condition is a conjunction of four independently delivered events, so each of them has to be able to perform the transition, and making the stdin close do what the other three already do is the whole fix. Exactly one transition happens per command in any interleaving: the transition tears the subprocess down synchronously, and the other events have by then already been delivered (they are what made has_finished() true). The alternative would be to close the stdin writer when the child exits, as Bun.spawn's Subprocess does for its buffer stdin; that would also end the hang, but it changes what < means: a process that inherited the child's stdin and outlives it would get a truncated redirect. Waiting for the writer keeps stdin consistent with how the shell already treats stdout/stderr capture (the command finishes when the pipes close, whoever holds them), and it is what BufferedIoClosed tracking stdin was evidently written for; the new tests pin that choice.

#37774 fixes a separate leak of the writer's start() ref on the same EPIPE path; it also edits StaticPipeWriter::on_close, so one of the two needs a trivial rebase, and they compose (it releases the ref after the dispatch). #37652 refactors the SubprocExec fields finish_if_done reads. #36895 adds ReadableStream stdin and calls the old on_static_pipe_writer_done from the exit handler; on top of this PR that call would need to move to the Cmd side, since the replacement runs the Yield. None of them covers this hang.

Tests

test/js/bun/shell/bunshell.test.ts, "stdin redirect still held open by a helper after the command's process has exited". The child (bun -e) spawns a detached helper that inherits its stdin, writes its own pid to a file, prints to stdout and stderr and exits 3 without reading anything. The test waits until that pid is gone (the exit and both EOFs are then processed, or queued ahead of anything the helper can still cause) and only then releases the helper through a file, so the stdin close is always the last event, on every platform and without a fixed delay. Every helper records what it did once released, so a helper that died early (which also fails the pending write) cannot pass as one that exited on cue; the redirect is 4 MiB so the write is still pending when the child exits whatever the socket buffer size. The command's environment drops BUN_FEATURE_FLAG_NO_ORPHANS, which the ASAN CI lanes set and under which the child kills the helper as it exits (that is what the first CI run of the draining cases showed). Cases, for a Buffer and a Blob redirect each:

the helper exits without reading (the pending write fails): exit code, captured output and the helper's "released" record must come back;
the helper drains the redirect after the child is gone: the command must still wait for it, and the helper must receive all 4 MiB (a close-at-exit implementation would pass the first group and fail this one; this also exercises the drained close rather than the error close);
plus the exiting helper with the command inside a pipeline and as the left operand of ||, so the completion reaches a Pipeline and a Binary parent, not only a Stmt.

All six time out without the src/ changes and pass with them, both with and without BUN_FEATURE_FLAG_NO_ORPHANS=1 in the test process's environment (bun 1.4.0 and a debug ASAN build). Also passing on the debug ASAN build: the rest of bunshell.test.ts (422 tests), shell-worker-terminate-leak (covers deinit_in_flight_io closing the writer mid-flight), shell-hang, epipe, lazy, yield, pipeline_stack, bunshell-file, shelloutput, exec; for the other users of the writer, spawn.test.ts plus the stdin spawn tests both with memfd and with BUN_FEATURE_FLAG_DISABLE_MEMFD=1 (which forces Bun.spawn onto the pipe writer), and bun-install-security-provider.test.ts.