shell: keep both streams when `&>` redirects to a buffer or a Bun.file(fd)

RO
11d ago
+83-13

Problem

  • cmd &> ${buf} loses output (buf is an ArrayBuffer or typed array). Each stream writes from offset 0, so the second writer overwrites the first. sh -c 'echo out-line; echo err-line >&2' &> ${buf} leaves only err-line\n.
  • A builtin gets two BuiltinIO::ArrayBuf { buf, i: 0 } values (src/runtime/shell/Builtin.rs:686). A spawned command gets two pipes, each with its own cursor (src/runtime/shell/states/Cmd.rs:751).
  • A spawned cmd &> ${Bun.file(fd)} loses stderr. The Blob arm (Cmd.rs:757) is an else if chain: only stdout gets the fd.

Fix

  • Builtin: both streams share one Rc<Cell<u32>> cursor.
  • Spawned command: with a buffer target, stderr becomes a Stdio::Dup2 of stdout's pipe, as for a plain 2>&1. Two pipes cannot keep write order, because the shell reads them in poll order. With a Bun.file(fd) target, both streams get the fd.
  • Windows: spawn_process_windows frees the unused uv::Pipe of a dup target. A plain 2>&1 leaked it.
  • Verified: test/js/bun/shell/bunshell.test.ts, four new tests. They fail on main a8b5e920c4 and pass with the fix.

Background

  • A builtin runs inside bun. A spawned command writes to a pipe, and a PipeReader copies each chunk into the buffer.
  • Stdio::Dup2 { out, to } makes the child's out fd a duplicate of its to fd.
  • Bun.file(path) is inlined into the script as a path. Only Bun.file(fd) reaches the Blob arm.

Downsides

  • A builtin with a buffer redirect makes one more 24-byte allocation, for > and 2> too. .text stays at 82,257,737 bytes.
  • Still broken: a builtin's > ${Bun.file(fd)} writes nothing, a separate defect on main. 2>&1 ${buf} keeps two cursors.
Notes

Found by a read of the code during other shell work. No user reported it. The separate cursors date from the first shell commit (#7748), which copied one { buf, i: 0 } value into both slots. Workarounds on a released bun: cmd 2>&1 | cat > ${buf}, or cmd 2>&1 and read .stdout.

Reproduction (1.4.3 and main a8b5e920c4):

import { $ } from "bun";
$.nothrow();
const a = new Uint8Array(256);
await $`ls /usr/bin/env /nonexistent-dir-zz &> ${a}`.quiet();
// a holds only "ls: /nonexistent-dir-zz: No such file or directory\n"
const b = new Uint8Array(256);
await $`sh -c 'echo out-line; echo err-line >&2' &> ${b}`.quiet();
// b holds only "err-line\n"

Left alone on purpose, with the reason:

  • 2>&1 ${buf} and 1>&2 ${buf} keep one pipe and one cursor per stream, as on main. These spellings reach a buffer target by accident: the parser takes the word after an fd dup as a file target. bash passes that word to the command as an argument. #34901 makes bun do the same, which removes the path. A test here would pin the accident.
  • A builtin with a Bun.file(fd) target (Builtin.rs:721). On main a8b5e920c4, echo out > ${Bun.file(fd)} exits 0 and the file stays empty, because BuiltinIO::Blob writes are a no-op. With &>, stdout is dropped and stderr still reaches .stderr, because the else if chain never assigns it. To assign it now would drop the error text too. The fix is a real writer on the fd that both streams share. #35318 had that fix. The stale bot closed it on 2026-10-23 with no maintainer comment, and no open PR covers it now. It is a change of its own, so it is not in this PR.
  • The Response arm of init_subproc_redirections (Cmd.rs:778). cmd &> ${new Response(Bun.file(fd))} throws Blobs are immutable, and cannot be used for stdout/stderr before the spawn, because the second use_as_any_blob() sees a used body. That is a loud failure, not lost output. #46687 turns a Response over Bun.file(path) into a path redirect.

Other open PRs that touch the same code:

  • #46687 and #42786 change the head of the two JS object arms. This diff is inside the arm bodies, so there is no textual overlap.
  • #34901 edits the two duplicate_out() lines of the buffer arm in Cmd.rs. The merge is mechanical.
  • #36480 rejects >> to a buffer. For that reason there is no &>> test here. &>> takes the same code path as &>.
  • #42819 removes libuv on Windows and deletes spawn_process_windows. If it lands first, the src/spawn/process.rs hunk goes away with it.

This branch is rebased on #46021, which made the builtin write arm read the live range of the buffer. The rebase kept that and changed only how the cursor is read and written. A second rebase, onto #47238 (main 2871f0cb47), dropped the use std::sync::Arc import that #47238 removed. The diff is otherwise the same.

Cost, measured on release builds of a8b5e920c4 and of this branch (size, nm -S):

  • .text is 82,257,737 bytes in both. The stripped binary is 107,837,152 bytes in both.
  • The shell functions that changed net to -5 bytes. The larger ones: BuiltinIO::write_no_io_to +23, drop_glue::<BuiltinIO> +25, a new Rc<Cell<u32>>::drop_slow of 27, Cmd::transition_to_exec +15, Rm::next -144.
  • The one new allocation per builtin buffer redirect is the Rc (two counts and the u32, 24 bytes). A spawned &> ${buf} now opens one pipe, not two.
  • There is no instruction count. perf and valgrind are not installed in the container.

The Windows free has no test of its own. The leak is one never-initialized uv::Pipe (a few hundred bytes) per command. An RSS probe ran the same command 2000 times on a stock binary. Two runs gave -1084 KB and +416 KB, so the signal is below the noise. as_spawn_option (src/runtime/api/bun/spawn/stdio.rs:199) documents the contract: the caller must move the pipe into WindowsStdioResult::Buffer or free it with close_and_destroy. The dup-target branch did neither on the success path. The error path already frees it through WindowsStdio::deinit. close_and_destroy frees a pipe whose loop_ is null without a uv_close. Only the shell builds a Stdio::Dup2, so Bun.spawn never reaches this branch. The existing 2>&1 tests and the new buffer test run the free on every Windows run.

Test notes:

  • ls with two operands runs one thread-pool task per operand, so the order of its two lines is not fixed. That test sorts the lines.
  • The yes test is the deterministic builtin case. yes fills the 64-byte buffer, then writes yes: ENOSPC\n to stderr. Before the fix that message replaced the first 12 bytes of the output.
  • The spawned-command tests assert strict write order. The existing 2>&1 tests in the same file assert the same order for a bun child.

Probe: a child that writes 2000 lines of 100 bytes to each stream, in turns. All 404000 bytes arrive and the lines alternate, on Linux and on Windows. On 1.4.3 about half of the bytes survive.

Runs:

  • Linux x64, debug build (ASAN) of a8b5e920c4 plus this branch: bunshell.test.ts 468 pass, 0 fail. With the src/ of a8b5e920c4 the four new tests fail. The four tests run concurrently. 20 repeats gave no failure.
  • Linux x64, debug build (ASAN) of 2871f0cb47 plus this branch, after the second rebase: bunshell.test.ts 468 pass, 0 fail. The four new tests fail on bun 1.4.3.
  • Windows x64, before the rebase (0d3492e353 plus this branch): bunshell.test.ts 424 pass, 5 fail. The five are tilde_expansion tests that fail the same way with the stock binary on that machine. CI passed on every Windows lane at that point.
  • Windows after the rebases: not run locally. cargo check -p bun_spawn --target x86_64-pc-windows-msvc passes on 2871f0cb47 plus this branch.

Self-reviewed: three concerns raised, three addressed. The first version also rerouted 2>&1 ${buf}, which is out. The Bun.file(fd) arm came in. The Windows free was out for one push and is back. Without it this PR adds a leak that main does not have for &> ${buf}.


no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/shell/bunshell.test.ts