#40078 spawnSync: release polls and keep-alives on the loop they counted on

Merge with main (ba3f27d1d1, 2026-09-30), in two merge commits (bb97ecd34b, 0dedb9f91b), followed by 87f1190cdc (see Costs). Four files conflicted. ParentDeathWatchdog.rs: main renamed file_poll::Pollable to file_poll::Flags, and the register call keeps this branch's EventLoopCtx argument. WindowsNamedPipe.rs: main replaced the impl_streaming_writer_parent! invocation with hand-written Windows impls, so main's file is taken whole and this PR no longer touches it. timer/mod.rs: main removed a clippy allow above increment_timer_ref and increment_immediate_ref, and this branch's signatures stay. posix_event_loop.rs: #43868 added Flags::Tty at the end of the enum where this branch added Flags::SpawnSyncLoop. Both stay, and the flag set (25 variants) is still 4 bytes. main's other changes to PipeReader.rs (#43868, #43900, #44086), PipeWriter.rs and FileSink.rs merged without conflict.

Fail-before on current main: a debug build with src/ and packages/ from main dc3660dea6 and this branch's tests. finalizers that run inside spawnSync do not stall the next spawnSync fails with {"collectedDuringCall":3,"stdout":"","exitedDueToTimeout":true,"exitCode":0}. a writer finalized during spawnSync does not break the next writer on the same fd fails with EEXIST: file already exists, epoll_ctl. The other six cases of the first file pass there. On this branch the two files pass (7 pass with --timeout 120000, and 18 pass with 2 skip).

build-toolchain-identity.test.ts: the flaky annotation for the case a file changes when its tool's path resolves to another tool, and only then, with the message Cannot tell which /…, is on 47 of the 60 most recent finished builds (121638 to 121728). The case fails in the parallel batch and passes alone. toolIdentity (scripts/build/tools.ts:217) runs child_process.spawnSync(exe, ["--version"], { timeout: 30_000, stdio: ["ignore", "pipe", "pipe"] }) on a two-line /bin/sh script, and the call ends at its own timeout. Stand-in: three Bun.stderr.writer() sinks held on globalThis with stderr on a pipe, a warm-up Bun.spawnSync, the drop, one more Bun.spawnSync under BUN_JSC_slowPathAllocsBetweenGCs=5, then that child_process.spawnSync with a 3s timeout. Debug build of main dc3660dea6: ETIMEDOUT after 3.4s to 3.7s in 3 of 3 runs. Release 1.4.3-canary 367d939d9: ETIMEDOUT in 3 of 3. This branch: clang version 21.1.8 with no error in 0.3s to 0.7s in 9 of 9. With zero writers main returns normally. The test file alone passes 15 of 15 on this branch.

Costs, how they were measured. Struct sizes: a compile-time size_of probe on both trees (KeepAlive 1 and 2 bytes, FilePoll 40 and 40, FlagsSet 4 and 4). timer::All gains two bools, one instance per VM. Lookups, counted from the diff: FilePoll::init reads the ctx's loop pointer and uws_get_loop() (a thread-local read) once each to set the bit, where main did no lookup. register_with_fd and unregister_with_fd resolve the loop once, as their callers did on main. EventLoopCtx::loop_ref does the same two reads as init before the ref, and loop_unref_for does one lookup, as loop_unref did. Binary size: size and a per-symbol nm -S diff on linux-x64 release builds of the merge base ba3f27d1d1 and of this branch, same checkout, only src/ swapped. The build is reproducible (two builds of 0dedb9f91b are byte-identical). The first merged revision, 0dedb9f91b, had 66,560 more bytes of .text than the base (58,217,461 against 58,150,901). The lookups that KeepAlive::ref_ and unref gained were inlined with them into 572 symbols: the node:fs async tasks, work-pool jobs, dns, sockets and the other users. CI's binary-size annotation on build 121730 agrees (+64.0 KB on linux-x64 and on linux-x64-musl). In 87f1190cdc EventLoopCtx::loop_ref returns the bit itself, and it and loop_unref_for are #[inline(never)], so a caller keeps a status check and one call. FilePoll::register_with_fd and unregister_with_fd resolve the loop once and pass &mut Loop to the private helpers, as on main. Result for 2d3f147ea0: .text 58,144,245 bytes, 6,656 fewer than the base, and the file is 8,192 bytes smaller. CI's binary-size annotation on build 121775 compares against main #121761. No target is larger. linux-x64 is 12.0 KB smaller and linux-aarch64 is 64.0 KB smaller. strace, perf, valgrind and bloaty are not installed in this container, so there is no instruction or syscall count from a tool.

Suites on the merged debug build (0dedb9f91b), on a host under heavy load: spawnsync-isolated-event-loop.test.ts (7 pass, where the two #40508 cases take 10s to 13s under collectContinuously on this build and on main, above the 5s default), spawnSync.test.ts (18 pass, 2 skip), process-stdio.test.ts (18 pass), then spawn.test.ts, 10887.test.ts, filesink.test.ts, node-timers.test.ts, spawnsync-no-microtask-drain.test.ts and build-toolchain-identity.test.ts in one run with --timeout 120000 (278 pass, 8 skip, 1 fail). The one failure is the with BUN_FEATURE_FLAG_FORCE_WAITER_THREAD wrapper in spawn.test.ts, which hit its own 192s cap. The same inner run, started directly with a 60s per-test timeout, passes (177 pass, 9 skip, 210s). The time goes to the 200 debug bun -e children that several of its cases start. cargo check passes for x86_64-pc-windows-msvc, x86_64-apple-darwin and aarch64-unknown-linux-musl. Again on 87f1190cdc: the two test files, the stand-in above (3 of 3 return normally), 10887.test.ts, filesink.test.ts, node-timers.test.ts, spawnsync-no-microtask-drain.test.ts, build-toolchain-identity.test.ts, process-stdio.test.ts and spawn-streaming-stdin.test.ts (120 pass in one run), spawn.test.ts in waiter-thread mode (177 pass, 9 skip) and in the default mode (177 pass, 8 skip, and the same wrapper at its 192s cap), and the same three cargo check targets. One run of spawnSync.test.ts failed should not set a timeout if timeout is 0 and inherit is used for stdout on its toBeLessThan(1000) wall-clock check (1190ms for a debug child). A spawnSync of sleep 0.3 takes 300ms to 1600ms on this host with the release builds of main and of this branch alike.

Earlier summary, moved here from the visible part. The user-visible face (#34069): under bun test --isolate, a file that touches process.stderr and calls Bun.spawnSync times out after 5000ms, or the run hangs until it is killed. spawnSync never returns, the child stays a zombie, and the parent spins at 100% CPU. Other signatures of the spin and of the stale epoll entry: ✗ does not segfault [90001.51ms] (10887.test.ts), and in the shell ShellError: Failed with exit code 17 (#40611). An epoll entry is keyed by the open file and the fd number. The kernel drops it only when the last fd for that open file closes. A dup of stdio never reaches that point while the process runs, so a missed EPOLL_CTL_DEL leaves the entry behind for the next dup that lands on the same number. FilePoll is one registration of an fd on a loop. KeepAlive is the on/off loop ref used by servers, sockets, fetch, fs and dns. FilePoll::activate and deactivate take the EventLoopCtx too, and SpawnSyncEventLoop::prepare asserts in debug builds that the isolated loop's counters are zero before each spawn. This PR supersedes #37754 (same cause, predates #38883) and #40611 (same FilePoll cause, stored a loop pointer instead of the bit, and its test is folded in here). main fixed the double close that #40611 also reported in #38354. This PR's tests still fail on main with #40508, because FilePoll, KeepAlive and the timer counters resolve their loop through EventLoopCtx, not through the EventLoop.

The notes below predate the merge of 2026-09-30.

How the bucket was identified: every flaking build ran the same 65-file batch, and in that batch 10887.test.ts was the tenth file of the third worker, right after web-globals.test.js (four stdin: "pipe" children), the napi test_object suite (four pipeless spawnSyncs), 013880, 04893 and 05828. 11 of the last ~110 PR builds flaked on at least one of debian 13 x64/aarch64 and ubuntu 25.04 x64/aarch64, no flake on any build with a different bucket. #38883 saw the same state on 07261.

Repro of the mechanism: with a debug build instrumented to record, per FilePoll, the loop it was activated on, a forced collectNow(Sync) at the end of spawnSync logged FileSink__finalize → PosixStreamingWriter::drop → FilePoll::deinit_force_unregister → deactivate against the isolated loop for polls activated on the main loop, then prepare: num_polls=-1 on the next spawnSync and a pipeless napi spawnSync spinning with num_polls=0. Without the forced GC the drift did not reproduce locally in ~100 batch runs (release canary and debug, with CI's BUN_GARBAGE_COLLECTOR_LEVEL=1), so the CI trigger is GC timing. The isolated loop is reused for every spawnSync in the process, which is why the drift accumulates.

The asan abort (#40611): 15 of 16 asan builds, always the worker that ran test/regression/issue/20753.test.js right after 19652.test.ts (a Bun.spawnSync). The GC epilogue runs inside spawnSync on the first JS allocation after the wait (create_resource_usage_object) and finalizes the previous test file's process.stderr FileSink. Timeline from an instrumented asan build: epoll ADD ok (default loop), epoll DEL ENOENT (other loop) from FileSink::finalize, Closer::close on the work pool, epoll ADD EEXIST (default loop) for the next Bun.file(2).writer(), FileSink::setup closes the fd, then the writer's Drop schedules a second close that fails with EBADF. The panic was visible in the worker log but not in annotations: the runner re-ran the crashed files alone and marked the job flaky. On macOS kqueue knotes are per fd, so the EEXIST symptom cannot appear there; the spawnSync.test.ts case is Linux-only for that reason. The wrong-loop EV_DELETE and the counter drift applied on macOS too and are fixed by the same change.

Merge with main (367d939d9a): main's #38354 fixed the double close with the opposite fd contract. A failed POSIX start() now leaves the fd with the caller, and FileSink::setup closes it. This branch first made the writer own the fd. The merge takes main's contract: FileSink::setup and PosixStreamingWriter::start keep main's bodies, and this branch's filesink.test.ts case is gone because main's "whose registration fails closes the dup exactly once" covers it. The EEXIST half stays here, with its spawnSync.test.ts case. unregister_with_fd_impl takes main's disarmed_only logic and resolves the watcher fd through loop_mut(vm). On the merged debug build: spawnsync-isolated-event-loop.test.ts (the new case passes; the two #40508 cases still need more than the 5s default on a local debug build), spawnSync.test.ts (18 pass, 2 skip), filesink.test.ts (68 pass), spawn-pipe-start-error.test.ts (9 pass, 1 skip), spawn.test.ts (144 pass, 8 skip), isolation.test.ts (41 pass), node-timers.test.ts (24 pass), 10887.test.ts, spawn-streaming-stdin.test.ts and spawnsync-no-microtask-drain.test.ts. child_process.test.ts has 65 pass and 2 fail: extra stdio pipes are not double-closed on GC needs 6.4s on this debug build and passes with a longer timeout, and should allow us to spawn in the default shell fails because $SHELL is empty in the test environment of this container. cargo check --workspace passes for x86_64-pc-windows-msvc and aarch64-apple-darwin.

Stand-in for the user-visible face: 30 files under bun test --isolate --smol with stderr on a pipe. Each file touches process.stderr and process.stdout, allocates 20000 small objects, then calls Bun.spawnSync(["/bin/true"]) 30 times and counts the fds on the stderr pipe around each call. Release build of main b52d51348: 11 of 20 runs were killed at the 25s limit after (fail) t04 [5019.48ms] or (fail) t08. Release build of this branch merged with main: 0 of 20 hung and 0 failed. In 4 of those 20 runs, 13 sinks of earlier files were finalized inside a Bun.spawnSync call: the fd count and the main loop's numPolls both dropped across the call. A second variant reads, at the end of every file, the main loop's numPolls minus the fds on the stderr pipe. On main the value moves from -2 to 0 and 1 in the runs that hang. On this branch it is -2 in all 600 files of 20 runs, and in 8 of those runs sinks were finalized inside a call. The same stand-in on the debug build never finalizes a sink inside a call, so it shows nothing there.

The script from #34069: spawnsync-spin.test.mjs from the comment of 2026-09-19, unchanged, in the shape its runner uses (8 copies of the file, bun test --isolate --timeout 0, stdout on a pipe, 3 such processes at once, a process that is silent for 40s counts as wedged). Release builds, blocks of 6 process-runs that alternate between the two binaries, Linux x64. main 367d939d9: 15 of 72 wedged, each in state R with its CPU time rising 5s per 5s and one sh child a zombie. This branch (2cd5c710be): 0 of 72. bun 1.4.2: 5 of 6.

Why BUN_JSC_slowPathAllocsBetweenGCs: a finalizer runs inside spawnSync only when JSC sweeps at one of the allocations spawnSync makes after the wait. LocalAllocator::doTestCollectionsIfNeeded runs collectNow(Sync, Full), which sweeps synchronously, every N slow-path allocations. Two details make the window deterministic. The first spawnSync call of a process creates lazy structures and Bun.spawnSync itself, about 11 slow-path allocations before the loop swap, so the fixture warms spawnSync up while the writers are still reachable; the next call then makes no allocation before the swap (measured with markers in prepare/cleanup and BUN_JSC_logGC=1: 0 GCs before the swap, 3 inside at N=5) and the first forced GC after the drop lands inside it. The probe child sleeps 300ms so the parent has to poll for its exit; with echo the child could exit before the polls were registered and the unfixed build returned without spinning. N=5 spins on the unfixed release, canary and main (with #40508) builds in 105 of 105 runs with the allocation phase shifted by 0 to 150 objects of five kinds. N=1 also works but makes a debug build spend 12s in startup GCs; N=5 costs about 1s. An earlier revision used a debug-only run_gc(true) hook behind BUN_INTERNAL_SPAWN_SYNC_GC, which no CI lane exercised; the hook is gone.

Why a bit and not a pointer: an earlier revision stored the loop pointer on every FilePoll, KeepAlive and timer counter and dropped the ctx argument from unref. Review asked for a 1-bit identifier instead, since only two loops exist per thread. The bit also leaves the existing ref_(ctx)/unref(ctx) call sites unchanged.

Isolated-loop lifetime: every poll that records SpawnSyncLoop is created and freed inside the same spawnSync call. The stdio readers and the process poller start after prepare() swaps the handle in, SubprocessT::finalize runs before the cleanup() scopeguard swaps it back, and the sync Subprocess never gets a JS wrapper. loop_for asserts this in debug builds, and KeepAlive::unref_on_next_tick (deferred through pending_unref_counter, drained on the thread's loop) asserts its ref was not taken on the isolated loop. Timer refs are never taken during spawnSync, since no JS runs there, but a finalizer can release one, so the timer counters record the bit too.

Windows: VirtualMachine::uws_loop() returns Loop::get() unconditionally there, so is_spawn_sync_loop() is always false and every release resolves to the same loop as before. The Windows FilePoll is unchanged.

Why the timer takes the EventLoopCtx too: timer::All first carried its own copy of the spawnSync predicate and release path over a raw *mut Loop, without the release-after-return debug assert. It now records the bit from EventLoopCtx::is_spawn_sync_loop and releases through loop_unref_for, like FilePoll and KeepAlive. The four callers (timer_object_internals.rs, subprocess.rs, two in dns.rs) already had the per-thread ctx in reach.

Why the fixture uses WeakRefs and not bun:internal-for-testing or node:fs: fileSinkInternals.liveCount() or an fd listing would give exact counts, but loading either module under slowPathAllocsBetweenGCs=5 costs a debug build 5s to 15s (1s without). A WeakRef made before a job boundary (await Bun.sleep(0)) does not keep its target alive afterwards (JSC currentWeakRefVersion), and deref() after the probe call returns undefined exactly when the writer was collected during the call. The writers hang off globalThis until the drop: a module binding that is only written after the await is dead at the suspension point, and the writers were collected in the warm-up call instead (measured: 0 of 3 alive after the warm-up). With the global, an unfixed build spins with collectedDuringCall: 3 for every padding tried before the warm-up (0, 2, 5 and 40 objects). What the check cannot see is a GC that runs after the drop but before the loop swap: an allocation between the drop and the call (the comment forbids code there), or a slow-path allocation in spawnSync's argument parsing, of which there is none today (N=1 also spins).

Scope: the drift requires a teardown under the isolated loop. GC finalizers of FileSink, Subprocess, FetchTasklet, MessagePort and similar are the only code that runs there since #38883 moved the bun:test timeout callback out of the wait. EventLoopCtx::loop_unref, loop_add_active and loop_sub_active are gone on POSIX (the last two remain for the Windows FilePoll). EventLoopCtx::platform_event_loop and PosixStreamingWriterParent::loop_ (with the uws_loop arm of impl_streaming_writer_parent!) lost their last callers and are removed.

Fail-before was checked against a debug build of the merge base (6d13a3a9bf) in a second worktree, the same shape as the gate's stash run: all three tests this branch had then fail there (spin, EEXIST, err.is_none() abort) and pass on this branch. The third one left with the merge above.

Suites run on the debug build: spawn.test.ts (140 pass, 8 skip), spawnSync.test.ts (17 pass, 2 skip), filesink.test.ts, spawn-streaming-stdin.test.ts, spawn-stdin-readable-stream-sync.test.ts, spawn-stdin-pipe-fd-leak.test.ts, node-timers.test.ts, dns-resolver-concurrent-timeout.test.ts, dns-lookup-keepalive.test.ts, dns-interleave.test.ts, spawn-kill-signal.test.ts, spawn-maxbuf.test.ts, spawn-signal.test.ts, spawnsync-isolated-event-loop.test.ts, spawnsync-no-microtask-drain.test.ts, spawn-streaming-stdout.test.ts, spawn-streaming-stdin.test.ts, spawn-stdin-readable-stream-sync.test.ts, exit-code.test.ts, spawn-kill-signal.test.ts, spawn-many-teardown.test.ts, 10887.test.ts, filesink.test.ts and bun-write.test.js (123 pass), child_process.test.ts and child_process-node.test.js, node-timers.test.ts. resolve-dns.test.ts needs network access this container lacks (getaddrinfo ETIMEOUT). cargo check passes for linux-gnu x64, darwin x64 and windows x64. Rebased onto main after #40508: the only conflicts were the test file's imports. #40508's two new tests in the same file pass on this branch (OK, exit 0) but need more than the 5s default timeout on a local debug build (8s and 11s under collectContinuously). The same applies to extra stdio pipes are not double-closed on GC in child_process.test.ts (passes alone in 9s).