fetch/S3: replace ResumableSink with proper JSSinks
Using a node:stream Readable as a fetch() request body wedges the process at 100% CPU when the Readable's _read() pushes synchronously: the event loop never runs again, destroy() never fires, and RSS climbs until OOM.
Reproduction
import net from 'node:net';
import { Readable } from 'node:stream';
const srv = net.createServer(s => s.on('data', () => {}));
await new Promise(r => srv.listen(0, '127.0.0.1', r));
const chunk = Buffer.alloc(16 * 1024, 0x47);
const rd = new Readable({ read() { this.push(chunk); } });
setTimeout(() => rd.destroy(new Error('upstream went away')), 100);
await fetch(`http://127.0.0.1:${srv.address().port}/`, { method: 'POST', body: rd, duplex: 'half' });
Node/undici rejects after ~120ms. Bun spins forever; gdb on the main thread shows ArrayBufferSink.write re-entered from JS indefinitely.
Cause
ResumableSink (the sink fetch/S3 used for request-body streaming) was a parallel implementation of what the JsSinkType/generate-jssink.ts/assignToStream machinery already does for every other sink. Its pump (assignStreamIntoResumableSink) materialised a DirectPending stream into an ArrayBufferSink-backed JSDirectStreamController whose write returns bytes-written (>= 0), so driveAsyncIterator's only backpressure check (wrote < 0) never fires and a synchronous-yielding iterator spins forever.
Fix
Delete ResumableSink and fold its two consumers into the JSSink family:
FetchRequestBodySinkis a newJsSinkTypethat wraps theFetchTaskletback-ref.write_bytescarries the existing chunked-framing +ThreadSafeStreamBufferwrite, returningWritable::Backpressureonce the buffer reacheshighWaterMark;flush_from_js(true)returns a pending promise the HTTP-thread drain resolves, andSignal::ready()re-enters the pump.- S3
upload_stream()switches to the existingNetworkSink+assign_to_stream.NetworkSink::write_*now propagateMultiPartUpload's backpressure asWritable::Backpressure, andon_writablefiressignal.ready()on drain.
Both callers go through assign_to_stream, so a DirectPending body reaches readDirectStream, which hands the sink controller straight to pull(). driveAsyncIterator's existing wrote < 0 check fires, the pump suspends on flush(true)'s drain promise, and the event loop runs.
Removed: ResumableSink.rs, ResumableSink.classes.ts, JSResumableSinkPumpOperation, assignStreamIntoResumableSink and its resumable* pump, ReadRequestKind::ResumableSinkPump, and the associated handler/structure registrations. Net -460 lines.
Verification
$ USE_SYSTEM_BUN=1 bun test test/js/web/fetch/fetch.test.ts -t 'does not wedge on Readable.destroy'
(fail) ... { exited: "timeout", stdout: "" }
$ bun bd test test/js/web/fetch/fetch.test.ts -t 'does not wedge on Readable.destroy'
(pass) ... stdout: "rejected:upstream went away ticks:N"
test/js/bun/http/async-iterator-stream.test.ts (91), test/js/web/fetch/body-stream.test.ts (9086), test/js/web/fetch/body.test.ts + fetch-abort-stream-body.test.ts (451), and the fetch.test.ts duplex block pass.
Benchmark
Release build, 50 concurrent operations x 10 MB each (500 MB total per scenario), median of 3 runs, linux x64. Each scenario runs in a fresh subprocess; RSS delta is peak RSS minus baseline sampled every 50 ms; CPU is process.cpuUsage() user+sys for the worker process (the echo server runs in a separate subprocess and is not counted).
| Scenario | Build | Elapsed | Peak RSS delta | CPU (u+s) |
|---|---|---|---|---|
| proxy-response (ByteStream -> HTTPResponseSink) | this PR | 1.30 s | 543 MB | 1.67 s |
| main | 1.07 s | 170 MB | 1.62 s | |
| upload-proxy (req.body -> fetch body) | this PR | 1.45 s | 795 MB | 1.42 s |
| main | 1.29 s | 55 MB | 1.31 s | |
| spawn-pipe (fetch.body -> spawn stdin) | this PR | 1.21 s | 502 MB | 1.68 s |
| main | fails (EPIPE in readStreamIntoSink) | - | - | |
| file-upload (Bun.file().stream() -> fetch body) | this PR | 1.04 s | 352 MB | 839 ms |
| main | 1.14 s | 238 MB | 1.01 s | |
| js-stream-upload (ReadableStream pull -> fetch body) | this PR | 1.05 s | 369 MB | 856 ms |
| main | 1.06 s | 164 MB | 766 ms | |
| node-readable (Readable.from(async gen) -> fetch body) | this PR | 1.08 s | 301 MB | 922 ms |
| main | 757 ms | 172 MB | 1.28 s |
Builds: this PR fd8b82482, main 1.3.14+0d9b296af.
- spawn-pipe on main fails outright: piping a
fetch()response body intoBun.spawnstdin surfaces unhandledEPIPEfromreadStreamIntoSinkand the operation never completes at 50x concurrency. This PR handles it cleanly. - node-readable on main burns ~1.7x CPU per wall-clock second (1.28 s CPU in 757 ms wall = ~170%), which is the async-iterator pump spinning without yielding. This PR brings it to ~85% (922 ms CPU in 1.08 s wall), i.e. proper backpressure with one core doing useful work.
- file-upload is ~9% faster wall-clock and ~17% less CPU on this PR (FileReader -> FetchRequestBodySink native path).
- proxy-response / upload-proxy show higher peak RSS on this PR. At 50x concurrent 10 MB transfers the new
SinkHandlepath keeps more data in flight (eachreq.body -> fetch bodyholds its chunked buffer until the echo response arrives), whereas main'sResumableSinkpath applied tighter backpressure earlier. Throughput is within ~15% either way and memory stays bounded below the 500 MB payload total per iteration, but this is a known trade-off worth noting.
no test proof · iteration 17 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/http/serve.test.ts test/js/bun/spawn/spawn-stdin-readable-stream.test.ts test/js/web/fetch/fetch.test.ts