test: stop the Bun.write copy_file_range fallback test from starving its concurrent siblings

RO
3mo ago
+27-69

Problem

  • On a debug or ASAN build, should work when copyFileRange is not available > on large files times out after 5000ms and 1 to 5 unrelated tests in the same file fail with it. Which siblings fail varies from run to run.
  • That test was fully synchronous: it built and hashed a 256 MB buffer, wrote it, copied it in a spawnSync child, then read and hashed the result, holding the event loop for 5 s or more in debug (3 s even in release).
  • The file is describe.concurrent, so a test that blocks the loop that long also burns the timeout of every sibling in flight.

Fix

  • The small and large cases become one it.each that spawns the child with Bun.spawn and awaits it, so nothing in the file blocks the loop for more than tens of milliseconds.
  • The large payload shrinks from 256 MB to 2 x 8 MiB + 100000 bytes. That still takes the heap-buffer path, loops more than once, ends on a partial pass, and preallocates the destination; 256 MB only repeated the same loop. The 4 KiB case still covers the stack-buffer path.
  • The check gets stricter: the child prints the byte count Bun.write resolved with, and the copy is compared byte for byte against random data. The old hash of a 1 KiB-periodic pattern could not detect a misplaced or repeated chunk.
  • Verification: test-only, nothing to prove against a source diff. Same debug build went from 1 to 5 failures per run to 50/50 in 3 runs; an LD_PRELOAD trace confirmed the fallback path is still the one exercised; a deliberately truncated copy fails the new assertion.

Background

  • Bun.write(dest, Bun.file(src)) on Linux normally uses copy_file_range; BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1 forces the read/write loop fallback, which is what this test covers.
  • That fallback uses a 64 KiB stack buffer for sources up to 1 MiB, a heap buffer capped at 8 MiB above that, and preallocates the destination above 2 MiB. The two payload sizes are chosen around those thresholds.
  • describe.concurrent in bun:test runs the tests in a block at the same time, so their per-test timeouts overlap and one blocking test can fail all of them.

no test proof · iteration 0 · Platform-specific test-only change; deferring to CI.

Original description

What

test/js/bun/io/bun-write.test.js is describe.concurrent on POSIX. Inside it, should work when copyFileRange is not available > on large files was fully synchronous: it filled a 64M-element Int32Array (256 MB) in a JS loop, Bun.hashed it, writeFileSync'd it, Bun.spawnSync'd a child bun to copy it with BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1, then readFileSync + Bun.hash again. On a debug/ASAN build that holds the event loop for 5 s or more, so the test times out and so does every concurrent sibling that happened to be in flight at the time. Which siblings fail varies from run to run.

Unmodified main (626034fda0), bun bd test test/js/bun/io/bun-write.test.js in a 12-core container:

(fail) Bun.write > should work when copyFileRange is not available > on large files [5094.54ms]
  ^ this test timed out after 5000ms.
(fail) Bun.write > Bun.write(Bun.file(fd), Bun.file(path)) does not truncate the fd > preserves bytes past the slice window in an r+ fd [6926.91ms]
(fail) Bun.write > Bun.write(Bun.file(fd), Bun.file(path)) does not truncate the fd > does not fallocate an O_APPEND fd for a source above the preallocate threshold [6633.41ms]
(fail) Bun.write > Bun.write(Bun.stdout, Bun.stdin) copies the whole pipe (> 4096 bytes) [6112.58ms]
(fail) Bun.write > Bun.write('output.html', HTMLRewriter.transform(Bun.file))) [5520.49ms]
 45 pass
 5 fail
Ran 50 tests across 1 file. [14.37s]

Even the release binary spends 3 s in this one test (USE_SYSTEM_BUN=1: on large files [3064.48ms], almost all of it sys time).

Fix

Both cases of that describe (on small files / on large files, names unchanged) are now one it.each that spawns the child with Bun.spawn and awaits it, so nothing in the file blocks the loop for more than a few tens of milliseconds.

The large payload shrinks from 256 MB to 2 * 8 MiB + 100000 bytes. With copy_file_range disabled, Bun.write(path, Bun.file(path)) goes through copy_file_using_read_write_loop (src/runtime/node/node_fs.rs), whose only size-dependent behaviour is: a 64 KiB stack buffer up to 1 MiB, above that a heap slab clamped to 8 MiB, plus the destination preallocate path above 2 MiB in blob/copy_file.rs. The new size still allocates the clamped slab, runs the loop more than once, ends on a partial pass and takes the preallocate path; 256 MB only added more iterations of the same loop. The 4 KiB case still covers the stack-buffer path.

The assertions got slightly stronger while here: the child prints the byte count Bun.write resolved with and the test checks it, and the copy is compared byte for byte (Buffer#equals) against a randomBytes payload instead of comparing hashes of a 1 KiB-periodic pattern, which could not tell a misplaced or repeated chunk from a correct copy. The bun-write-exdev-fixture.js fixture is still used by the POSIX_FADV_SEQUENTIAL test.

Verification

Test-only change, so there is no source diff to prove against. Before/after on the same debug build and container:

  • before: 3 runs, 1 to 5 failures each (the large-files test plus whichever siblings were in flight), file takes ~14 s
  • after: 3 runs, 50/50 pass each, file takes ~7.8 s; on large files is 340 to 530 ms in debug, 117 ms in release (was 3064 ms)
  • the fallback path is the one being exercised: running the child under an LD_PRELOAD shim shows fallocate(len=16877216), posix_fadvise(advice=SEQUENTIAL) (only issued by the read/write loop) and ftruncate(len=16877216) for the large case, and no fallocate for the small one
  • a deliberately corrupted copy (dest truncated by 100000 bytes after the write) fails the new assertion with size: 16777216 vs 16877216, identical: false