Honor Bun.file().slice() bounds in Bun.write
Fixes #4930
Problem
Bun.write(dest, Bun.file(p).slice(a, b))copies the whole source.Bun.write("key.bin", Bun.file("/dev/urandom").slice(0, 16))writes until the disk is full, then rejects withENOSPC.write_file_with_source_destination(src/runtime/webcore/Blob.rs:4416) givesCopyFileonly the destination's offset and size, which.sizeorexists()can have cached (#4930).
Fix
CopyFiletakes the source window. It seeks a regular source to the window's start, and every copy path stops at the window's end.Blob::size_is_explicitmarks a size thatslice()set.BlobExt::file_windowreads it for the source and the destination.- The stdin half of this PR moved out. It read the whole window into memory. Draft #44301 is on top of this PR and sends the window in chunks.
- Verified on Linux:
test/js/bun/io/bun-write.test.js(35 cases fail on 1.4.3). Self-reviewed twice (Notes).
Background
- A
Blobis a view (offset,size) onto a refcounted store.slice()changes only the view. - A file Blob's
sizeisMAX_SIZEuntil something stats the file. - Considered a cut that passes only the window's length.
slice(5, 10)then copies bytes 0 to 4.
Downsides
- Behavior change: a program that passes a sliced
Bun.file()toBun.writenow gets only the window. Bun.file(p).slice(-5)of a file that nothing has stat'd copies nothing (main: the whole file). #41257 fixesslice(). Merge it first, or with this PR.- Cost, measured by a pre-merge check at 8206895c3a:
Bun.write(path, Bun.file(small))8,607 to 8,657 instructions,slice()1,098 to 1,106.bun-linux-x64.zipgrows by 9,864 bytes.
Notes
Scope change (2026-09-29). Until 8206895c3a this PR also changed spawn stdin: it read the whole window of a sliced file into the parent with pread before the child started. A pre-merge check measured +537 MB of RSS and 616 ms on the JS thread for a 512 MiB window, and a 3 GiB window in a 2 GB container killed the parent. A design review ranked that buffer last of six designs. Its pick is a handle that Bun opens before the child exists, and a pump that sends one chunk of 64 KiB per writable event. That work is draft #44301, one commit on this branch. A later self-review said that a person must choose between the buffer and the chunk sender, so #44301 waits for that choice. src/runtime/api/bun/spawn/stdio.rs and test/js/bun/spawn/spawn.test.ts are as on main in this PR.
What changed in this head against 8206895c3a.
- The stdin half is gone (see above).
- A window of a file descriptor starts at the window's offset, also at offset 0. Before,
Bun.file(fd).slice(0, 5)on a descriptor at byte 4 copied bytes 4 to 8. A test coversslice(0, 5)andslice(0, size). - The stream of a file carries the flag.
new Response(file.stream())of a file whose.sizewas read is no window, so the copy takes the whole file. A test covers it. - The Windows run without
uv_fs_copyfileskips the window block. A window never takesuv_fs_copyfile.
What changed in e5bdfd6c81.
- macOS: a clone has the mode of the source, so
truncate(2)fails on a clone of a read-only file. The copy now checks the result. If it fails, the copy removes the clone and copies the bytes of the window. A test covers a source with mode 0444. - The capped copy loop and
NodeFS::copy_file_using_read_write_loopare one loop innode_fs.rs. It reads until EOF or until the cap. The net change is 51 lines less. - The socket test compares the bytes that the copy wrote.
What changed in 1852be9cbc. deserialize() bounds the offset and the size of a file record by MAX_SIZE. The offset comes from bytes that the caller controls, and a file store of unknown size had no bound for it. On Windows the copy loop casts the offset to i64, so an offset above i64::MAX stopped the process. A new case covers offsets of 2^52 + 123, 2^63 + 5 and 2^64 - 1.
What changed in d36bbf6fa5. FileReader keeps the flag inside max_size (Option<MaxSize>), because the flag has a meaning only when max_size is set. The mordant check asked for that.
What changed in 457b6e958d and 704fa307fc. A window of the destination itself on Windows (Bun.write(p, Bun.file(p).slice(10, 20))). A window takes the read and write loop there, and the loop opened the destination with O_TRUNC, so the file was empty before the first read. On main uv_fs_copyfile does nothing for one file, so that call left the file whole. The loop now compares the stat of the destination path with the fstat of the source (volume and file index, as uv_fs_copyfile does). For one file it opens the destination without O_TRUNC, moves the window to the start of the file and cuts the rest. For two files nothing changes. The first form of this fix (457b6e958d) called fstat on the write-only handle of the destination. That call fails on Windows, so CI showed the file empty (Buildkite 121911), and 704fa307fc replaced it. A window of a directory now rejects with EISDIR before the destination is opened. Five cases cover both on Windows only. I did not run them by hand. They pass on both Windows lanes in CI (Buildkite 121926).
The same file on POSIX, with #43388. On POSIX main empties a file that is copied onto itself (#20462), and this PR does not change that. #43388 fixes it. At its head 20ea576758, do_open_file stats the destination path before each open. When the path names the regular file of the source, it opens nothing, and run_async takes a same-file arm that does nothing. That is wrong for a window of the file: the result must be the window. The PR that merges second must add the case to that arm. For a window that is not the whole file, the arm must open the destination without O_TRUNC, seek the source to the window, run the capped loop (the reads stay ahead of the writes) and ftruncate the file to the bytes copied. A whole file stays a no-op. The row a slice of the file as the source gives the same result as for another destination of #43388 must fail on a merge of the two PRs until that case exists. The session of #43388 merged the two branches in a scratch tree (two conflicts, both sides stay) and did not build that merge. The four same-file cases of this PR, which run on Windows only today, then run on POSIX too.
Merge order. #41257 makes slice() count a negative index back from the file's real end. Without it, slice(-5) of a file that nothing has stat'd has an offset near MAX_SIZE, so this PR copies nothing. A pre-merge check got the last 5 bytes with both PRs, in 4 of 4 cases. The two merge clean in both orders.
Measurements. The pre-merge check ran a release build of 8206895c3a against main a4f1429148 (instructions per call, one pinned core, 15 rounds): Bun.write(path, Bun.file(small)) 8,607 to 8,657 (+0.6 %), .size of a known handle 62 to 64, slice() of any Blob 1,098 to 1,106. Its storm of 12,000 random operations gave 0 wrong bytes and 0 ASan reports. Whole files take the same syscalls as on main.
Counts from /proc/self/io for a 256 MiB source (release 1.4.3 against a debug build of this PR):
1.4.3 this PR
fallback copy, whole file 39 reads, 33 writes 37 reads, 33 writes
fallback copy, window of 199 MiB whole file copied 28 reads, 26 writes
copy_file_range copy, whole file 4 reads, 3 writes 5 reads, 3 writes
A window that starts past byte 0, or a window of a caller's descriptor, adds one lseek. An unsliced copy adds no syscall. strace is not on this machine.
Binary size (zip sizes from the CI artifact list, base build 121139 against build 121190 of 8206895c3a, which still had the stdin half): bun-linux-x64.zip 37,920,840 to 37,930,704 (+9,864), bun-linux-aarch64.zip +11,696, bun-windows-x64.zip +14,439, bun-darwin-aarch64.zip +81. The artifacts cannot be downloaded from this machine, so size was not run on the binaries.
The small cut. Main already takes a length from the destination: Bun.write(Bun.file(dst).slice(0, 5), Bun.file(src).slice(5, 10)) writes abcde on 1.4.3. So a cut that passes only the source's length stops the copy of /dev/urandom, and it makes slice(5, 10) of a regular file copy the wrong 5 bytes.
Probe on a 26-byte file a..z, f = Bun.file(p):
1.4.3 / main this PR
Bun.write(dst, f.slice(10, 20)) 26, whole file 10, "klmnopqrst"
Bun.write(dst, new Response(f.slice(10,20))) 26, whole file 10
Bun.write(dst, new Response(slice.stream())) 26, whole file 10
Bun.write(dst, f.slice(0, 5)), (3), (20,100) whole file the window
Bun.write(dst, 8 MiB.slice(1 MiB+3, 5 MiB-7)) 8 MiB 4194294 bytes, identical
Bun.write("key.bin", /dev/urandom.slice(0,16)) until ENOSPC 16
f.exists(), create p, f.size / f.text() 0 / "" 26 / the file
d = 3-byte file, d.size, Bun.write(d, f) 3, "abc" 26, the file
d.size, Bun.write(d.slice(0, d.size), f) 3 3
Sources that are not regular files. A socket, a pipe or a character device has no position. Bun.write copies the window's size from where the stream is. The two character device tests run under ulimit -f and ulimit -t, so a build without the cap cannot fill a disk or run without end.
Replaces older PRs. #31515 (Bun.write window, its review asked to seek only regular files and to test a source that cannot seek), #37877 (the destination's cached size, its size_is_explicit flag and its 17 test cases) and #33715 (the byte count, which #42066 fixed on main since). They are closed with a comment that points here. #31931 (stdin) is replaced by the stack on this PR.
Overlap with open PRs. A maintainer can pick one shape.
- #43910 (draft): another design for the cached size, for reads too. It conflicts with this PR in
Blob.rs. - #39763: the same
resolve_sizeand.sizechange, plus a stat on everyexists(). This PR carries only that hunk, because the window check needs it. - #37856: the same
read_write_fallbackchange, for destination windows. - #42819 removes libuv on Windows. It rewrites the Windows half of
copy_file.rs.
Not changed here.
- A destination that starts past byte 0 (
Bun.write(Bun.file(dst).slice(5, 10), src)) resolves 0 and leaves the destination empty, as on main. The destination offset is applied on no copy path. It is reported as a separate bug. - On POSIX
Bun.write(p, Bun.file(p))empties the file (#20462). #43388 is open for it. See "The same file on POSIX" above. f.slice(20, 100).sizeis 80. The size of a file Blob that nothing has stat'd is lazy.Bun.file(fd).slice(0, 5).text()on a descriptor that is not at byte 0 reads from the descriptor's position. The read path seeks only for an offset above 0.Bun.serveanswers 206 withContent-Rangefor a sliced file.docs/runtime/http/routing.mdxdocuments that.- The shell turns a path-backed file Blob in a template into its path, so
cat < ${f.slice(1, 3)}reads the whole file.
Reviews. Two self-reviews and the bot reviews found these, all fixed and covered by tests: a failed stat cached a size of 0, so exists() on a missing file and a later Bun.write copied nothing. A size comparison stood in for the flag and was wrong both ways. On macOS and FreeBSD a window took a copy path that reads the whole source. Not taken: the merge order against #40221, and a 1.4.x release for this change. A maintainer decides those.
Test runs on this head.
- Linux, debug build: the two
Bun.writeblocks pass (46 cases) when they run alone. On 1.4.3 the same blocks fail 35 cases. - Linux, for the shared copy loop:
cp.test.tsand thecopyFilecases offs.test.tspass. A 256 MiB copy through the fallback takes 36 to 39readcalls and 33writecalls. - Linux, also green:
structured-clone-blob-file,blob,blob-write,bun-file-exists,bun-file, and the stream cases ofbody. - Linux, full files: the machine had a load average above 300 on 16 cores. Cases that start a child time out at 5 s, also cases that this PR does not touch, and also on a base build.
- Windows and macOS in CI: Buildkite build 121567 passed for 1852be9cbc.
- Windows: not run by hand on this head. The machines were off. At 8206895c3a all of
bun-write.test.jspassed on a Windows x64 debug build. bun run rust:check-allfor Windows, macOS, FreeBSD and Linux: no errors, and no warning in the changed files.- macOS was not run. The macOS lanes in CI are the first run of the clone fallback.
no test proof · iteration 2 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/io/bun-write.test.js