Honor Bun.file().slice() bounds in Bun.write

RO
1d ago
+807-200

Fixes #4930

Problem

  • Bun.write(dest, Bun.file(p).slice(a, b)) copies the whole source.
  • Bun.write("key.bin", Bun.file("/dev/urandom").slice(0, 16)) writes until the disk is full, then rejects with ENOSPC.
  • write_file_with_source_destination (src/runtime/webcore/Blob.rs:4416) gives CopyFile only the destination's offset and size, which .size or exists() can have cached (#4930).

Fix

  • CopyFile takes the source window. It seeks a regular source to the window's start, and every copy path stops at the window's end.
  • Blob::size_is_explicit marks a size that slice() set. BlobExt::file_window reads it for the source and the destination.
  • The stdin half of this PR moved out. It read the whole window into memory. Draft #44301 is on top of this PR and sends the window in chunks.
  • Verified on Linux: test/js/bun/io/bun-write.test.js (35 cases fail on 1.4.3). Self-reviewed twice (Notes).

Background

  • A Blob is a view (offset, size) onto a refcounted store. slice() changes only the view.
  • A file Blob's size is MAX_SIZE until something stats the file.
  • Considered a cut that passes only the window's length. slice(5, 10) then copies bytes 0 to 4.

Downsides

  • Behavior change: a program that passes a sliced Bun.file() to Bun.write now gets only the window.
  • Bun.file(p).slice(-5) of a file that nothing has stat'd copies nothing (main: the whole file). #41257 fixes slice(). Merge it first, or with this PR.
  • Cost, measured by a pre-merge check at 8206895c3a: Bun.write(path, Bun.file(small)) 8,607 to 8,657 instructions, slice() 1,098 to 1,106. bun-linux-x64.zip grows by 9,864 bytes.
Notes

Scope change (2026-09-29). Until 8206895c3a this PR also changed spawn stdin: it read the whole window of a sliced file into the parent with pread before the child started. A pre-merge check measured +537 MB of RSS and 616 ms on the JS thread for a 512 MiB window, and a 3 GiB window in a 2 GB container killed the parent. A design review ranked that buffer last of six designs. Its pick is a handle that Bun opens before the child exists, and a pump that sends one chunk of 64 KiB per writable event. That work is draft #44301, one commit on this branch. A later self-review said that a person must choose between the buffer and the chunk sender, so #44301 waits for that choice. src/runtime/api/bun/spawn/stdio.rs and test/js/bun/spawn/spawn.test.ts are as on main in this PR.

What changed in this head against 8206895c3a.

  • The stdin half is gone (see above).
  • A window of a file descriptor starts at the window's offset, also at offset 0. Before, Bun.file(fd).slice(0, 5) on a descriptor at byte 4 copied bytes 4 to 8. A test covers slice(0, 5) and slice(0, size).
  • The stream of a file carries the flag. new Response(file.stream()) of a file whose .size was read is no window, so the copy takes the whole file. A test covers it.
  • The Windows run without uv_fs_copyfile skips the window block. A window never takes uv_fs_copyfile.

What changed in e5bdfd6c81.

  • macOS: a clone has the mode of the source, so truncate(2) fails on a clone of a read-only file. The copy now checks the result. If it fails, the copy removes the clone and copies the bytes of the window. A test covers a source with mode 0444.
  • The capped copy loop and NodeFS::copy_file_using_read_write_loop are one loop in node_fs.rs. It reads until EOF or until the cap. The net change is 51 lines less.
  • The socket test compares the bytes that the copy wrote.

What changed in 1852be9cbc. deserialize() bounds the offset and the size of a file record by MAX_SIZE. The offset comes from bytes that the caller controls, and a file store of unknown size had no bound for it. On Windows the copy loop casts the offset to i64, so an offset above i64::MAX stopped the process. A new case covers offsets of 2^52 + 123, 2^63 + 5 and 2^64 - 1.

What changed in d36bbf6fa5. FileReader keeps the flag inside max_size (Option<MaxSize>), because the flag has a meaning only when max_size is set. The mordant check asked for that.

What changed in 457b6e958d and 704fa307fc. A window of the destination itself on Windows (Bun.write(p, Bun.file(p).slice(10, 20))). A window takes the read and write loop there, and the loop opened the destination with O_TRUNC, so the file was empty before the first read. On main uv_fs_copyfile does nothing for one file, so that call left the file whole. The loop now compares the stat of the destination path with the fstat of the source (volume and file index, as uv_fs_copyfile does). For one file it opens the destination without O_TRUNC, moves the window to the start of the file and cuts the rest. For two files nothing changes. The first form of this fix (457b6e958d) called fstat on the write-only handle of the destination. That call fails on Windows, so CI showed the file empty (Buildkite 121911), and 704fa307fc replaced it. A window of a directory now rejects with EISDIR before the destination is opened. Five cases cover both on Windows only. I did not run them by hand. They pass on both Windows lanes in CI (Buildkite 121926).

The same file on POSIX, with #43388. On POSIX main empties a file that is copied onto itself (#20462), and this PR does not change that. #43388 fixes it. At its head 20ea576758, do_open_file stats the destination path before each open. When the path names the regular file of the source, it opens nothing, and run_async takes a same-file arm that does nothing. That is wrong for a window of the file: the result must be the window. The PR that merges second must add the case to that arm. For a window that is not the whole file, the arm must open the destination without O_TRUNC, seek the source to the window, run the capped loop (the reads stay ahead of the writes) and ftruncate the file to the bytes copied. A whole file stays a no-op. The row a slice of the file as the source gives the same result as for another destination of #43388 must fail on a merge of the two PRs until that case exists. The session of #43388 merged the two branches in a scratch tree (two conflicts, both sides stay) and did not build that merge. The four same-file cases of this PR, which run on Windows only today, then run on POSIX too.

Merge order. #41257 makes slice() count a negative index back from the file's real end. Without it, slice(-5) of a file that nothing has stat'd has an offset near MAX_SIZE, so this PR copies nothing. A pre-merge check got the last 5 bytes with both PRs, in 4 of 4 cases. The two merge clean in both orders.

Measurements. The pre-merge check ran a release build of 8206895c3a against main a4f1429148 (instructions per call, one pinned core, 15 rounds): Bun.write(path, Bun.file(small)) 8,607 to 8,657 (+0.6 %), .size of a known handle 62 to 64, slice() of any Blob 1,098 to 1,106. Its storm of 12,000 random operations gave 0 wrong bytes and 0 ASan reports. Whole files take the same syscalls as on main.

Counts from /proc/self/io for a 256 MiB source (release 1.4.3 against a debug build of this PR):

                                     1.4.3                this PR
fallback copy, whole file            39 reads, 33 writes  37 reads, 33 writes
fallback copy, window of 199 MiB     whole file copied    28 reads, 26 writes
copy_file_range copy, whole file     4 reads, 3 writes    5 reads, 3 writes

A window that starts past byte 0, or a window of a caller's descriptor, adds one lseek. An unsliced copy adds no syscall. strace is not on this machine.

Binary size (zip sizes from the CI artifact list, base build 121139 against build 121190 of 8206895c3a, which still had the stdin half): bun-linux-x64.zip 37,920,840 to 37,930,704 (+9,864), bun-linux-aarch64.zip +11,696, bun-windows-x64.zip +14,439, bun-darwin-aarch64.zip +81. The artifacts cannot be downloaded from this machine, so size was not run on the binaries.

The small cut. Main already takes a length from the destination: Bun.write(Bun.file(dst).slice(0, 5), Bun.file(src).slice(5, 10)) writes abcde on 1.4.3. So a cut that passes only the source's length stops the copy of /dev/urandom, and it makes slice(5, 10) of a regular file copy the wrong 5 bytes.

Probe on a 26-byte file a..z, f = Bun.file(p):

                                              1.4.3 / main         this PR
Bun.write(dst, f.slice(10, 20))               26, whole file       10, "klmnopqrst"
Bun.write(dst, new Response(f.slice(10,20)))  26, whole file       10
Bun.write(dst, new Response(slice.stream()))  26, whole file       10
Bun.write(dst, f.slice(0, 5)), (3), (20,100)  whole file           the window
Bun.write(dst, 8 MiB.slice(1 MiB+3, 5 MiB-7)) 8 MiB                4194294 bytes, identical
Bun.write("key.bin", /dev/urandom.slice(0,16)) until ENOSPC        16
f.exists(), create p, f.size / f.text()       0 / ""               26 / the file
d = 3-byte file, d.size, Bun.write(d, f)      3, "abc"             26, the file
d.size, Bun.write(d.slice(0, d.size), f)      3                    3

Sources that are not regular files. A socket, a pipe or a character device has no position. Bun.write copies the window's size from where the stream is. The two character device tests run under ulimit -f and ulimit -t, so a build without the cap cannot fill a disk or run without end.

Replaces older PRs. #31515 (Bun.write window, its review asked to seek only regular files and to test a source that cannot seek), #37877 (the destination's cached size, its size_is_explicit flag and its 17 test cases) and #33715 (the byte count, which #42066 fixed on main since). They are closed with a comment that points here. #31931 (stdin) is replaced by the stack on this PR.

Overlap with open PRs. A maintainer can pick one shape.

  • #43910 (draft): another design for the cached size, for reads too. It conflicts with this PR in Blob.rs.
  • #39763: the same resolve_size and .size change, plus a stat on every exists(). This PR carries only that hunk, because the window check needs it.
  • #37856: the same read_write_fallback change, for destination windows.
  • #42819 removes libuv on Windows. It rewrites the Windows half of copy_file.rs.

Not changed here.

  • A destination that starts past byte 0 (Bun.write(Bun.file(dst).slice(5, 10), src)) resolves 0 and leaves the destination empty, as on main. The destination offset is applied on no copy path. It is reported as a separate bug.
  • On POSIX Bun.write(p, Bun.file(p)) empties the file (#20462). #43388 is open for it. See "The same file on POSIX" above.
  • f.slice(20, 100).size is 80. The size of a file Blob that nothing has stat'd is lazy.
  • Bun.file(fd).slice(0, 5).text() on a descriptor that is not at byte 0 reads from the descriptor's position. The read path seeks only for an offset above 0.
  • Bun.serve answers 206 with Content-Range for a sliced file. docs/runtime/http/routing.mdx documents that.
  • The shell turns a path-backed file Blob in a template into its path, so cat < ${f.slice(1, 3)} reads the whole file.

Reviews. Two self-reviews and the bot reviews found these, all fixed and covered by tests: a failed stat cached a size of 0, so exists() on a missing file and a later Bun.write copied nothing. A size comparison stood in for the flag and was wrong both ways. On macOS and FreeBSD a window took a copy path that reads the whole source. Not taken: the merge order against #40221, and a 1.4.x release for this change. A maintainer decides those.

Test runs on this head.

  • Linux, debug build: the two Bun.write blocks pass (46 cases) when they run alone. On 1.4.3 the same blocks fail 35 cases.
  • Linux, for the shared copy loop: cp.test.ts and the copyFile cases of fs.test.ts pass. A 256 MiB copy through the fallback takes 36 to 39 read calls and 33 write calls.
  • Linux, also green: structured-clone-blob-file, blob, blob-write, bun-file-exists, bun-file, and the stream cases of body.
  • Linux, full files: the machine had a load average above 300 on 16 cores. Cases that start a child time out at 5 s, also cases that this PR does not touch, and also on a base build.
  • Windows and macOS in CI: Buildkite build 121567 passed for 1852be9cbc.
  • Windows: not run by hand on this head. The machines were off. At 8206895c3a all of bun-write.test.js passed on a Windows x64 debug build.
  • bun run rust:check-all for Windows, macOS, FreeBSD and Linux: no errors, and no warning in the changed files.
  • macOS was not run. The macOS lanes in CI are the first run of the clone fallback.

no test proof · iteration 2 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/io/bun-write.test.js