Bun.write(path, Bun.file(fd)): trim the preallocated destination to the bytes copied

Closed
RO
1mo ago
+41-0

Problem

  • Bun.write(path, Bun.file(fd)) on Linux leaves the destination at the source's full st_size when the fd is positioned past 0. The copied bytes are followed by a zero-filled tail. Example: a 4 MiB file read to 3 MiB, then copied, resolves 1048576 but dst is 4 MiB.
  • The cause is in src/runtime/webcore/blob/copy_file.rs. run_async preallocates the destination to st_size with fallocate (line 809). do_copy_file_range then copies from the fd's current position and stops at EOF. Nothing trims the destination. The strace is fallocate(dst, st_size), copy_file_range = 1048576, copy_file_range = 0, close.

Fix

  • After the copy_file_range / sendfile / splice loop, if Bun opened the destination (a path, so O_TRUNC) and total_written < max_length, ftruncate the destination to total_written.
  • This is the same reconciliation the read/write fallback already does in read_write_fallback (ftruncate(dest_fd, *total)). The kernel-copy path did not have it.
  • A destination given as an fd is not preallocated and is not touched. An unknown-size source (FIFO, char device) skips the check.
  • Verified: test/js/bun/io/bun-write.test.js (new test, fails on bun 1.4.3 with size: 2162688 instead of 65536). Also test/js/web/fetch/blob-write.test.ts.

Background

  • Bun.write(dst, Bun.file(src)) runs CopyFile on the work pool. When the source is a regular file it fstats it, clamps max_length to st_size, and on Linux preallocates a path destination above 2 MiB so the copy does not fragment.
  • copy_file_range(2) with null offsets uses and advances each fd's file position. A user fd that was already read from starts the copy in the middle of the file.
Notes

Repro under bun 1.4.3:

const fd = fs.openSync(src, "r"); // src is 4 MiB
fs.readSync(fd, Buffer.alloc(3 << 20), 0, 3 << 20, null);
await Bun.write(dst, Bun.file(fd)); // resolves 1048576, dst is 4194304 bytes

The should work when copyFileRange is not available > on large files test in the same file is slow under the debug ASAN build (a 256 MiB synchronous fill) and timed out once while the file ran concurrently. It passed on the next run and is not touched by this change.


no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/io/bun-write.test.js