#43388 Bun.write: compare the destination with the source before the truncating open

Merged with main on 11-01. The branch has main (3f3a9ff87c) merged in, at f299d81f8b. There was one conflict: the harness import list of the test file, where main added isMacOS and this PR added isLinux. Both stay.

• The change is the same as before the merge. git patch-id of the diff to copy_file.rs and to src/sys/lib.rs gives the same id before and after.
• Ran again on Linux x64: the 22 new rows pass on a debug build. On the canary build of main (1.4.4-canary.1+3f3a9ff87), 18 of them fail and the 4 guards pass.
• The whole file: 129 pass with --timeout=90000. CI sets 90 s or more for each test. With the 5 s default, one whole-file run had one timeout: Bun.file(0) survives GC at 5.09 s. That test passes alone in 0.4 s. #47667 reports the same timeout on debug builds of main.
• bun run rust:check-all: 12 of 12 targets compile.
• Not measured again: the cost figures and the door map below. They are from 9f70da0741.

This PR changed its mechanism. The first version opened the destination without O_TRUNC, compared the two open fds, and then called ftruncate(fd, 0) (the order that libuv uses on Linux). A review finding showed that this order empties a file that the old code never emptied. On macOS and FreeBSD, open("/dev/fd/N") returns a dup of fd N and ignores O_TRUNC (XNU open1 goes to dupfdopen and returns before the truncate step). So Bun.write("/dev/stdout", Bun.file(p)) with stdout redirected to a file got a new ftruncate. This version has no ftruncate: the destination open is the same call as on main.

Designs that were weighed.

• An overwrite without O_TRUNC at open is not the same operation. On overlayfs the kernel copies the old file to the writable layer first (measured with the first version: 14,471,168 bytes written for a 7.2 MB file, 7,188,480 on main). Mountpoint for S3 documents that it replaces an object "only when the O_TRUNC flag is used at open time" (not run). The /dev/fd/N case above is the third.
• Apple copyfile(3), GNU cp and Python shutil.copyfile use the order of this PR: compare by path, then open with O_TRUNC.
• The last two designs close the window. They are a maintainer's choice, because they change every overwrite.

The window that stays. The destination path is resolved two times: by the stat, then by the open. If the path names the source only at the second time, the open empties the source, as main does for every copy of a file onto itself.

• Orders that reach it: a rename, a link or a symlink swap by another process, or process.chdir() on the JS thread while a copy with a relative destination is in the queue.
• Replayed with gdb on a release build of this PR (catch syscall newfstatat on the destination path, ln -f src dest at its return, continue): the file ends at 0 bytes and the promise fulfils 0, for 100,000 bytes and for 3 MiB, with the destination missing or another file at the time of the stat. That is the result main gives for Bun.write(p, Bun.file(p)).
• A copy over 2 MiB preallocates the destination. In the window that call would fill the emptied source with zeros, and the promise would fulfil the old size. So the copy compares the opened destination with the source once more before it preallocates. With the result of that fstat forced to an error in gdb, the 3 MiB file ends as 3,145,728 zero bytes and the promise fulfils 3145728. No test can reach this branch without a second actor.

What the no-op does.

• It fulfils with the size of the file. A slice() of the source does not change that on main, see the #41209 section.
• It applies the mode option with fchmod on the source fd.
• It does not open the file for writing. So a file without write permission now fulfils. On main a user who is not root gets EACCES from the open, and root gets an empty file.
• It writes nothing, so the times of the file do not change.
• The compare runs before every open of the destination, inside the loop of do_open_file. That open can fail, make the parent directories, and try again. For new/../f with no directory new, the path names the source only after that. This case costs a second stat.
• Only a regular source can be "the same file". /dev/full as the source and the destination is still copied (a test pins this). An inode number of 0 is not an identity.
• The identity of the source comes from fstat of the opened source fd on every platform. On macOS the job reused a stat of the source path that it took before the open, for the clonefile attempt. The path can name another file by the time of the open.

Door map. Linux, 100,000-byte file, release builds of main (9f70da0741) and of this PR:

Other results that change. The source is now opened and checked before the destination. Each line has a test.

• A directory as the source still rejects with That doesn't work on folders. On Linux the destination was emptied before, or created empty. Now it is not opened.
• A source fd that is not open still rejects with EBADF. Before, the destination was emptied or created, and it got the mode option. Now it is not opened. A destination fd of the caller also got the mode option, because the job closed before it recorded the error. Now it keeps its mode.
• Not changed, with a test as a guard: when the destination open fails, the job closes the source that it opened. That close moved from do_open_file to run_async.

Cost per copy. Release builds of the earlier base (9f70da0741) and of this PR before the merge with main, Linux x64. 200 copies of a 4,096-byte file, one at a time, less a run with 0 copies. strace is not installed here, so a small ptrace counter made the counts. The write that wakes the event loop is not in the table.

• The rows with a caller's fd include the open and the close that the script makes for that fd.
• Allocator calls per copy to a new path: 11.54 and 11.57 on main, 11.56 and 11.62 on this PR (gdb hit counts on the mi_* entry points, 1,000 copies less a run with 0, two runs for each build). The difference is inside the spread of two runs of one build. bun_sys::stat would add one allocation and one free for each new path, because its error copies the path.
• Release .text: 58,148,853 to 58,147,573 bytes (size -A). The job function is 1,068 bytes smaller, because do_open_file is now built for two cases and not three.
• One stat(2) on this file system (overlayfs), from a C loop of 200,000 calls: 0.40 us for a path that does not exist, 1.00 us for a path that exists.
• End to end, 15 interleaved rounds of 2,000 copies: 77.5 us per copy to a new path on main and 76.3 us on this PR. Two halves of the main runs differ by 9.5 us, so the difference is not measurable here. The load average of the host was 250 to 300.
• Not measured: instructions per copy (perf and valgrind are not installed, and perf_event_paranoid is 4). NFS (no mount here): read from the code, a copy to a new path makes one more LOOKUP when the client has no negative entry for the name.
• macOS, read from the code and not measured: a new destination is cloned by clonefile and pays nothing. An existing destination pays one stat of the destination and one fstat of the source.

Sites with the same class of bug that this PR does not change. The class is: a copy destroys its destination before it knows that the destination is not the source.

• The Windows read/write loop (CopyFileWindows::prepare_pathlike) opens the destination with O_TRUNC before the source. On main it runs only with BUN_FEATURE_FLAG_DISABLE_UV_FS_COPYFILE=1, or when an fd is a pipe or a character device. uv_fs_copyfile handles every other copy and skips a file copied onto itself.
• Windows: after the no-op of uv_fs_copyfile, on_complete cuts the file to the size that the destination BunFile cached. The todo row for Windows records it.
• A bare Bun.file(p).stream() source. A ReadableStream does not use CopyFile. The file sink opens the destination with O_TRUNC, and the stream opens the source later. fs.promises.writeFile(p, fs.createReadStream(p)) in Node 26.3.0 also leaves 0 bytes.
• One fd on both sides. Bun does not open or truncate a destination fd that the caller supplies, so the compare does not run for it.
• macOS: /dev/fd/N as a destination path goes to fcopyfile, which ends with ftruncate(dst_fd, totalCopied). With stdout redirected to a file that has content, main cuts that file to the copied size. This PR does not change it.
• fs.copyFile and fs.cp in src/runtime/node/node_fs.rs, and shell cp: #37962 owns these sites.

#38186 moves the source checks before the destination open too, with a larger change (it also covers a FIFO source on Linux and the Windows read/write loop). The two PRs conflict in run_async. The PR that merges second keeps only what the other does not have.

#47667 copies a pipe or block-device source with the read/write loop. Its changes to copy_file.rs and the changes here do not overlap: the two merge with no conflict. On a debug build of that merge (f299d81f8b with ba56fce519), the 22 rows here pass. The 23 tests of bun-write-pipe-source.test.ts pass too, and a directory source still leaves the destination as it was.

A slice() of the same file, and #41209.

• On main, a file-to-file copy ignores the source's slice() window. So in this PR a sliced copy onto itself is a whole-file copy onto itself.
• #41209 gives CopyFile the source window. After it, a window of the same file must give the window, not a no-op. The row a slice of the file as the source gives the same result as for another destination fails until the merged code does that.
• A merge of #41209 (704fa307fc) with this branch (f299d81f8b) has two conflict hunks in copy_file.rs. One is the size that the macOS clonefile block reads. The other is the block after the directory check (keep both sides). The test file has one more, with the block of this PR (keep both blocks). #41209 also conflicts with main alone: in src/jsc/webcore_types.rs, in Blob.rs and in the import list of the test file.
• The window case belongs in the same-file arm of run_async, which runs when do_open_file returns with no destination open. For a window that is not the whole file, that arm must open the destination without O_TRUNC, copy the window to the start of the file, and cut the file to the bytes copied. I did not build or run that merge. The code that an earlier version of these notes gave for it no longer applies.

Tests.

• bun bd test test/js/bun/io/bun-write.test.js: the 22 new rows pass. With a release build of main, 18 fail and 4 pass. The 4 are guards: a longer destination is replaced, /dev/null as the destination, /dev/full on both sides, and the source fd after a failed destination open.
• Each clause has a row. With four clauses removed in one build (the regular-file check, the fchmod of the no-op, the close after a failed destination open, the error recorded before the close), exactly four rows fail. The row for new/../f fails when the compare runs only once, before the loop.
• The whole file, before the merge: 101 pass, 5 time out at 5 s on a debug build under load. Each of the 5 passes with a longer timeout. #37792 describes these timeouts. The first section has the run after the merge.
• bun run rust:check-all: 12 of 12 targets compile for the first commit. For the second commit I ran 4 (macOS arm64, FreeBSD x64, Windows x64, Linux musl x64), for the third 2 (macOS arm64, FreeBSD x64). After the merge with main: 12 of 12.
• Not run: macOS and Windows. CI runs the rows there.
• Self-review: the first version of this PR had 18 concerns, and 16 were addressed. A review of this version did not finish. The automated reviews of its three commits raised two points (the retry after mkdir, dead arms in do_open_file). Both are fixed.