#45761 install: fall back to copyfile when linkat fails with EMLINK, ENOTSUP or ENOSYS

Prior art

pnpm (pnpm/pnpm#14778) and uv (astral-sh/uv#17699) have the same fallback at the link limit. The uv change names the 177 links of AWS EFS.

The two questions of the request

• Should EMLINK copy only the file that is full and keep linking the rest? It makes no difference for a cache that installs filled. Each install adds one link to every file of a cached package, so all of its files reach the limit together. Measured on ext4 with all 23 files of a cache entry at 65,000 links: the package is copied whole either way. What matters is the next package, so EMLINK does not store the copyfile backend.
• Does the wider list belong in #40628? That PR is closed (#44968 took it in), so this is a PR of its own.

Repro

An LD_PRELOAD shim makes linkat fail with a chosen errno (the test file has one). Package p is a tarball dependency. x86_64 Linux.

No shim: on ext4, link every file of a cache entry until each has 65,000 links, then install. Main exits 1 on both linkers: EMLINK: failed copying files from cache to destination for package p, EMLINK: Too many links: failed to link package: p@./p.tgz (link). This branch exits 0 and the package is a copy.

The regression since #44968: a hoisted file:../x folder with every linkat refused with ENOTSUP or ENOSYS. 1.4.3 installs it with symlinks and never calls linkat. Main exits 1: ENOTSUP: failed copying files from cache to destination for package x. This branch copies the folder. A folder inside the project (file:./x) fails on 1.4.3 too.

A refusal after some files are linked

A file can reach the limit alone, for example when a project deleted or replaced its copy of another file. Then the link loop stops in the middle of a package.

• Hoisted: the copy creates each file with O_EXCL and unlinks a file that is there (on Windows CopyFileW and a delete). This is the code of main.
• Isolated: the copy (FileCopier) opens a file that is there and writes into it. On POSIX a linked file stayed a hard link and the copy wrote the cache file onto itself, which changed its mtime. On Windows CopyFileW cannot overwrite a hard link of its source, so the package failed. Hardlinker::delete_dest removes the package directory before the copy. That directory is the staging directory of this install, or the package directory in the store when the linker works in place. It also runs for EXDEV, EACCES and EPERM, where main has the same partial state.
• The removal is best effort, like the other delete_tree calls of the file. A link that stays gives the result of the last point. A strict error would fail an install that works on main: when the linker works in place, the package directory could not be moved away or deleted before, and the copy into it still works.

Left out on purpose

• The isolated file: folder (link_folder). On main its copy fallback goes to FileCopier, which opens the files that are there without O_EXCL or O_TRUNC. With ENOTSUP on this gate: the first install works, a source file then shrinks, and the second install leaves the new text followed by the old tail. That is issue #45194, which exists today for EXDEV, EACCES and EPERM. #45569 makes that copy replace the file. After it lands, link_folder can ask link_refusal too. A folder is linked into a directory that stays between installs, so this PR does not delete it.
• The macOS gates that compare with EOPNOTSUPP only (clonefile, fcopyfile, a rename fallback, and the Darwin value in the .bin gate). Darwin reports ENOTSUP (45). #46022 has them.
• Windows ERROR_INVALID_FUNCTION, the answer of a volume without hard links. The Win32 table turns it into EISDIR, which is not on the list. This is from reading the code, not from a run: CI has no such volume. On FAT32 and exFAT the isolated layout also needs junctions or symlinks, which those file systems do not have, so a row for this code alone would not make the install pass there.
• Linux errno 524 (ENOTSUPP, which NFS can let out). get_errno() decodes it as EUNKNOWN. It still fails. Say if it should copy too.
• A fresh inode for a cache file at the limit, so that later installs link again. uv does that (astral-sh/uv#17699). A comment on this PR reports the probes and why it is not built for now.
• A copy of only the refused file inside the link loop. The branch robobun/0d97fe3b/install-link-or-copy-per-file has it, with its measurements. It needs two decisions that this patch does not: a copy on every errno that is not listed, and a package that is part links and part copies.

Checks

• bun-install-hardlink-fallback.test.ts on the debug build, Linux: 30 pass on this branch. With the src/install of main, 15 fail: the six EMLINK, ENOTSUP and ENOSYS rows of the errno grid, the four folder tests with ENOTSUP and ENOSYS, the two link limit tests, the two partial link tests and the ENOSYS row of the .bin tests.
• The link limit tests pin the EMLINK rule. With a build that stores the copyfile backend for EMLINK, both fail (6 of 6 runs).
• The isolated partial link test pins delete_dest. With a build that skips the call, the two files that were linked before the refusal keep 2 links and the test fails.
• Real ext4, no shim, all 23 files of a cache entry at 65,000 links: the patch exits 0 on both linkers and the package is a copy. Main exits 1.
• Windows: two tests reach the real NTFS limit, with every file of the package at the limit and with only package.json at the limit. NTFS lists a directory by name, so in the second test the other files are linked before the refusal. Both pass on Windows x64 and aarch64 in CI, the only place where they run.
• macOS: four tests refuse linkat with ENOTSUP (45) and EOPNOTSUPP (102) through a DYLD_INSERT_LIBRARIES interposer. They run only in CI.
• Size and instructions were measured at 4c06fb25b4 against main 91e7e3b140, before delete_dest was added. Size, release builds of main and of main with the src/install part of the patch (the rest is Windows code): .text is 81,665,765 and 81,666,021 bytes, .rodata is the same, and the stripped bun is 107,243,232 bytes on both. Instructions per linked file (ptrace single-step of the walk, packages of 1,003 and 3,003 files): hoisted 29.00, main 28.00 (the same code with one more padding NOP), isolated 112.00 on both. delete_dest runs only after a link is refused.
• cargo check -p bun_install for Linux, macOS (aarch64) and Windows (x64). cargo clippy -p bun_install.
• Suites on the debug build, all pass: bun-install-backends (28), bun-link (57), isolated-relink (6), isolated-install (283), bun-install (771). The last three ran with --timeout 120000: on this machine some of their tests pass the 5 s default only when they run alone.

Self-review

A review of a first design (a per-file link-or-copy walk) asked for this gate patch instead. A second review, of this patch, raised 8 points. All are in the diff or in this text: link_folder stays on its own list until #45569, the partial link test, ENOSYS in the .bin gate, the macOS rows, the NTFS precondition, the cost of a package at the limit with the limits and the prior art, the Windows code that is left out, and two wording fixes. The review on the PR then found the isolated copy over the partial links. delete_dest and its tests answer it.