install: stop aborting on link: specifiers and patch paths longer than the path buffers
Closed
RO
2mo ago+302+302+012345678901234567890123456789-109-109-012345678901234567890123456789
Problem
bun installaborts on two more package.json values that are longer than bun's fixed path buffers (4096 bytes on Linux, 1024 on macOS, 32767 * 3 + 1 on Windows). Both are pre-existing in 1.4.0; the siblingworkspace:and local.tgzcases are fixed by #37462 and are not touched here."x": "link:<name>",src/install/resolvers/folder_resolver.rs,normalize_package_json_path, three overflows in one function:- the name is normalized through the 1024 byte thread-local scratch of
normalize_string, so any name over 1024 bytes aborts:panic: range end index 1025 out of range for slice of length 1024 "/package.json"and the NUL terminator are appended to the absolute path without a length check. This one is also reachable with afile:dependency: the folder path itself is length-checked when package.json is parsed (lockfile/Package.rs), the 13 appended bytes are not, so a folder whose absolute path is within 13 bytes of the buffer size aborts:panic: range end index 4104 out of range for slice of length 4096, orpanic: index out of bounds: the len is 4096 but the index is 4096for the NULget_or_putcopies the name as written into a stackPathBufferbefore reading the target's package.json, so a name that only fits once normalized (x/../x/../.../foo) aborts after resolving:panic: range end index 100003 out of range for slice of length 4096
- the name is normalized through the 1024 byte thread-local scratch of
- Once the resolver accepts such a name, the hoisted installer (
src/install/PackageInstaller.rs,Symlinkarm ofinstall_package_with_name_and_resolution) concatenates the global link directory and the name intofolder_path_bufwithout a length check:panic: range end index 5063 out of range for slice of length 4096. "patchedDependencies": { "x@1.0.0": "patches/<long>.patch" },src/install/patch_install.rs:calc_hash(on a thread pool worker, for every entry, whether or notxis a dependency) andapplyjoin the path onto the project directory withjoin_z_bufinto aPathBuffer:panic: range end index 5033 out of range for slice of length 4094.
Fix
normalize_package_json_pathreturnsNonewhen the package.json path does not fit: the name is normalized withnormalize_string_spill(heap when longer than the scratch), the.-prefixed branch joins withabs_buf_checked, the other branch computes the length up front, and the last byte of the buffer is reserved for the NUL.get_or_putmapsNonetoError::Sys(ENAMETOOLONG), which prints exactly what the OS-rejected case already prints (error: ENAMETOOLONG...x@link:... failed to resolve, exit 1). TheSearched informatter now builds its./-prefixed message by appending the relative path to an owned buffer (instead of writing the prefix in front of the path buffer through a pointer cast) and prints the value as written when it does not fit; its output is unchanged andbun-workspaces.test.tsnow asserts it.- The resolver's copy of the name moves to a
Box<[u8]>. The copy is still needed (the slice can point into the lockfile string buffer, which reading the target's package.json grows), it just no longer has a fixed size. - The hoisted installer checks the concatenated length and fails the package (
ENAMETOOLONG: link path for package foo is too long,Failed to install 1 package, exit 1; nothing is printed under--silent), the same way theFolderarm above it fails an over-long folder path. Such a target could never be linked anyway:symlink(2)rejects targets longer than PATH_MAX. - The patch tasks join with
join_z_buf_spill, so the path is built on the heap when it does not fit and the OS rejects it.calc_hash's non-ENOENT stat failure used to add a warning saying the patch file is empty (never printed, the main thread then printed a generic "Failed to calculate hash"); it now adds an error with the errno:error: failed to read patch file: ENAMETOOLONG: /proj/patches/ppp...patch: File name too long (stat()), exit 1 as before. FileSystem::normalizeinsrc/resolver/lib.rswas the only remaining wrapper around the 1024 byte scratch and has no callers left, so it is removed. The other two directnormalize_stringcallers outsidebun_paths(run_command.rs, Windows only, and the shell'srm) have their own open PRs (#37528, #37521).- Tests, each of which aborts on the unfixed build (
USE_SYSTEM_BUN=1) and passes withbun bd test:test/cli/install/bun-link.test.ts: a 100 kBlink:name fails withENAMETOOLONG; a 100 kBx/../name that normalizes to a registered link resolves and then fails in the installer (and fails quietly with--silent); the same name for an unregistered package fails with the usualPackage "..." is not linked(pins that a long name goes through normal resolution)test/cli/install/bun-install.test.ts(POSIX): afile:dependency whose package.json path is exactly the buffer size, or longer by less than"/package.json", fails withENAMETOOLONG; one byte below the buffer size it is still looked up on disk (Could not find package.json), which passes before and after and pins the boundarytest/cli/install/bun-install-patch.test.ts: a 100 kB patch path fails with the error above (the errno assertion is skipped on Windows, which may report the path as missing instead)
- Also run: the whole of
bun-install-patch.test.ts,bad-workspace.test.tsandbun-workspaces.test.ts, and thefile:tests ofbun-install.test.ts, all green.bun-link.test.ts's "should link dependency without crashing" fails on main with a debug build independently of this change (the debug-only stack dump on install failure lands in stdout, see #37335).cargo check -p bun_installpasses forx86_64-pc-windows-msvcandaarch64-apple-darwin; clippy is clean.
Not in this PR
--linker isolatedhas its own copy of the installer overflow:isolated_install/Installer.rsappend_store_pathappends the link name into a length-assuming path (panic: index out of bounds: the len is 4096 but the index is 4096). It is reachable today from a bun.lock with a longlink:resolution and, after this PR, from alink:name that only fits once normalized. Reported separately; it sits in the area #37424 is changing.workspace:<long path>and./<long>.tgz: #37462.
Background
PathBufferis a stack array ofMAX_PATH_BYTES(the OS PATH_MAX) that most of the install code builds paths in. The joining helpers inbun_paths::resolve_pathwrite into whatever buffer they are given and index out of bounds if the result does not fit; the_checkedvariants returnNoneinstead and the_spillvariants grow a caller-providedVecinstead.normalize_stringis the variant that writes into a 1024 byte thread-local buffer.- The folder resolver (
folder_resolver.rs) is shared byfile:folders,workspace:packages andlink:names: it builds the absolute path of the target'spackage.json, reads it and records the package. Forlink:the prefix is the global link directory (bun linkregisters packages there) and the recorded resolution is the name exactly as written, which is what the installers later symlink to. patchedDependenciesare hashed on a thread pool before anything is installed (PatchTask::calc_hash), so the patch path is joined even when the patched package is not a dependency.
Probe: all shapes on the unfixed and fixed builds (Linux, offline)
unfixed (1.4.0) fixed
link 300 bytes error: ENAMETOOLONG (OS) same
link 1025 bytes panic ... length 1024 error: ENAMETOOLONG
link 5000 / 100000 panic ... length 1024 error: ENAMETOOLONG
link 100k of x/../foo panic ... length 4096 (get_or_put) Package "x" is not linked
(foo registered) panic ... length 4096 (get_or_put) ENAMETOOLONG: link path for package foo is too long
file: pkg.json path
= buffer - 1 Could not find package.json same
= buffer panic: index out of bounds 4096 error: ENAMETOOLONG
= buffer + 8 panic ... 4104 out of range for 4096 error: ENAMETOOLONG
patch path 300 bytes Couldn't find patch file same
patch path 5000 / 100k panic ... length 4094 error: failed to read patch file: ENAMETOOLONG: ...
workspace: not found Searched in "./packages/x" same
Rebase onto 1b88ad3: bun-install-patch.test.ts had gained a describe block at the end of the file on main (patchedDependencies declared by a dependency), so the new block here now follows it and the harness import merges both lists. install_package_with_name_and_resolution had turned its IS_PENDING_PACKAGE_INSTALL const generic into the is_pending_package_install parameter, so the new Symlink check passes that, like the Folder arm next to it. No other changes.
no test proof · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/bun-install-patch.test.ts test/cli/install/bun-install.test.ts test/cli/install/bun-link.test.ts