install: stop aborting on link: specifiers and patch paths longer than the path buffers

Closed
RO
2mo ago
+302-109

Problem

  • bun install aborts on two more package.json values that are longer than bun's fixed path buffers (4096 bytes on Linux, 1024 on macOS, 32767 * 3 + 1 on Windows). Both are pre-existing in 1.4.0; the sibling workspace: and local .tgz cases are fixed by #37462 and are not touched here.
  • "x": "link:<name>", src/install/resolvers/folder_resolver.rs, normalize_package_json_path, three overflows in one function:
    • the name is normalized through the 1024 byte thread-local scratch of normalize_string, so any name over 1024 bytes aborts: panic: range end index 1025 out of range for slice of length 1024
    • "/package.json" and the NUL terminator are appended to the absolute path without a length check. This one is also reachable with a file: dependency: the folder path itself is length-checked when package.json is parsed (lockfile/Package.rs), the 13 appended bytes are not, so a folder whose absolute path is within 13 bytes of the buffer size aborts: panic: range end index 4104 out of range for slice of length 4096, or panic: index out of bounds: the len is 4096 but the index is 4096 for the NUL
    • get_or_put copies the name as written into a stack PathBuffer before reading the target's package.json, so a name that only fits once normalized (x/../x/../.../foo) aborts after resolving: panic: range end index 100003 out of range for slice of length 4096
  • Once the resolver accepts such a name, the hoisted installer (src/install/PackageInstaller.rs, Symlink arm of install_package_with_name_and_resolution) concatenates the global link directory and the name into folder_path_buf without a length check: panic: range end index 5063 out of range for slice of length 4096.
  • "patchedDependencies": { "x@1.0.0": "patches/<long>.patch" }, src/install/patch_install.rs: calc_hash (on a thread pool worker, for every entry, whether or not x is a dependency) and apply join the path onto the project directory with join_z_buf into a PathBuffer: panic: range end index 5033 out of range for slice of length 4094.

Fix

  • normalize_package_json_path returns None when the package.json path does not fit: the name is normalized with normalize_string_spill (heap when longer than the scratch), the .-prefixed branch joins with abs_buf_checked, the other branch computes the length up front, and the last byte of the buffer is reserved for the NUL. get_or_put maps None to Error::Sys(ENAMETOOLONG), which prints exactly what the OS-rejected case already prints (error: ENAMETOOLONG ... x@link:... failed to resolve, exit 1). The Searched in formatter now builds its ./-prefixed message by appending the relative path to an owned buffer (instead of writing the prefix in front of the path buffer through a pointer cast) and prints the value as written when it does not fit; its output is unchanged and bun-workspaces.test.ts now asserts it.
  • The resolver's copy of the name moves to a Box<[u8]>. The copy is still needed (the slice can point into the lockfile string buffer, which reading the target's package.json grows), it just no longer has a fixed size.
  • The hoisted installer checks the concatenated length and fails the package (ENAMETOOLONG: link path for package foo is too long, Failed to install 1 package, exit 1; nothing is printed under --silent), the same way the Folder arm above it fails an over-long folder path. Such a target could never be linked anyway: symlink(2) rejects targets longer than PATH_MAX.
  • The patch tasks join with join_z_buf_spill, so the path is built on the heap when it does not fit and the OS rejects it. calc_hash's non-ENOENT stat failure used to add a warning saying the patch file is empty (never printed, the main thread then printed a generic "Failed to calculate hash"); it now adds an error with the errno: error: failed to read patch file: ENAMETOOLONG: /proj/patches/ppp...patch: File name too long (stat()), exit 1 as before.
  • FileSystem::normalize in src/resolver/lib.rs was the only remaining wrapper around the 1024 byte scratch and has no callers left, so it is removed. The other two direct normalize_string callers outside bun_paths (run_command.rs, Windows only, and the shell's rm) have their own open PRs (#37528, #37521).
  • Tests, each of which aborts on the unfixed build (USE_SYSTEM_BUN=1) and passes with bun bd test:
    • test/cli/install/bun-link.test.ts: a 100 kB link: name fails with ENAMETOOLONG; a 100 kB x/../ name that normalizes to a registered link resolves and then fails in the installer (and fails quietly with --silent); the same name for an unregistered package fails with the usual Package "..." is not linked (pins that a long name goes through normal resolution)
    • test/cli/install/bun-install.test.ts (POSIX): a file: dependency whose package.json path is exactly the buffer size, or longer by less than "/package.json", fails with ENAMETOOLONG; one byte below the buffer size it is still looked up on disk (Could not find package.json), which passes before and after and pins the boundary
    • test/cli/install/bun-install-patch.test.ts: a 100 kB patch path fails with the error above (the errno assertion is skipped on Windows, which may report the path as missing instead)
  • Also run: the whole of bun-install-patch.test.ts, bad-workspace.test.ts and bun-workspaces.test.ts, and the file: tests of bun-install.test.ts, all green. bun-link.test.ts's "should link dependency without crashing" fails on main with a debug build independently of this change (the debug-only stack dump on install failure lands in stdout, see #37335). cargo check -p bun_install passes for x86_64-pc-windows-msvc and aarch64-apple-darwin; clippy is clean.

Not in this PR

  • --linker isolated has its own copy of the installer overflow: isolated_install/Installer.rs append_store_path appends the link name into a length-assuming path (panic: index out of bounds: the len is 4096 but the index is 4096). It is reachable today from a bun.lock with a long link: resolution and, after this PR, from a link: name that only fits once normalized. Reported separately; it sits in the area #37424 is changing.
  • workspace:<long path> and ./<long>.tgz: #37462.

Background

  • PathBuffer is a stack array of MAX_PATH_BYTES (the OS PATH_MAX) that most of the install code builds paths in. The joining helpers in bun_paths::resolve_path write into whatever buffer they are given and index out of bounds if the result does not fit; the _checked variants return None instead and the _spill variants grow a caller-provided Vec instead. normalize_string is the variant that writes into a 1024 byte thread-local buffer.
  • The folder resolver (folder_resolver.rs) is shared by file: folders, workspace: packages and link: names: it builds the absolute path of the target's package.json, reads it and records the package. For link: the prefix is the global link directory (bun link registers packages there) and the recorded resolution is the name exactly as written, which is what the installers later symlink to.
  • patchedDependencies are hashed on a thread pool before anything is installed (PatchTask::calc_hash), so the patch path is joined even when the patched package is not a dependency.
Probe: all shapes on the unfixed and fixed builds (Linux, offline)
unfixed (1.4.0)                                               fixed
link 300 bytes          error: ENAMETOOLONG (OS)              same
link 1025 bytes         panic ... length 1024                 error: ENAMETOOLONG
link 5000 / 100000      panic ... length 1024                 error: ENAMETOOLONG
link 100k of x/../foo   panic ... length 4096 (get_or_put)    Package "x" is not linked
  (foo registered)      panic ... length 4096 (get_or_put)    ENAMETOOLONG: link path for package foo is too long
file: pkg.json path
  = buffer - 1          Could not find package.json           same
  = buffer              panic: index out of bounds 4096       error: ENAMETOOLONG
  = buffer + 8          panic ... 4104 out of range for 4096  error: ENAMETOOLONG
patch path 300 bytes    Couldn't find patch file              same
patch path 5000 / 100k  panic ... length 4094                 error: failed to read patch file: ENAMETOOLONG: ...
workspace: not found    Searched in "./packages/x"            same

Rebase onto 1b88ad3: bun-install-patch.test.ts had gained a describe block at the end of the file on main (patchedDependencies declared by a dependency), so the new block here now follows it and the harness import merges both lists. install_package_with_name_and_resolution had turned its IS_PENDING_PACKAGE_INSTALL const generic into the is_pending_package_install parameter, so the new Symlink check passes that, like the Folder arm next to it. No other changes.


no test proof · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/bun-install-patch.test.ts test/cli/install/bun-install.test.ts test/cli/install/bun-link.test.ts