pack/publish: stop panicking on package.json bin and files entries longer than the path buffer

Closed
RO
2mo ago
+312-21

Problem

  • bun pm pack and bun publish abort with panic: range end index 5000 out of range for slice of length 4096 (exit 134 plus a crash report) when package.json has a bin string, a bin object value, or a directories.bin longer than the path buffer (4096 bytes on Linux, 1024 on macOS, 98302 on Windows). bun pm pack does the same for a files entry of that length, and bun publish <tarball> also for a bin object key.
  • Cause: those strings are normalized with resolve_path::normalize_buf / normalize_buf_z into a stack PathBuffer, and the normalizer indexes past the end of whatever buffer it is given (src/paths/resolve_path.rs, buf[buf_i..buf_i + count].copy_from_slice(..) in normalize_string_generic_tz). Sites: get_package_bins (three) and the files loop in pack() in src/runtime/cli/pack_command.rs; normalize_bin (four) in src/runtime/cli/publish_command.rs. normalize_bin runs for bun publish <tarball> too, which never goes through pack, so both files need the change.

Fix

  • Add resolve_path::normalize_buf_spill and normalize_buf_z_spill: normalize into the caller's buffer when the result is known to fit, otherwise into a caller-owned Vec grown to size. Same shape as the existing join_z_buf_spill, and the same approach #37526 took for the browser map, which is this bug in another package.json field. The size bound is input + 2: normalizing only removes bytes, except that "" becomes "." and two Windows spellings grow by one byte, and the _z variant appends a NUL. Unit tests in bun_paths pin the fits / spills / exact-length / empty cases.
  • The eight sites use them, with one spill Vec per function hoisted out of the loops, so inputs that fit are handled exactly as before and longer ones just land in the Vec.
  • No length cutoff, because the downstream behavior is already right once the string survives normalization. A bin that long cannot exist on disk, and both commands already have a path for a bin that does not exist: pack queues file bins as optional and ignores a bin directory it cannot open (openat / open_dir / fstatat return ENAMETOOLONG for these, I checked the POSIX and Windows wrappers), publish prints its existing bin '...' does not exist warning and keeps the entry in the registry metadata like npm does. A files entry, on the other hand, is a glob, and a long one (a brace group with many names) is legitimate and has to keep matching; cutting it off would be wrong, so it is not a "bad input" case at all.
  • normalize_bin's directories.bin branch only downgraded ENOENT to the warning; ENAMETOOLONG now takes the same branch, since like ENOENT it means nothing exists at that path (other errors, where something exists but cannot be opened, still fail the publish). Without this line the directories.bin tarball test below exits 1 with failed to open bin directory.
  • Verified with:
    • test/cli/install/bun-pack.test.ts: bins > longer than the path buffer (string, object value, directories.bin are skipped; a normal bin listed next to a long one is still packed executable) and files > an entry longer than the path buffer is still matched (a 100 KB brace glob still selects dist/). The values are 100,000 bytes so the tests also exceed the Windows buffer and are not platform gated. These 5 panic on the current release and pass with this change; the whole file passes (81).
    • test/cli/install/bun-publish.test.ts: bin longer than the path buffer, publishing a tarball (built with Bun.Archive so the test does not depend on the pack fix) with each of the four shapes, asserting the exact set of warnings, plus a directory publish. --dry-run stops before the registry is contacted; the userinfo in the registry URL only satisfies the auth check, the same trick an existing dry-run test uses. These 5 panic on the current release and pass with this change; the whole file passes (44, run with a raised timeout because the debug build is too slow for the file's lifecycle-script tests at the default 5s).
    • cargo test -p bun_paths (22 pass) and bun run rust:miri -p bun_paths (22 pass).
  • Overlap: #38749 (--destination) and #38753 (the bun publish <tarball> argument) fix the other two length panics in these commands; they are different buffers and different inputs. #38753 also edits the use bun_paths::resolve_path::{..} line in publish_command.rs, so one of the two will need the one-line import merge (union of both lists). Against the current heads of #38720 and #38707, which edit other lines of get_package_bins, src/ merges cleanly and only the test additions at the end of describe("bins") conflict.
  • Not covered here: bun install of a dependency whose bin value is this long still aborts, in the bin linker's join_abs_string_z (src/install/bin.rs, resolve_bin_target), a different buffer reached from the lockfile rather than from a package.json being packed. Reproduced separately and filed for its own fix.

Background

  • PathBuffer is a [u8; MAX_PATH_BYTES] stack scratch buffer used for paths about to be handed to the OS. MAX_PATH_BYTES is the platform PATH_MAX, which is why the threshold differs per OS. Strings read out of package.json have no such bound.
  • resolve_path::normalize_buf is path.normalize into a caller-provided buffer: collapses ./, // and .., and here also converts \ to / (the Posix platform parameter), which is why pack and publish run bin paths through it before comparing them against tarball entry names. It assumes the buffer is big enough; the *_spill functions in resolve_path.rs are the family of wrappers that remove that assumption by falling back to a heap Vec, which stays empty unless it is needed.
  • pack handles bins separately from the file walk: get_package_bins reads bin / directories.bin, file bins are pushed onto the pack queue as optional entries (dropped if they cannot be opened) and a bin directory is walked if it can be opened. publish additionally rewrites bin into the object form npm registries expect (normalize_bin), which is where it warns about bins that do not exist; bun publish <tarball> only runs this second part, on the package.json found inside the tarball.
Repro on the current release (1.4.0-canary.1)
mkdir binrepro && cd binrepro
bun -e 'require("fs").writeFileSync("package.json", JSON.stringify({name:"bin-repro",version:"1.0.0",bin:Buffer.alloc(5000,"b").toString()}))'
bun pm pack --dry-run
# panic: range end index 5000 out of range for slice of length 4096
bun publish --dry-run
# same panic (directory publish goes through pack)
mkdir -p pkg/package && cp package.json pkg/package && tar -czf long-bin.tgz -C pkg package
bun publish ./long-bin.tgz --dry-run
# same panic, this time from normalize_bin in publish_command.rs

Same result for bin: {x: <long>}, directories: {bin: <long>}, files: [<long>] (pack) and bin: {<long>: "x.js"} (tarball publish). With this change all of them exit 0; the publish variants print warn: bin '<long>' does not exist / warn: bin directory '<long>' does not exist.


no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/bun-publish.test.ts