pack/publish: stop panicking on package.json bin and files entries longer than the path buffer
Closed
RO
2mo ago+312+312+012345678901234567890123456789-21-21-01234567890123456789
Problem
bun pm packandbun publishabort withpanic: range end index 5000 out of range for slice of length 4096(exit 134 plus a crash report) when package.json has abinstring, abinobject value, or adirectories.binlonger than the path buffer (4096 bytes on Linux, 1024 on macOS, 98302 on Windows).bun pm packdoes the same for afilesentry of that length, andbun publish <tarball>also for abinobject key.- Cause: those strings are normalized with
resolve_path::normalize_buf/normalize_buf_zinto a stackPathBuffer, and the normalizer indexes past the end of whatever buffer it is given (src/paths/resolve_path.rs,buf[buf_i..buf_i + count].copy_from_slice(..)innormalize_string_generic_tz). Sites:get_package_bins(three) and thefilesloop inpack()insrc/runtime/cli/pack_command.rs;normalize_bin(four) insrc/runtime/cli/publish_command.rs.normalize_binruns forbun publish <tarball>too, which never goes through pack, so both files need the change.
Fix
- Add
resolve_path::normalize_buf_spillandnormalize_buf_z_spill: normalize into the caller's buffer when the result is known to fit, otherwise into a caller-ownedVecgrown to size. Same shape as the existingjoin_z_buf_spill, and the same approach #37526 took for the browser map, which is this bug in another package.json field. The size bound isinput + 2: normalizing only removes bytes, except that""becomes"."and two Windows spellings grow by one byte, and the_zvariant appends a NUL. Unit tests inbun_pathspin the fits / spills / exact-length / empty cases. - The eight sites use them, with one spill
Vecper function hoisted out of the loops, so inputs that fit are handled exactly as before and longer ones just land in theVec. - No length cutoff, because the downstream behavior is already right once the string survives normalization. A bin that long cannot exist on disk, and both commands already have a path for a bin that does not exist: pack queues file bins as optional and ignores a bin directory it cannot open (
openat/open_dir/fstatatreturnENAMETOOLONGfor these, I checked the POSIX and Windows wrappers), publish prints its existingbin '...' does not existwarning and keeps the entry in the registry metadata like npm does. Afilesentry, on the other hand, is a glob, and a long one (a brace group with many names) is legitimate and has to keep matching; cutting it off would be wrong, so it is not a "bad input" case at all. normalize_bin'sdirectories.binbranch only downgradedENOENTto the warning;ENAMETOOLONGnow takes the same branch, since likeENOENTit means nothing exists at that path (other errors, where something exists but cannot be opened, still fail the publish). Without this line thedirectories.bintarball test below exits 1 withfailed to open bin directory.- Verified with:
test/cli/install/bun-pack.test.ts:bins > longer than the path buffer(string, object value,directories.binare skipped; a normal bin listed next to a long one is still packed executable) andfiles > an entry longer than the path buffer is still matched(a 100 KB brace glob still selectsdist/). The values are 100,000 bytes so the tests also exceed the Windows buffer and are not platform gated. These 5 panic on the current release and pass with this change; the whole file passes (81).test/cli/install/bun-publish.test.ts:bin longer than the path buffer, publishing a tarball (built withBun.Archiveso the test does not depend on the pack fix) with each of the four shapes, asserting the exact set of warnings, plus a directory publish.--dry-runstops before the registry is contacted; the userinfo in the registry URL only satisfies the auth check, the same trick an existing dry-run test uses. These 5 panic on the current release and pass with this change; the whole file passes (44, run with a raised timeout because the debug build is too slow for the file's lifecycle-script tests at the default 5s).cargo test -p bun_paths(22 pass) andbun run rust:miri -p bun_paths(22 pass).
- Overlap: #38749 (
--destination) and #38753 (thebun publish <tarball>argument) fix the other two length panics in these commands; they are different buffers and different inputs. #38753 also edits theuse bun_paths::resolve_path::{..}line inpublish_command.rs, so one of the two will need the one-line import merge (union of both lists). Against the current heads of #38720 and #38707, which edit other lines ofget_package_bins,src/merges cleanly and only the test additions at the end ofdescribe("bins")conflict. - Not covered here:
bun installof a dependency whose bin value is this long still aborts, in the bin linker'sjoin_abs_string_z(src/install/bin.rs,resolve_bin_target), a different buffer reached from the lockfile rather than from a package.json being packed. Reproduced separately and filed for its own fix.
Background
PathBufferis a[u8; MAX_PATH_BYTES]stack scratch buffer used for paths about to be handed to the OS.MAX_PATH_BYTESis the platformPATH_MAX, which is why the threshold differs per OS. Strings read out of package.json have no such bound.resolve_path::normalize_bufispath.normalizeinto a caller-provided buffer: collapses./,//and.., and here also converts\to/(thePosixplatform parameter), which is why pack and publish run bin paths through it before comparing them against tarball entry names. It assumes the buffer is big enough; the*_spillfunctions inresolve_path.rsare the family of wrappers that remove that assumption by falling back to a heapVec, which stays empty unless it is needed.- pack handles bins separately from the file walk:
get_package_binsreadsbin/directories.bin, file bins are pushed onto the pack queue as optional entries (dropped if they cannot be opened) and a bin directory is walked if it can be opened. publish additionally rewritesbininto the object form npm registries expect (normalize_bin), which is where it warns about bins that do not exist;bun publish <tarball>only runs this second part, on the package.json found inside the tarball.
Repro on the current release (1.4.0-canary.1)
mkdir binrepro && cd binrepro
bun -e 'require("fs").writeFileSync("package.json", JSON.stringify({name:"bin-repro",version:"1.0.0",bin:Buffer.alloc(5000,"b").toString()}))'
bun pm pack --dry-run
# panic: range end index 5000 out of range for slice of length 4096
bun publish --dry-run
# same panic (directory publish goes through pack)
mkdir -p pkg/package && cp package.json pkg/package && tar -czf long-bin.tgz -C pkg package
bun publish ./long-bin.tgz --dry-run
# same panic, this time from normalize_bin in publish_command.rs
Same result for bin: {x: <long>}, directories: {bin: <long>}, files: [<long>] (pack) and bin: {<long>: "x.js"} (tarball publish). With this change all of them exit 0; the publish variants print warn: bin '<long>' does not exist / warn: bin directory '<long>' does not exist.
no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/bun-publish.test.ts