install: document and pin the isolated linker's store layout

Closed
RO
1mo ago
+479-17

Context

Requested by @dylan-conway: under the isolated linker, each package lives at node_modules/.bun/<name>@<version>[+<hash>]/node_modules/<name> with relative sibling symlinks to its resolved dependencies. To cache each package as its own build action (deduplicated across a monorepo), or to assemble the same tree from already-extracted packages, tools need this directory shape to be a documented contract: entry naming (the scope / to + mapping, the peer-set suffix and what it is derived from), the inner layout, and .bin placement. Today it is observable but undocumented, so anything built against it can silently drift from what bun install produces.

What this PR does

Documents the store layout as a stable format in a new "Store layout reference" section of docs/pm/isolated-installs.mdx:

  • Entry naming per dependency source (npm, folder, local/remote tarball, git, GitHub), including the / \ : # to + character mapping that turns @scope/pkg into @scope+pkg.
  • The exact peer-set suffix derivation: collect the entry's resolved peers, sort by package name (byte order), concatenate <name><resolution> with no separator, hash with 64-bit wyhash (the original v1 variant, seed 0), format as 16 lowercase zero-padded hex digits. With a worked example: peer-deps-fixed@1.0.0 whose no-deps@^1.0.0 peer resolves to 1.0.0 hashes the bytes no-deps1.0.0 into peer-deps-fixed@1.0.0+7347ae2d86f1441a.
  • The inner entry layout (<entry>/node_modules/<real name> plus one relative sibling symlink per resolved dependency, alias-named, ../../<dep entry>/node_modules/<dep real name> targets), .bin placement (each entry carries its own bins and its dependencies' bins; the project root and each workspace get their direct dependencies' bins), project/workspace linking, and the node_modules/.bun/node_modules fallback directory.

It also fixes the existing peer-dependency section of that page, which showed a package@1.0.0_react@18.2.0 naming scheme that shipped code never produced.

Pins the layout with regression tests (store layout contract in test/cli/install/isolated-install.test.ts): exact store directory listings with hardcoded peer hashes (2-peer-deps-c@1.0.0+eadc6a10a694605e for the two-peer case, verifying the name-sorted concatenation), scoped/folder/tarball entry names, sibling symlink targets including the extra .. for scoped aliases, bin placement at all three levels (entry, project root, workspace), range deduplication onto existing entries, per-peer-set entry splitting across workspaces, and workspace-to-workspace links. A failure in these tests now means a breaking change to the documented format.

Points the code at the contract: the stale layout comment in Store.rs (same never-shipped _peer@version scheme) now shows the real shape, and the peer-hash derivation site in isolated_install.rs warns that the derivation names on-disk entries and is documented as stable.

Why document-and-lock rather than a producer command

Both options from the request were considered: (a) commit to the layout in docs plus tests, or (b) keep the format internal behind something like bun pm store-entry --tarball=<file> --deps=<resolved list> --out=<dir>.

The command does not remove the need for the contract. To produce one entry, a caller must supply the resolved peer set of the package and the full entry names of its dependencies (their peer hashes included), because the sibling symlinks and .bin links embed those names. Computing per-node resolved peer sets is the output of bun's whole-tree resolution, so the command's --deps input is exactly the information the layout contract describes, and the caller still has to obtain it from an install or a resolution pass. Documenting and pinning the layout is therefore the prerequisite for either path, and per the request the layout is not expected to change, so locking it is the smaller, complete deliverable. A producer command can still be layered on top later without changing the format, if a concrete consumer wants one.

The hash derivation was verified by reimplementing it independently and matching every hardcoded suffix against live installs (two single-peer cases, one two-peer case for the sort order, and the no-resolved-peers case which gets no suffix).

Verification

bun bd test test/cli/install/isolated-install.test.ts
# 58 pass, 0 fail (includes the 2 new store layout contract tests)

The Rust changes are comments only; no behavior changes outside docs and tests.

no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/isolated-install.test.ts