install: fix default-trusted lifecycle scripts being blocked after yarn.lock migration

Closed
RO
2mo ago
+336-33

Found while looking at lifecycle scripts after lockfile migration (the trustedDependencies side of that is #38773); this part is about the URLs the yarn.lock migrator records and is independent of it. Touches the same name_to_use block as #38803 (a crash fix in the name extraction); see "Relation to #38803" below.

Problem

  • After migrating a yarn.lock whose entries resolve to any registry other than registry.yarnpkg.com / registry.npmjs.org (a private registry, or the test Verdaccio), packages on bun's default trusted list (electron, esbuild, ...) have their lifecycle scripts blocked on the install that follows. The same bun install without the yarn.lock runs them.
  • yarn v1 writes tarballs as resolved "<url>#<sha1>". The migrator (src/install/yarn.rs, resolution block) stored that string verbatim as the package's npm tarball URL, so bun.lock ends up with
    "electron": ["electron@1.0.0", "http://localhost:PORT/electron/-/electron-1.0.0.tgz#f1b8bc2c...", {}, "sha512-..."].
    Lockfile::has_trusted_dependency (src/install/lockfile.rs:3105) only grants default-list trust when the stored URL equals the canonical <registry>/<name>/-/<name>-<version>.tgz, so the suffix fails the comparison.
  • Entries whose URL has no suffix (what bun install --yarn itself writes, and what registries without a shasum produce) took a different wrong turn: the resolved.ends_with(".tgz") check turned them into remote tarball packages ("electron": ["electron@http://.../electron-1.0.0.tgz", ...]), which are never default-trusted and also skip the post-migration bin/os/cpu manifest fill-in. package-lock.json and pnpm-lock.yaml migration already store the bare canonical URL for both shapes.
  • The same suffix on a GitHub archive download (resolved "https://github.com/o/r/archive/v1.tar.gz#<sha1>") was read as a git commit, because Entry::is_git_dependency matched every https://github.com/ URL. The entry migrated to ["r/archive/v1.tar.gz@github:o/r/archive/v1.tar.gz#<sha1 prefix>", {}, ""], which the next bun install --frozen-lockfile rejects as an invalid git tag.
  • The canonical-URL check landed in #31339 and is not in a release yet (LATEST is still 1.3.14). Every bun.lock the yarn migrator has written so far on a non-default registry carries the suffix, and a plain bun install keeps the recorded URL, so without a reader-side fix the first release with #31339 would start blocking those projects' default-trusted scripts even though this PR fixes new migrations.

Fix

  • Entry::url_without_hash strips the suffix; the migrator computes the stripped URL once per entry and uses it both to name the package and to build the resolution, so the npm URL and the remote tarball URL of direct URL dependencies are both stored without it.
  • A non-URL dependency with a semver version is always stored as an npm package whose tarball URL is resolved (empty for the two default registries, as before). The .tgz to remote-tarball fallback now only applies to entries whose version is not semver, the one case where it was not misclassifying registry packages. Direct URL dependencies (foo@https://... specs) still become remote tarballs.
  • is_git_dependency classifies https://github.com/... with the dependency parser's existing is_github_tarball_path rule (applied to the URL without the suffix), so archive downloads become remote tarballs like any other URL dependency and repositories are still git.
  • has_trusted_dependency compares the stored URL without its fragment. This is what keeps already-migrated lockfiles working once #31339 ships. It is exactly as strict as before: the request path bun sends is URL.pathname, which bun_url cuts at # (src/url/lib.rs:649-668, used by build_request in src/http/lib.rs), so <canonical>#anything downloads the canonical tarball and nothing else; a URL whose path differs still fails, fragment or not.
  • The name block now only reads a package name out of the URL for direct URL dependencies; registry entries take the spec's name, which is what every suffixed entry already did. With the reclassification above, a default-registry entry's name is what the download URL is built from, so it should come from the spec rather than from however the URL happened to spell it.
  • Verified (each new test fails on the build without src/ changes and passes with them, except the two guards noted):
    • test/cli/install/bun-install-lifecycle-scripts.test.ts, describe("default trusted dependencies after yarn.lock migration"): electron@1.0.0 via yarn.lock with and without the suffix runs its preinstall and records the canonical URL; a yarn.lock pointing electron at all-lifecycle-scripts's tarball keeps the scripts blocked; a hand-written bun.lock in the shape the migrator produced until now (...electron-1.0.0.tgz#<sha1>) runs the scripts, and the same lockfile pointing at the other package's tarball (also suffixed) stays blocked (guard, passes before and after). The pre-existing "require the canonical registry tarball URL" test (wrong path, wrong origin) still passes.
    • test/cli/install/migration/yarn-lock-migration.test.ts, describe("yarn.lock migration of registry tarball URLs"): bun pm migrate against a local 404 registry checks the migrated bun.lock for a private registry (suffixed and suffix-less, with a dependency edge between them), the default registry without a suffix (plain and npm: alias), a direct URL dependency on a registry host and on a GitHub archive, and a github.com repository URL still migrating as git (guard).
    • Existing snapshots: only two lines change, both losing a #sha1 suffix (onetime in yarn's own lockfile, an http://registry.npmjs.org/ URL, and the remote direct URL dependency in yarn-stuff). The other ~1000 entries of the yarn-cli-repo snapshot, including its git and codeload entries, are unchanged.
    • Also run: the whole yarn-lock-migration.test.ts file, nested-overrides.test.ts "yarn.lock resolutions paths become nested rules" (suffix-less local-registry URLs, migrate + install), the trust tests in bun-pm.test.ts and bun-install-registry.test.ts, and the full lifecycle scripts file (124 pass; the 3 failures need node on PATH and fail on main in this environment too).

Relation to #38803

  • #38803 replaces the index arithmetic inside the name_to_use block (a panic on crafted URLs, found while working on this). This PR only changes that block's guard (is_direct_url_dep) and the URL it reads (resolved_url), so whichever lands second has a one-block conflict: the combined shape is #38803's extraction, gated on is_direct_url_dep, reading the stripped URL. #38803's pinned row (a semver entry resolved to a suffix-less tarball) becomes a registry package with this PR, as its description already notes. Landing #38803 first is simplest; this branch gets rebased either way.

Background

  • Default trusted dependencies: when package.json has no trustedDependencies, bun runs lifecycle scripts only for packages on a built-in list (src/install/default-trusted-dependencies.txt). Because the grant is keyed by name, has_trusted_dependency (since #31339) additionally requires the package to be an npm package whose tarball URL is either empty (default registry) or exactly the canonical tarball URL on the configured registry, so a lockfile cannot obtain the grant by recording a different tarball under a trusted name.
  • npm resolution URL: an npm package in the lockfile carries the tarball URL the registry reported (dist.tarball); bun.lock prints it as "" when it is under the default registry and verbatim otherwise. A remote tarball resolution is a different package kind used for dependencies declared as a URL; it has no version and no registry, so it never qualifies for the default list.
  • yarn v1 resolved field: <tarball url>#<hex sha1 of the tarball> for registry tarballs, URL dependencies and GitHub archives alike; the suffix is omitted when the registry did not report a shasum. For git dependencies the part after # is the commit instead. bun's own bun install --yarn printer writes URLs without a suffix.
  • URL fragment: the part of a URL after #. HTTP clients never transmit it; bun's URL parser puts it in hash and excludes it from pathname, which is what becomes the request path.
Earlier shape of this PR

The first revision only stripped the suffix in the migrator and reclassified suffix-less registry entries. Self-review turned up the GitHub archive case (same suffix, misread as a commit one layer up) and the population of lockfiles that were already migrated with the suffix, which the unreleased canonical-URL check would otherwise start blocking; both were folded in, the strip was hoisted so the name block sees the same URL as the resolution block, and the tests above were extended to cover them.

no test proof · iteration 4 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/bun-install-lifecycle-scripts.test.ts