fix(lookup): date every proposable release and admit prereleases only on purpose

Merged
SW
1d ago
+1015-234

Fixes from the lookup and datasource scan. Each one was letting through a release that policy should have held back.

Invariant

Every version the lookup could propose is either dated and flagged by its source, or marked unchecked. A positive waitHours holds an unchecked release instead of treating it as undated. It shows up in the report as a notice, and a later lookup asks about it again. A missing answer can no longer skip the release-age wait.

Fixes

  1. Release dates cover every version, not the newest N (Go, Maven, NuGet).
    • New Version::unchecked (catalog.rs) and a release-dates state (dates.rs). The state keeps each registry answer per version: a date, or "cannot be served". It is keyed by package and, for Maven, by the operator registry entry's scope.
    • Go .info and Maven POM HEAD requests go only to versions with no stored answer, wherever they sort, with a bounded buffer_unordered fan-out of 8.
    • A cold lookup dates every version once. After that, a lookup asks only about versions it hasn't seen. This makes guide/sources.md's ".info for new versions only" true.
    • A failed request stores nothing and leaves the version unchecked. This replaces Maven's fail-open .ok()?.
    • A 404 from the Go proxy or a missing Maven POM drops the version.
    • The old top-N cut (GO_DATED_RELEASES, Maven's DATED) is gone, and so are reached_known and the head-stopping merge().
    • NuGet reads every registration page (nuget-registration v2 lists them all), up to 4 at once.
      • The flat-container listing and the registration entries are matched without build metadata.
      • A version the registration doesn't describe yet is unchecked.
      • A page link off nuget.org fails the lookup instead of being ignored.
    • I didn't plumb each repository's baseline into the shared catalog fetch. Catalogs are shared across repositories and reused for up to 23 h, so a floor recorded at fetch time could be stale for another repository's lower baseline, which fails open. Dating everything once and storing the answers is simpler, and it fails closed by construction.
  2. A permitted filter admits prereleases only when it asks for them (lookup/src/lib.rs, config/src/matching.rs).
    • Filter::asks_for_prerelease implements three rules:
      • A non-negated regex: asks when it matches the prerelease but not the same version without its prerelease part (regex:-rc\. and regex:nightly ask; regex:^2\. does not).
      • A semver: range asks when it names a prerelease version itself and matches (semver:>=3.0.0-rc.1 and semver:[1.0-beta1,2) ask; semver:[1,2) and semver:<3 do not).
      • A negated filter never asks.
    • Documented in guide/lookup-policy.md and configuration.md.
    • The existing nightly-pins example (regex:-nightly\.) keeps working.
  3. mise prerelease flag. mise() now carries prerelease: true into Version::prerelease, and the lookup treats it like a prerelease version, including for the current version, latest, and dist-tag targets. This matches github-releases.
  4. mise-versions tag dates. When the mirror's tags.json has a tag that releases.json doesn't date, the github-tags lookup falls back to GitHub's own tag catalog. That path dates annotated tags by their tagger and is cached by TagCatalog/TagObject. guide/lookup-policy.md already promised this; guide/sources.md now says how.
  5. Go .info fetches run in parallel with a bounded fan-out, as part of fix 1.

Cache and version bookkeeping:

  • catalog.rs and config/src are hashed sources, so every derived representation and state that lists them got a version bump, and its row in tests/derived-versions.tsv / tests/state-versions.tsv was updated.
  • release-dates is in tests/state-exempt.tsv, since it holds facts about immutable releases.
  • internals/cache.md describes the new state.

Tests

  • datasource/tests/go_proxy.rs:
    • every version is dated once, and a warm lookup makes no .info requests;
    • a new v0.29.4 below a known v0.30.0 is dated;
    • a failed .info leaves the version unchecked and is retried.
  • datasource/tests/sources.rs:
    • NuGet reads an older linked page holding an unlisted 6.0.5+build (yanked, dated);
    • an undescribed version is unchecked;
    • an off-site page fails the lookup;
    • Maven dates all 13 versions, retries a 503, dates new older-line releases on the next lookup and drops a 404 POM.
  • datasource/tests/versions_host.rs: an annotated tag the mirror leaves undated gets its tagger date through GitHub.
  • datasource/tests/mise_lookup.rs: prereleases_stay_in_the_catalog_for_the_policy_to_decide now asserts that the flag is carried (jq 1.7rc2, prettier alphas), and that it is absent on stable releases.
  • lookup/tests/policy.rs:
    • npm !regex:^3\., regex:^2\. and !semver:>=3 admit no rc; regex:-rc\. and semver:>=2.2.0-rc.1 do;
    • Maven semver:[1,2) admits no beta; semver:[1.5-beta1,2) and regex:-beta do;
    • PyPI semver:<3 admits no rc; semver:>=2.2.0rc1,<3 does;
    • a flagged stable-looking prerelease is held, unless the current version is flagged;
    • an unchecked release waits and is reported, and is proposed with waitHours = 0.
  • Ran on the VM: cargo test -p patchula-config -p patchula-datasource -p patchula-lookup -p patchula-engine, and clippy -D warnings on those crates plus patchula-tools.
  • The corpus snapshots are unchanged: the corpus test passes without regenerating them.

Dropped

Nothing.

Follow-ups

  • A cold lookup of a Go module or Maven artifact with hundreds of versions now makes one date request per version, once. Watch cold-run timings on large monorepos.
  • Rows the old Go code wrote to the releases table (source = 'go') are no longer read. A later cache migration can drop them. I didn't add one here, to avoid colliding with other open cache migrations.
  • Repositories in mise's registry whose tags are mostly lightweight with no releases now always take the GitHub API path for github-tags.