Audit: add gallery geometry evidence validator

Merged
TE
9d ago
+791-0

Adds a shared geometry validator so gallery audits fail on missing or malformed evidence and report measurable alignment, spacing, inset, text and hover-highlight violations. Findings include numeric deltas and bounded screenshot crops; accepted debt requires a non-empty explanation and a non-negative limit.

Leo’s ask (2026-10-28): “I've generally been feeling like a lot of our alignment stuff, like, I don't know, I feel like you guys could be doing more audits of the app. Like there's some amount of, like, satisfying and pretty stuff that can happen that we're not quite taking full advantage of.”

The validator binds captures to the full configured revision identity, verifies PNG hashes and dimensions, and preserves structured identities for findings and capture tuples. Native evidence requires AX or view frames; web evidence requires DOM rects. Individual-control rules require one element by default, while peer comparisons require two.

“How geometry is measured” in scripts/gallery-matrix/README.md documents the durable contracts: normalize to CSS pixels before comparison, use four-edge containment for highlights, reject non-boolean evidence flags, and keep regressions for the old failures. Raw screenshots remain the crop source. Gallery collection and workflow integration follow in #26504 and #26557.

Validation at c96e824dcd894756a88840bdbe6bd53b7b0dacd2:

  • Focused fleet run 40653966316: 20 tests passed, 119 assertions; snapshot a83faff40196cdc18567884452774a15b2aa2eec.
  • Regressions cover sources, booleans, finite numeric font weights, zero geometry tokens, debt limits, rule cardinality, scale normalization, four-edge containment, full revision parity, unambiguous identities and full-page screenshots.
  • Static pre-push checks passed. No runtime UI changes.

Review (codex gpt-6-astra high): SHIP at c96e824dcd894756a88840bdbe6bd53b7b0dacd2. No blocking findings; the independent reviewer could not run Bun in its environment, so the fleet test receipt above supplies execution coverage.

CI reconciliation: gallery run 40653926815 has the same failing cases and errors as base run 40650194830. Release-notary 40653927097 matches base 40651648247 by test and error: test_nightly_sparkle_key.sh requires the appcast helper, and Workflow.test_the_appcast_comes_from_the_tested_plist raises ValueError: substring not found. Guard status propagates those failures. No failure is attributable to this change.

Agent session: agent session