What the reads of #628's merge with #625 left (#612)
A follow-on to #628 (issue 612, "The eval job reads a named set, and a gate set's report is aggregates only"). It fixes the two low findings from reading the code that #628's merges with #625 and #624 changed, after #628 landed as 78f6836e.
What changes
-
modelsis held as required in eval-aggregate/1. It adds:- a counter-example,
examples/invalid/eval-aggregate.no-models.json; - a case in
test_the_gate_passes_a_good_aggregate_report_and_refuses_a_bad_onethat refuses a gate report with nomodels; - the mutant "a gate set's report need not name the models that answered", on the schema.
eval-report/1 already had the same three, from #625.
- a counter-example,
-
Record 0164 gains a Boundaries row. The row names the answering models, each case's usage in the ledger, and the six mutants that hold them. A row is only added, as
check_decisionsrequires. The Decision row "The ledger" stands unedited: it lists fewer fields than the ledger holds, and says nothing false.
The delta reads
| Read | Result |
|---|---|
| Argon, on the merge with #625 (from the mechanical merge, conflicts as git left them, to the resolution) | A clean read: 4 of 4 parts, 0 findings |
| Claude review, on both merges | No correctness bug; the two low findings fixed here |
| Claude security, on both merges | Nothing to fix. A maxItems on models was suggested as an optional backstop, and is not taken: the CLI names each model, not the model's own text |
| Muse, Astra | Not run (quota); owed after 2026-11-09 |
Checks
| Check | Result |
|---|---|
SIM_SEEDS=40 make verify | OK, 1677 tests (at 4e5757e1) |
| Mutants: the new one and #625's on eval-report | 2 of 2 killed |
| gitleaks | No leaks |
Reads of this PR (54c74fb4): Argon clean (0 findings); Claude review and security, no blocking finding, one low (the mutant filed under decision 0162's block) fixed in 4e5757e1. Muse and Astra: not run (quota), owed after 2026-11-09.
š¤ Generated with Claude Code