What the reads of #628's merge with #625 left (#612)

Merged
DO
2d ago
+84-0

A follow-on to #628 (issue 612, "The eval job reads a named set, and a gate set's report is aggregates only"). It fixes the two low findings from reading the code that #628's merges with #625 and #624 changed, after #628 landed as 78f6836e.

What changes

  • models is held as required in eval-aggregate/1. It adds:

    • a counter-example, examples/invalid/eval-aggregate.no-models.json;
    • a case in test_the_gate_passes_a_good_aggregate_report_and_refuses_a_bad_one that refuses a gate report with no models;
    • the mutant "a gate set's report need not name the models that answered", on the schema.

    eval-report/1 already had the same three, from #625.

  • Record 0164 gains a Boundaries row. The row names the answering models, each case's usage in the ledger, and the six mutants that hold them. A row is only added, as check_decisions requires. The Decision row "The ledger" stands unedited: it lists fewer fields than the ledger holds, and says nothing false.

The delta reads

ReadResult
Argon, on the merge with #625 (from the mechanical merge, conflicts as git left them, to the resolution)A clean read: 4 of 4 parts, 0 findings
Claude review, on both mergesNo correctness bug; the two low findings fixed here
Claude security, on both mergesNothing to fix. A maxItems on models was suggested as an optional backstop, and is not taken: the CLI names each model, not the model's own text
Muse, AstraNot run (quota); owed after 2026-11-09

Checks

CheckResult
SIM_SEEDS=40 make verifyOK, 1677 tests (at 4e5757e1)
Mutants: the new one and #625's on eval-report2 of 2 killed
gitleaksNo leaks

Reads of this PR (54c74fb4): Argon clean (0 findings); Claude review and security, no blocking finding, one low (the mutant filed under decision 0162's block) fixed in 4e5757e1. Muse and Astra: not run (quota), owed after 2026-11-09.

šŸ¤– Generated with Claude Code