A dispatch's run is split into a preflight job, shards and a merge, and a gate set's shards are drawn by a salt (#611, record NNNN)
Issue #611, "A case's head is read from the private home, pinned by its hash, and the run is sharded", PR 2 of 2: the run sharded. It follows the Factory Builder's rulings (e) of 2026-11-08 and (a) to (d) of 2026-11-09. It is stacked on #662 (PR 1, the heads), so this diff is the sharding alone, and it goes to main once #662 lands.
The .github and tools/check_workflows.py edits, and the actions/download-artifact pin, are pre-approved by the Maintainer 2026-11-08, ruled by the Factory Builder. The pin is the one already in this repo's workflows, 9000827c… # v8.0.2, which run-jobs.yml uses. The ruling asked for the upload's release line, v7. Say if you want v7 instead.
What changes
- Three jobs in
eval.yml.preflightasks a gate set's preflight. A development set has none, so the job only writes an empty part.evalis a matrix of the shardsconfig/harness.jsonnames (eval_shards: 4). It needspreflight, each shard has its own time limit, andfail-fast: false.mergeneeds both and runsif: always(). It holds neither credential and has no environment. It downloads the parts and writes the one report, uploaded aseval-reportas before, soadapters/candidate.pyis unchanged.tools/check_workflows.pyholds all three, step by step.
- Which cases a shard asks.
- A development set's case
kgoes to shardk mod N. - A gate set's case goes to the shard given by
sha256(salt ‖ run id ‖ attempt ‖ id) mod N. The salt is read raw from the home (<role>/<set>.salt, 16 to 256 bytes). - A gate set with no salt stops at fetch with
no_shard_salt. I added that to #653's checklist for the seating, with the salt made asos.urandom(32).
- A development set's case
- What a part writes:
eval-part/1, a new schema.- A gate set's part holds counts alone: by kind, origin, faults, usage, models, and its preflight. For its blocking cases it holds two words, never how many (decision 0120). It also records the attempt its shard was drawn for, and its own sealed ledger, uploaded as
eval-ledger-<part>. - A development set's part keeps its report rows.
- No step's line, and no gate set's part, names a case.
- A gate set's part holds counts alone: by kind, origin, faults, usage, models, and its preflight. For its blocking cases it holds two words, never how many (decision 0120). It also records the attempt its shard was drawn for, and its own sealed ledger, uploaded as
- The merge is
harness/eval/merge.py, throughparts_ofandmerged.- A part is refused if it is missing, of another run, commit, set or part, lists a case for a gate set, holds any ledger but its own, or is a gate set's shard drawn for another attempt. Its cases are then
unanswered, a fault of the run. - The split and blocking words are said from summed counts, so the report of the whole equals a whole run's. The development-set test checks this exactly.
ledgerbecomes a list of parts,{part, salted_sha256, sealed_sha256}, with the preflight's first. The gate refuses a part named twice, a whole run's ledger beside a part's, or an empty ledger.- The run's one output,
ledger_sha256, is set by the merge step alone: the sha256 of the canonical JSON list of sealed hashes (ruling (c)).
- A part is refused if it is missing, of another run, commit, set or part, lists a case for a gate set, holds any ledger but its own, or is a gate set's shard drawn for another attempt. Its cases are then
- A run not split. With no
EVAL_PART, each step runs the whole set as before. Its ledger list has one entry,whole. - Record
NNNN, numbered at enqueue, after #662's. It amends 0170's row "When", and 0164's rows "The seal" and "The upload". It names these residuals:- per-part counts of unknown subsets, and what many runs could pool from them;
- the salt's entropy and secrecy, and how to rotate it;
- a shard re-run by GitHub's "re-run failed jobs", which leaves the run faulted.
Checks
| Check | Result |
|---|---|
SIM_SEEDS=40 make verify | OK, 1721 tests (at 7b2f64cf, on a local commit that numbers the two records) |
Filtered mutants: the 30 new and 27 refreshed ones, every one on seal.py, heldout.py, eval.yml and the eval schemas, and every one whose test this changes | 208 in the first run, its six artifacts of the run re-run; and the 41 the reads' fixes touched: every one killed |
| gitleaks | No leaks |
| Argon (eab31e1), 25 parts | 2 major, 3 minor. Fixed: a shard drawn on another attempt is now exercised through the step; one mutant killed its test for the wrong reason and now widens the shards it accepts instead; a redundant clause is removed; a job-specific workflow case is added; the test comment now matches the cases. Not changed: the workflow test's replace-all edits still name the step, which every job holds alike |
| Claude review (eab31e1) | Blocking, fixed: a part published how many blocking cases it had; now two words. Medium, fixed: shards of two attempts could merge into a pass; the merge now refuses another attempt's. Low, fixed: tests and mutants for a part of another commit and a gate part that lists a case; stale docs; a flat download layout is read |
| Claude security (eab31e1) | The same blocking and medium findings, fixed. Low, fixed: each part held to its own ledger; residuals named (origin counts, usage, the salt's entropy and rotation); "holds neither credential" worded exactly |
| Muse (73bf1e9) | Major, fixed: the merge's blocking clause had no test at its real least of one; now tested, with a mutant. Minor, fixed: a merge of no part wrote a report its schema refused; the gate now refuses an empty ledger instead |
| Astra | Only where Muse gives no read |
Until both records are numbered, CI's checks, mutants and verify fail on the NNNN records alone, as on #662.
🤖 Generated with Claude Code