The gate pools every dispatch of a gate set on the commit, under the set's own bars (#613, record 0169)
Merged
DO
12h ago+990+990+012345678901234567890123456789-68-68-01234567890123456789
Issue #613, "The gate pools k dispatches on one commit and reads all of them", PR 1 of 2. It follows the Builder's ruling of 2026-11-08 (defaults 1 to 7). The preflight is PR 2.
Dormant until a gate set is named. Until config/harness.json names a gate set, the Release's gate reads pr-review as before, by its newest successful dispatch. The record states the remaining gap as a residual: a development set, pr-review (which gates v0.1.x until v2 is seated), stays the best of several runs, because re-dispatching until one succeeds is still possible there.
What changes
- Which set the gate reads.
adapters/candidate.pyfirst readsconfig/harness.jsonat the commit, through the transport.- If it names
gate_set, by one path part, the gate readsevals/<gate_set>/cases.json, which must be a gate set. - Otherwise it reads
pr-review, which must not be a gate set. - Anything else gives
eval_set_unreadable.
- Every dispatch counts (
pooled_gate).- It reads every completed dispatch of
.github/workflows/eval.ymlon the commit, oldest first, whatever it concluded. - One still running, or a run of another commit, workflow or event in the answer, gives
eval_not_run. - Any of these, for a dispatch of any set, is
eval_dispatch_faultedand closes the gate:- a conclusion other than success or failure;
- a dispatch run again (
run_attemptnot 1); - a report not read whole, or one that names its set by more than its count and sha256.
- A dispatch whose report names another set is not counted.
- It reads every completed dispatch of
- The pooled verdict (
harness.eval.pooled_verdict).- It needs at least
POOL_LEAST= 3 reports, all of them pooled; fewer giveseval_pool_short. - Each report is held whole by
aggregate_verdict:- an Engine fault, a run fault or an unreadable report is
eval_dispatch_faulted; case_failedis pooled;- any other code closes the gate by its own code.
- an Engine fault, a run fault or an unreadable report is
- Each kind's cases and passes are summed over the reports and held to the set's bar.
- The blocking word is true only if it is true in every report, and it closes nothing, as decision 0150 has it.
- It needs at least
- A gate set's own bars.
- eval-set/1 gains
bars,kind_leastandrecord. A gate set must give all three; no other set may give any. - The job writes, and the gate holds, a gate set's report to its own bars and least of a kind.
- Neither may be below
BAR_OF_A_KIND/KIND_LEAST. A manifest underevals/needs no person's approval, so it may raise the gate but not lower it. tools/check_examples.pyholds a gate set's manifest inLIVEto its record, which must containbars_said's sentence word for word. It holds an eval-set example toheld(). Nothing reads a record at run time.
- eval-set/1 gains
- Record 0169. For gate sets only, it amends decision 0150's rows that fix the bars in code or let the gate take the newest successful dispatch. It writes down four residuals:
- a development set stays the best of several runs;
- another set's faulted dispatch closes the pool;
- a deleted run leaves no trace;
- who may change a gate set:
evals/is not a protected path. Protecting it is a change toconfig/hooks.json, yours to decide.
- Rules are in
harness/eval/AGENTS.md. The cut skill says to dispatch a named gate set three times on the commit.
Checks
| Check | Result |
|---|---|
SIM_SEEDS=40 make verify | OK, 1709 tests (at cd431f04, main 46fea214 merged, record 0169) |
Filtered mutants: the 47 new ones, the 13 brought up to date, and every one on adapters/candidate.py, tools/check_examples.py, the eval-set and harness schemas and the gate's shared tests | 165 of 165 killed (two survivors in the first run, now killed by added test cases) |
| gitleaks | No leaks |
| Argon (cc2ac91f), 17 parts | 1 major, 2 minor. Not changed: "_bars_stated runs only on LIVE", because check_examples refuses any file under evals/ that is not in LIVE. Already fixed in 9d0bfcfc: a set with extra keys. Fixed in 6d53380a: one mutant dropped two checks, now two mutants, each named for what it drops |
| Claude review (cc2ac91f) | 1 high, fixed: a re-run hid a faulted attempt. 4 low, fixed: deleted runs are named as a residual, the faulted-dispatch row is widened, the example is held by held(), and zero bars are refused through the floor |
| Claude security (cc2ac91f) | 1 medium, fixed: the floor on bars and least of a kind, and the record no longer calls evals/ protected. 2 low, fixed: re-runs, and a malformed set |
| Codex (502c73b6) | 1 P2, fixed in e8f11b59: least_passed read only the ends of the split, and rounding can make a split between them need fewer passes |
| Muse | Not run (quota); owed after 2026-11-09 |
| Astra | Not run (quota); owed after 2026-11-09 |
š¤ Generated with Claude Code