The context's task is held by its sentence's sha256, and a run records whether it added a test, a mutant and the code, never what (#8)
The Maintainer chose hash only for the context's test: the task's rule is named nowhere in this repo, even once the runs are recorded. Hashing the sentence alone would still have named the rule in every run, through the reason code and the names the agent chose for its test and mutant. So, as the Builder ruled (option b), a run now records whether it added each part, never what.
The record (evals/context/task.json, context-eval/1):
taskbecomestask_sha256: 64 hex, or null until the runs. The sentence is never here.- Each run gains
task_sha256, the sentence it ran on, so the test holds that the newest three ran on the task's hash. addedbecomes three booleans: a test added, a mutant added, and the sentence's reason code on a line the agent added to the Kernel, as a whole word. No name the agent chose and not the code.- A command keeps only its program, its flags and the paths or modules the export as made holds (and a few fixed words such as
unittestandverify), sogrep -rn <code> kernelis recorded asgrep -rn. - A file the export as made does not hold, made or only named, is read and changed under its deepest folder that the export holds, or
./at the top. An existing file the agent changed keeps its path: it is the repo's own name and says where, not what, which the Builder accepted and 0129 says. - The schema stays
context-eval/1: no run has ever been recorded, and the file held only nulls.
The runner (tools/context_eval.sh):
- It takes the sentence's sha256 of its one line, with no newline, and passes it to the assembler with the code. The code is used only to judge at the terminal, and never written.
checked.jsoncounts the agent's mutants instead of naming them, andrequests.jsonsays the sentence is carried as its sha256 alone. Nothing in OUT or on the terminal names the sentence, its code, or the agent's test or mutant; a test holds that.- The stale "trial P-26 has no fixture" lines now say what P-26 found: the CLI's sandbox hands a tool the parent's environment.
The tests:
- The
@pendingtest now asserts the two hashes, the same sentence hash on each of the newest three runs, andaddedall true. The backticked-code check is gone, and the test stays pending, since the runs still wait on the private session and the Maintainer's terminal. - The counter-example "a task of two sentences" becomes "the task in words": the sentence where its hash belongs is rejected.
- Seventeen mutants for these rules, each killed; counter-examples of a run that names its test and of a run with no sentence's hash are rejected.
Record. 0042's rows change: the sentence's place, what a run records, and its P-26 line. Record 0129 replaces it and restates the rest. #508's 0128 is on main and this branch is up to date with it, so the numbering is whole. It is not queued while the Builder holds merges for the first eval and cut.
Nothing here reads, names or guesses the held-out rule; the examples use the built empty_diff. Nothing in kernel/ is touched.
Part of #8
Reviews
Four reads at f0d8d58c, applied in b3d6df89 and c134fbd9:
| Read | Findings | Disposition |
|---|---|---|
| Security pass | blocking: ran keeps the code when a command names it; major: a new file named after the rule is recorded by its path | Fixed: a command keeps only its program, flags and the export's own names; a new file is recorded by its folder. Tests plant both, with mutants |
| Claude review | major: the same leak through ran and new files; minor: the lookbehind of the whole-word check is untested; info: the pending test cites 0042; info: an orphan comment and a schema title that holds booleans to a pattern | Fixed, each with a mutant where it is a rule |
| Muse | blocking: the same leak through ran; minor: no case holds added.test apart from added.mutant; minor: a code inside a longer word with a digit or capital counts | Fixed: a test alone and a mutant alone are each held; the whole-word check now refuses a letter, digit or underscore on either side |
| Argon (gemini-4-argon-high), 9 findings | major: the leak through paths and words (two findings); major: an invalid code ("Empty Diff") passed without the pattern check; minor: no counter-example holds the runs' new fields; minor: the orphan comment; minor: the lookbehind | Fixed: an invalid or empty code is planted in the Kernel's line and is not added, with a mutant; two counter-examples added |
| Argon | major: a run that read outside the export, or ran an unreadable command, is counted and not failed | Not changed here: 0042's rule, restated by 0129 unchanged. The shell cannot be sandboxed at the pin (P-26), and counting it was the ruling. Raised with the Builder as a follow-up |
| Argon | major: tools/mutants.py is not put back before the judge, and it holds the mutation runner as well as the list | Not changed here: 0042's rule, restated unchanged. Real, and raised with the Builder as a follow-up: the runner could be put back while the agent's list is kept |
| Argon | major (PR-5): the newest-three and boolean rules are held only by the pending test | Declined: that test is the check, and it cannot run before the runs exist; 0042 stood the same way. The record's shape and every assembler rule have active tests and mutants |
š¤ Generated with Claude Code