The context's task is held by its sentence's sha256, and a run records whether it added a test, a mutant and the code, never what (#8)

Merged
DO
7d ago
+389-114

The Maintainer chose hash only for the context's test: the task's rule is named nowhere in this repo, even once the runs are recorded. Hashing the sentence alone would still have named the rule in every run, through the reason code and the names the agent chose for its test and mutant. So, as the Builder ruled (option b), a run now records whether it added each part, never what.

The record (evals/context/task.json, context-eval/1):

  • task becomes task_sha256: 64 hex, or null until the runs. The sentence is never here.
  • Each run gains task_sha256, the sentence it ran on, so the test holds that the newest three ran on the task's hash.
  • added becomes three booleans: a test added, a mutant added, and the sentence's reason code on a line the agent added to the Kernel, as a whole word. No name the agent chose and not the code.
  • A command keeps only its program, its flags and the paths or modules the export as made holds (and a few fixed words such as unittest and verify), so grep -rn <code> kernel is recorded as grep -rn.
  • A file the export as made does not hold, made or only named, is read and changed under its deepest folder that the export holds, or ./ at the top. An existing file the agent changed keeps its path: it is the repo's own name and says where, not what, which the Builder accepted and 0129 says.
  • The schema stays context-eval/1: no run has ever been recorded, and the file held only nulls.

The runner (tools/context_eval.sh):

  • It takes the sentence's sha256 of its one line, with no newline, and passes it to the assembler with the code. The code is used only to judge at the terminal, and never written.
  • checked.json counts the agent's mutants instead of naming them, and requests.json says the sentence is carried as its sha256 alone. Nothing in OUT or on the terminal names the sentence, its code, or the agent's test or mutant; a test holds that.
  • The stale "trial P-26 has no fixture" lines now say what P-26 found: the CLI's sandbox hands a tool the parent's environment.

The tests:

  • The @pending test now asserts the two hashes, the same sentence hash on each of the newest three runs, and added all true. The backticked-code check is gone, and the test stays pending, since the runs still wait on the private session and the Maintainer's terminal.
  • The counter-example "a task of two sentences" becomes "the task in words": the sentence where its hash belongs is rejected.
  • Seventeen mutants for these rules, each killed; counter-examples of a run that names its test and of a run with no sentence's hash are rejected.

Record. 0042's rows change: the sentence's place, what a run records, and its P-26 line. Record 0129 replaces it and restates the rest. #508's 0128 is on main and this branch is up to date with it, so the numbering is whole. It is not queued while the Builder holds merges for the first eval and cut.

Nothing here reads, names or guesses the held-out rule; the examples use the built empty_diff. Nothing in kernel/ is touched.

Part of #8

Reviews

Four reads at f0d8d58c, applied in b3d6df89 and c134fbd9:

ReadFindingsDisposition
Security passblocking: ran keeps the code when a command names it; major: a new file named after the rule is recorded by its pathFixed: a command keeps only its program, flags and the export's own names; a new file is recorded by its folder. Tests plant both, with mutants
Claude reviewmajor: the same leak through ran and new files; minor: the lookbehind of the whole-word check is untested; info: the pending test cites 0042; info: an orphan comment and a schema title that holds booleans to a patternFixed, each with a mutant where it is a rule
Museblocking: the same leak through ran; minor: no case holds added.test apart from added.mutant; minor: a code inside a longer word with a digit or capital countsFixed: a test alone and a mutant alone are each held; the whole-word check now refuses a letter, digit or underscore on either side
Argon (gemini-4-argon-high), 9 findingsmajor: the leak through paths and words (two findings); major: an invalid code ("Empty Diff") passed without the pattern check; minor: no counter-example holds the runs' new fields; minor: the orphan comment; minor: the lookbehindFixed: an invalid or empty code is planted in the Kernel's line and is not added, with a mutant; two counter-examples added
Argonmajor: a run that read outside the export, or ran an unreadable command, is counted and not failedNot changed here: 0042's rule, restated by 0129 unchanged. The shell cannot be sandboxed at the pin (P-26), and counting it was the ruling. Raised with the Builder as a follow-up
Argonmajor: tools/mutants.py is not put back before the judge, and it holds the mutation runner as well as the listNot changed here: 0042's rule, restated unchanged. Real, and raised with the Builder as a follow-up: the runner could be put back while the agent's list is kept
Argonmajor (PR-5): the newest-three and boolean rules are held only by the pending testDeclined: that test is the check, and it cannot run before the runs exist; 0042 stood the same way. The record's shape and every assembler rule have active tests and mutants

šŸ¤– Generated with Claude Code