A Run is given no part of its case's answer, and a head holds no .git (#614)
Merged
DO
2d ago+157+157+012345678901234567890123456789-1-1-0123456789
Closes #614.
What changes
tests/test_evals.py::test_a_run_is_given_no_part_of_its_cases_answer_and_its_head_no_historyruns a case whoseexpectedholds a marker through the eval job, as the workflow runs it (plan, fetch, run). At the CLI's call it reads every file in the head the CLI starts in. It finds the marker, and the answer whole, in none of: the Taskharness/eval/run.pybuilds, the prompt, or any head file. It also finds no.gitthere.harness/head.pyrefuses an archive with a member named.git, at the top or below, in any case of its letters. The eval's head of such an archive ishead_unreadableand asks nothing; the test holds that too.- This goes beyond the issue's "done when". The eval's head is not held to its commit's tree, as a live Run's is (decision 0142). Without the refusal, the test's
.gitassertion would test nothing. - Git never archives
.git, so no real head is refused.
- This goes beyond the issue's "done when". The eval's head is not held to its commit's tree, as a live Run's is (decision 0142). Without the refusal, the test's
- Mutants: "the Task carries the case's answer" (the issue's), "the prompt carries the case's answer", "the head may hold git's folder", "git's folder is refused only as git names it". All are killed by the tests they list.
- Record NNNN (number to be given).
No protected path is touched.
Verify
Pending.
Reviews
| Read | State |
|---|---|
| Claude + security | pending |
| Argon | pending |
| Muse | owed (quota until after 2026-11-09); Astra stands in |
| Codex | automatic on push |
š¤ Generated with Claude Code