#153 Mutation run: a kill is confirmed, and one copy per worker

Closes #147. Closes #152.

Why. Two problems with the mutation runner:

• False kills: a mutant counted as killed when any test failed, even one failing for its own reasons under load. On #145 four mutants looked killed locally and survived in CI.
• Disk churn: every mutant copied the whole repo into a fresh directory. On 2026-10-17, local runs across several sessions wrote about 550 GB in 12 hours. The file-event service grew to 42.6 GB, swap ran out, and the Mac had a kernel panic.

What changes

• A kill is confirmed. The first failing test is run again alone on the mutant, and alone on the unbroken code in the same tree.
  • It must fail on the mutant and pass on the unbroken code.
  • If not, the whole suite runs once more on the mutant. A pass is SURVIVED; a second unconfirmed failure is FLAKY, which fails the run (a working ruling).
  • A timeout or crash stays ERROR.
• Timing-sensitive tests: six tests are marked @wall_clock and skipped during the mutation run; they still run in make verify. A test holds that the marked set is exactly the set of tests that read the clock, by file, class and method, including from time import and import time as.
• One copy per worker (#152, folded in at the Maintainer's call):
  • Each worker copies the tree once, 2.8 MB and 227 files, with no .git and no caches. It runs mutants one at a time in that copy.
  • A mutant writes only the file it breaks. However the run ends, crash or hang included, the original bytes are written back and read again before the next mutant. A copy that doesn't read back is an ERROR, and the copy is never reused.
  • Each mutated file gets its own timestamp, so no bytecode is reused across mutants.
  • At most 4 workers by default.
• Test ids: tests are discovered from the repo top (-s tests -t .), so a failing test's id can be run alone. This matches tools/verify.py.
• Records:
  • Decision 0027 records it, tools/AGENTS.md names its test, and lesson L-60 is added.
  • The shard-time comment in verify.yml and its two copies is updated. The figure is an estimate until CI shows it.

Verified:

• make verify passes (524 tests) on a plain git archive export with PATH=/usr/bin:/bin; the record was renumbered in the export only, for the 0027 gap.
• The runner's 18 tests pass, including a test that a mutant leaves no trace in the next mutant's tree, and one where a test that fails without .git comes out FLAKY.
• Five named mutants with 2 workers: 5 of 5 killed in 101 s, including the one that skips the restore.
• No full local mutation run: it is banned on this Mac until this merges. CI's six shards are the check.

Reviewed by: a fresh-context Claude review, a security pass, and AGY (Gemini 4 Argon) on 9abeffd. All their findings are fixed in 4f31038:

• the unbroken-side rerun ran in the checkout rather than an export, so a change could fake its own kills;
• a multi-test only case was untested;
• three rules had no mutant;
• the clock-mark test checked a subset;
• a stale AGENTS.md row and the time comment.

The copy-once rework in d163dba was read again by a focused Claude review and by AGY (Gemini 4 Argon), and AGY approved with no findings. The review found no high or medium bugs. Its three low findings all fail safe:

• Only the mutated file is checked: a file a test leaves elsewhere in the copy is not detected. That gives FLAKY or ERROR, never a false kill, and the suite looks hermetic today.
• A second Ctrl-C can delete the worker directory while suites still run.
• The restored timestamp is not read back; the bytes are.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Testing
  • Mutation checks now verify that failures are caused by the change being tested, reducing false positives from flaky or load-sensitive tests.
  • Timing-sensitive tests are skipped during mutation checks. Unconfirmed failures and test errors fail the run.
• Documentation
  • Updated mutation-check timing estimates to about 450 mutants per shard, with a six-minute shard estimate. Six shards are still expected to handle about 800 mutants in under 15 minutes.