A boundary's check names a test, never a file, and a new or changed row names the mutant its test kills (#590)

Merged
DO
2d ago
+594-22

The Factory Builder's story for #590, "A boundary's check names a test, never a file, and a new boundary names the mutant its test kills". Record NNNN, numbered at enqueue. Not to be enqueued during the v0.1.0 hold.

What changes

  • A test is named as a test. tools/check_context.py's _held refuses a check under tests/ that is a file rather than path::name.

    • It holds wherever a rule names its check: a decision's Boundaries, an AGENTS.md rule, a lesson's Check, a standard's Held by:.
    • The issue counted 2 bare rows, in decision records. _held found 5, and each now names the test that holds it:
      • docs/decisions/0008-verification-binds.md, row 5;
      • decision 0049, row 17;
      • the root AGENTS.md's simulator rule;
      • tools/AGENTS.md's hook rule;
      • lesson L-25.
  • A new or changed row names its mutant. tools/check_decisions.py compares each record's Boundaries with the merge base, as it already does. A row new in the change, or one whose Check changed, must name mutant: "<description>". That mutant must:

    • be in tools/mutants.py;
    • change a file the record names in backticks, not under tests/;
    • list among its killers a test the row names. A killer counts by its whole id, or by module and name where the row gives no class; a module or a class does not count.
  • What is not a changed row.

    • A row main holds, as it stands.
    • A row whose only change is that a bare test file is now named by a test of that file, so the 5 fixes above stay rows main holds.

    Any other change to a Check, including a changed mutant, is a changed row.

  • What is exempt. A row that waits on a planned part, or whose test is still @pending.

  • Cost. The mutants are read once per run, by importing the tree's tools/mutants.py (under 0.2 s), and only when a new or changed row needs them. check_context, which the hooks run after each edit, reads no mutant.

  • verify prints boundary mutants: N of M boundary rows name no mutant. On main that is 1501 of 1501.

  • This record's own rows name 18 new mutants, so the record passes its own rule.

Accepted records

The fixes to 0008 and 0049 change a row's Check cell only. tools/check_decisions.py already allows that ("a row's Check cell may change"), so no record is replaced and no row is added.

The reads of fe56d5f, and the one fix round, a523b9d

ReaderFindingsDisposition
Claude security2 major, 1 minorAll fixed. A mutant a Boundary row names is run against its listed tests alone, never the whole suite, and each of those tests is one the row names. A row that names a mutant is read again on every run. A wait or "not held yet" exempts a row only if every test it names is @pending
Claude review2 major, 6 minor, 1 infoFixed: a rule that stops waiting or pending is a changed row; the module and depth matching is tested; the resolved path is judged; verify's notes are a function and tested; the mutant's file must be named in the row; duplicate descriptions are refused; the record's wording on cost is corrected. Not fixed here: two rows hold only part of their rule, 0049's seventeenth and the hook rule. Both named a whole file before; follow-on #595
Argon7 major, 2 minorAll fixed: the same points, plus a function that is not a test, and a record parse refuses, which has no rows
MuseNot run (quota)Owed
Astra, in Muse's place5 majorAll fixed. Two were new: a class named like a test, and a file path written another way
Codex2 P2Both fixed, replied to and resolved

Verify at a523b9d

  • Mutants: every mutant of tools/check_decisions.py, tools/check_context.py and tools/verify.py, and the run-alone mutant of tools/mutants.py, 66 of 66 killed. The issue's own mutants are 34 of them.
  • Full make verify (SIM_SEEDS=40): 1633 tests OK (8 expected failures), and every check passes.
    • Run with the record numbered 0150 in the working tree only. #592 now holds 0150 on main, so this record takes a later number at enqueue.
    • Committed with NNNN: checks, verify and the mutant shards fail on the number alone, until it is given.

Follow-on for other open branches

Once this merges, any open PR that adds a record must name a mutant in each of its rows. The bar PR at 40674617 is one.

šŸ¤– Generated with Claude Code