#527 Harness: the review is given each head line's number, and the bound counts the diff numbered (record 0137)

Eval run 41718110053 on 746603e9 passed 22 of 40; the finding kind failed with location_differs ×7, finding_not_raised ×7, finding_elsewhere ×3 and the blocking case. The Builder asked for the Role's side of any contract mismatch with what tools/eval_result.score matches. Path form, citing, category and severity agree with the scorer. The lines did not: the model was left to count each head line from its hunk's header, inside a diff carried as one JSON string, and on a long diff it counted one off.

The change (record 0137).

• harness/task.py's numbered() leads each line of a hunk the head's file holds, a context line or an added one, with its number in the head's file and |, as 87|+ if not items:.
  • A removed line, git's note that a file has no last newline, and every line outside a hunk stay as git wrote them.
  • A hunk ends when its header's counts are spent, or at a line no hunk holds, so the next file's --- a/ is never read as a removed line.
• The Task's diff text is numbered. Its id is still diff:<content id>, and the bytes are still held to the admitted content before they are numbered.
• One measure: adapters/pull.py counts characters with the same two functions, carried(numbered(text)), and the Task's budget counts each file's part the same way.
  • Numbered file by file, a diff is numbered as it is whole, so the Kernel's bound and the Task's cut stay one (decision 0065).
• roles/pr-review/prompt.md says what the numbers are: the head's line, and none on a removed line.
• The schemas' descriptions of the count say "numbered".

Evidence. I ran the eval's own Task and argv with the local CLI (2.1.317, model opus) on synthetic cases built from this repo's own files, each with one logic error planted at a known head line. "At the line" means the run's PR-3 Finding overlapped the planted line.

Diff	Raw	Prompt rule alone	Numbered	This PR
~30 lines, 3 cases	5/5 at the line
100–140 lines, 2 cases	4/6 (2 one off)	5/6	6/6	6/6 (and 6/6 on the short ones)

• Cost: the numbering lengthens the diff by 5.8–8.0% on the public examples, and by 4.4% over 42 of this repo's last 60 diffs on main (8.0% at most). The bound's cost to the held-out set was checked privately, and no case is affected.
• Limits: the samples are small and synthetic, and were not run at the pin. Nothing ties any held-out code to this. finding_not_raised and the blocking case read as judgement. The next eval run measures it.

Tests and mutants.

• test_each_line_the_head_holds_is_given_its_number_there holds the rendering: each count, the note, empty context lines, next-file headers, and file by file equal to whole.
• The issue-327 test test_the_diff_is_counted_as_the_harness_counts_the_task_it_carries now compares the port's count with the Task's numbered text. Mutants that drop the numbering from either side, the port or the Task, are killed by it.
• test_a_finding_at_the_line_its_task_numbers_is_where_the_scorer_looks runs the public example through the eval's reviewer and the score. A Finding placed at the line its Task numbers (87) passes, and the prompt says what the numbers are.
• 23 mutants of the numbering, and the port's re-aimed count mutant, all killed.

Record 0137 follows 0133–0136, so the context check reports a gap until they land. This queues after #516.

Nothing here reads, names or quotes a held-out case. Nothing in kernel/ is touched.

Reviews

Four reads at 48652a2a, applied in 38308308 and 020afafb:

Read	Findings	Disposition
Claude review	major: splitlines breaks a row at a form feed, U+2028 and the rest, shifting the numbers; minor: the \r\n empty context line untested	Fixed: a row ends at \n alone; both held by cases and mutants
Security pass	major: a forged header past such a break with 5000 digits makes the port's count raise ValueError, failing every pass; minor: an author's text can forge the numbers the prompt tells the review to trust	Fixed: rows at \n alone, a header's numbers at most nine digits; the port test feeds the forged header
Muse	major: the same splitlines fault	Fixed as above
Argon (gemini-4-argon-high, 2 parts)	major: the prompt's placing rule not asserted; major: the same splitlines fault; major: a hunk's two sides not held to their own counts; minor: the \r mark untested	Fixed: the placing rule asserted, with a mutant; each side held to its own count, a case and a mutant for each; the rest as above

After the reads.

• SonarCloud python:S8786 on ROW: it is now [^\n]+\n?|\n, whose alternatives never overlap, and it gives the same rows. A boundary assertion on the rows, and two mutants, are killed. Sonar shows 0 open issues at 7113624e.
• Merged with main at a3b397f5, after records 0133–0136. Verify on an export found four faults in record 0137, now fixed:
  • two examples were named without examples/;
  • tools/eval_result.score was read as a path;
  • a Limits section is not one a record holds, so it moved under Why.
• make verify passed at d2fe2eb8, run with /opt/homebrew/bin on PATH for timeout; the ReadConfinementTrial PATH fix is its own PR.

🤖 Generated with Claude Code