Measure synthetic report stability under budgets
Report scoring had no frozen corpus for measuring how much a report moves under repeats and controlled edits. This adds seven synthetic cases with 84 variants (exact repeats, equivalent wording and code, semantic changes, transcript-only identity swaps, ASR gaps and phase assessability), the metrics that read them (score spread, rubric-band agreement, verdict flips, assessability agreement, and the identity mean shift against repeat noise), and a sequential runner for the production report transport under explicit call and token ceilings. The runner keeps a durable journal of every call, resumes without re-spending finished rows, retries a row stopped by a 429, a 503 or a failure with reported usage, and refuses every other stop. Normal report callers take the same transport path with no observer, so their requests and retry policy are unchanged.
The result is a synthetic sensitivity probe, not calibration: docs/rubric-calibration.md keeps its NOT CALIBRATED status, and the probe is listed beside the other credentialed probes in docs/provider-cost-and-degradation.md.
Verified with ./scripts/test.sh (exit 0, no skipped checks) and cargo test --lib report_eval (21 passed, 1 ignored). Each commit builds on its own. Breaking each resume, baseline, sidecar and metric rule fails its test. The credentialed pilot (report_eval_run, ignored) has not been run, so no provider stability result is claimed; tests/fixtures/report-eval/README.md has the bounded two-repeat pilot command.
Summary by cubic
Report scoring had no frozen corpus for measuring how much a report moves under repeats and controlled edits. This adds seven synthetic cases with 84 variants, the metrics that read them, and a budget-bounded runner that executes the probe against the production report transport.
- Cases cover exact repeats, equivalent wording and code, semantic changes, transcript-only identity swaps, ASR gaps, and phase assessability; metrics read score spread, rubric-band agreement, verdict flips, assessability agreement, and identity mean shift against repeat noise.
- The runner journals every call durably, pinning each attempt's failure stage (lost send, decode failure, usable answer, closed port) alongside usage and HTTP status so resume distinguishes them; it resumes without re-spending finished rows, retries rows stopped by a 429, a 503, or a failure with reported usage, and refuses other stops.
- Normal report callers take the same transport path with no observer and keep their requests and retry policy. The probe is a sensitivity measure, not calibration: both docs keep their NOT CALIBRATED status.