perf(insights): present structured evidence before prose

Merged
IZ
24d ago
+4-4

Revenue investigations can spend their remaining turns submitting prose after reading native monetary evidence, which requires a structured claim. Put the existing revenue and retention object alternatives before prose in both finish schemas so the model sees those forms first. This is schema presentation, not a parser fix; valid structured claims already parsed correctly.

The change moves four lines in one file. It does not change accepted claim forms, native rendering, validation, publication rules, or the investigation budget.

Validation:

  • Root lint, type checks (33 tasks), and tests (29 tasks) pass; Insights has 692 passing tests and 6 skipped live tests. Focused review covered 67 claim/revenue and 41 retention checks, plus actual-schema valid/invalid objects.
  • Eight frozen synthetic trials reused two known cases with two repeats per arm. Accepted outcomes were 3/4 candidate versus 2/4 baseline; finish rejections were 3 versus 9; model turns were 17 versus 21. Full-window monetary comparisons were correct in 2/2 candidate outcomes versus 1/2 baseline.
  • Observed total time was 61.437s versus 86.940s with mocked tool I/O. This small confirmation does not establish production latency or broad quality improvement. Clipped-window handling remained inconsistent at 1/2 accepted in each arm; one candidate summary added an unsupported implication, and one accepted clipped result cited only the signal. Retention and prose behavior have compatibility tests, not live behavioral parity evidence.

The separate first-party live comparison is pending authorization and has not run. This remains a draft.

Based directly on current staging. No feature dependencies. Known shared-file overlaps: #849, #850, #851, and #751 modify Insights agent behavior; this slice only reorders the two claim unions and does not include those changes.

AI assistance: Codex implemented and reviewed the change and ran the checks and synthetic evaluations. Human review remains pending.


Summary by cubic

Reorders evidence claim schemas so structured revenue and retention objects appear before the prose variant in both finish schemas.

  • Affects only schema union ordering in apps/insights/src/agent.ts; no changes to accepted claim forms, validation, native rendering, or investigation budget.
  • Intended to reduce wasted turns where the model submits prose after seeing native monetary evidence that requires a structured claim.