feat(review): opt-in AI input coverage, hunk excerpts and a dbt manifest context card

RA
44m ago
+2175-22

Issue for this PR

Part of #1792

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Stacked on #1770. Gives the AI lane better input on large PRs. All of it is opt-in: with the flags unset, prompts are byte-identical to before (snapshot test).

  • ALTIMATE_REVIEW_AI_COVERAGE=1:
    • Per-kind file quotas (models 12, macros 8, tests 4, yml/config 6), so models can't crowd out supporting files.
    • Hunk-centred excerpts instead of cutting at the first 6,000 characters.
    • Files not sent are listed in the coverage ledger.
  • ALTIMATE_REVIEW_AI_CONTEXT_CARD=1: a short dbt context card per file from the manifest. For models: materialization, unique key, parents, children, exposures, declared columns with their tests, siblings. For macros: callers and dispatch variants.
  • ALTIMATE_REVIEW_AI_DRYRUN=1: builds the AI input and records its size without calling the model, for cost estimates.

Results (blind Claude judge)

SetWithoutWith coverage + cardTokens
21 tuning-repo PRs where the input changes (39 defects)6 caught10 caught1.32Ɨ
25 repos never used for tuning (129 snapshots, 188 defects), with the test-gap check from the next PR (config C vs A)43 caught, 44% useful, 1.2% wrong47 caught, 48% useful, 0.9% wrong1.25Ɨ
  • The fresh-repo gain is +4 catches (95% CI āˆ’4 to +12). That leans better but is within noise, so these stay opt-in.

How did you verify your code works?

  • bun test test/altimate/review: 725 pass.
  • bunx tsc --noEmit: clean.
  • New cases in test/altimate/review-ai-inputs.test.ts: quotas, redistribution, hunk windows, ledger, card rendering, default prompt snapshot.

Cost side effects

None by default; every option is opt-in and the default prompts are byte-identical. With ALTIMATE_REVIEW_AI_COVERAGE=1 and ALTIMATE_REVIEW_AI_CONTEXT_CARD=1 the AI lane's input grows with the PR's changed files (bounded by the per-kind quotas), measured at 1.25-1.32x AI-lane tokens; the number of AI calls is unchanged. The context card is built from the local manifest (no warehouse queries). ALTIMATE_REVIEW_AI_DRYRUN=1 makes no model call and records the input size for cost estimates.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

šŸ¤– Generated with Claude Code


Summary by cubic

Adds opt-in flags that give the AI reviewer better input on large PRs while leaving the default prompt byte-identical (pinned by a snapshot test).

What changed

  • ALTIMATE_REVIEW_AI_COVERAGE=1 sends per-kind file quotas (models 12, macros 8, tests 4, yml/config 6), most-changed files first, with unused quota redistributed; quotas are overridable via ALTIMATE_REVIEW_AI_QUOTA_*.
  • The diff section gets its own 16,000-char budget (ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS) so hunks stay whole; over-budget files send ±20-line windows around changed lines with omission markers, and batches are capped at 120,000 chars per call.
  • For compiled SQL whose line numbers drift after Jinja expansion, windows anchor on the added lines' text instead of line numbers.
  • Files left out are listed in the coverage ledger and in scope.notReviewed as "not sent to the AI (file quota)".
  • ALTIMATE_REVIEW_AI_CONTEXT_CARD=1 adds a short dbt context card per file from the manifest: materialization and keys, parents/children/exposures, declared columns with tests, and macro callers and variants; yml and singular tests get the card of the model they test.
  • ALTIMATE_REVIEW_AI_DRYRUN=1 plans the AI input and records its size without calling the model, for cost estimates.
  • Per-file ai.input metadata (chars, truncated, hunks sent/total, card) is recorded on the verdict envelope under these flags.

In a blind evaluation across 25 repos never used for tuning, coverage plus the context card caught 47 vs 43 defects with fewer wrong findings at about 1.25Ɨ the tokens; the gain is within noise, so the flags stay opt-in.

Migration

  • No changes required; the flags default to off.