feat(review): opt-in AI input coverage, hunk excerpts and a dbt manifest context card
Issue for this PR
Part of #1792
Type of change
- Bug fix
- New feature
- Refactor / code improvement
- Documentation
What does this PR do?
Stacked on #1770. Gives the AI lane better input on large PRs. All of it is opt-in: with the flags unset, prompts are byte-identical to before (snapshot test).
ALTIMATE_REVIEW_AI_COVERAGE=1:- Per-kind file quotas (models 12, macros 8, tests 4, yml/config 6), so models can't crowd out supporting files.
- Hunk-centred excerpts instead of cutting at the first 6,000 characters.
- Files not sent are listed in the coverage ledger.
ALTIMATE_REVIEW_AI_CONTEXT_CARD=1: a short dbt context card per file from the manifest. For models: materialization, unique key, parents, children, exposures, declared columns with their tests, siblings. For macros: callers and dispatch variants.ALTIMATE_REVIEW_AI_DRYRUN=1: builds the AI input and records its size without calling the model, for cost estimates.
Results (blind Claude judge)
| Set | Without | With coverage + card | Tokens |
|---|---|---|---|
| 21 tuning-repo PRs where the input changes (39 defects) | 6 caught | 10 caught | 1.32Ć |
| 25 repos never used for tuning (129 snapshots, 188 defects), with the test-gap check from the next PR (config C vs A) | 43 caught, 44% useful, 1.2% wrong | 47 caught, 48% useful, 0.9% wrong | 1.25Ć |
- The fresh-repo gain is +4 catches (95% CI ā4 to +12). That leans better but is within noise, so these stay opt-in.
How did you verify your code works?
bun test test/altimate/review: 725 pass.bunx tsc --noEmit: clean.- New cases in
test/altimate/review-ai-inputs.test.ts: quotas, redistribution, hunk windows, ledger, card rendering, default prompt snapshot.
Cost side effects
None by default; every option is opt-in and the default prompts are byte-identical. With ALTIMATE_REVIEW_AI_COVERAGE=1 and ALTIMATE_REVIEW_AI_CONTEXT_CARD=1 the AI lane's input grows with the PR's changed files (bounded by the per-kind quotas), measured at 1.25-1.32x AI-lane tokens; the number of AI calls is unchanged. The context card is built from the local manifest (no warehouse queries). ALTIMATE_REVIEW_AI_DRYRUN=1 makes no model call and records the input size for cost estimates.
Screenshots / recordings
Not a UI change.
Checklist
- I have tested my changes locally
- I have not included unrelated changes in this PR
- I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"
š¤ Generated with Claude Code
Summary by cubic
Adds opt-in flags that give the AI reviewer better input on large PRs while leaving the default prompt byte-identical (pinned by a snapshot test).
What changed
ALTIMATE_REVIEW_AI_COVERAGE=1sends per-kind file quotas (models 12, macros 8, tests 4, yml/config 6), most-changed files first, with unused quota redistributed; quotas are overridable viaALTIMATE_REVIEW_AI_QUOTA_*.- The diff section gets its own 16,000-char budget (
ALTIMATE_REVIEW_AI_COVERAGE_DIFF_CHARS) so hunks stay whole; over-budget files send ±20-line windows around changed lines with omission markers, and batches are capped at 120,000 chars per call. - For compiled SQL whose line numbers drift after Jinja expansion, windows anchor on the added lines' text instead of line numbers.
- Files left out are listed in the coverage ledger and in
scope.notReviewedas "not sent to the AI (file quota)". ALTIMATE_REVIEW_AI_CONTEXT_CARD=1adds a short dbt context card per file from the manifest: materialization and keys, parents/children/exposures, declared columns with tests, and macro callers and variants; yml and singular tests get the card of the model they test.ALTIMATE_REVIEW_AI_DRYRUN=1plans the AI input and records its size without calling the model, for cost estimates.- Per-file
ai.inputmetadata (chars, truncated, hunks sent/total, card) is recorded on the verdict envelope under these flags.
In a blind evaluation across 25 repos never used for tuning, coverage plus the context card caught 47 vs 43 defects with fewer wrong findings at about 1.25Ć the tokens; the gain is within noise, so the flags stay opt-in.
Migration
- No changes required; the flags default to off.