feat(review): incremental and snapshot semantics that fit any dbt project
Issue for this PR
Closes #
Stacked on #1724 (← #1723 ← … ← #1719); merge in that order. Draft: the new rules are not yet measured on a public-repo benchmark.
Type of change
- Bug fix
- New feature
- Refactor / code improvement
- Documentation
What does this PR do?
dbt incremental and snapshot semantics. Rules follow dbt-core behavior, which is the same on every adapter. The one adapter-dependent rule reads the capability table and stays silent where the behavior isn't established. Config comes from the manifest (project, yml and inline, as dbt merges them), with the head's inline {{ config() }} on top. Every finding is advisory and never blocks.
Rules
| Rule | Fires when | Notes |
|---|---|---|
| History plan (suggestion) | An existing incremental model gains a column (old rows get NULL) or its is_incremental() filter changes, and the PR doesn't say how existing rows are handled | Tokens are compared, so formatting never counts |
partition_by change | Changed on an incremental model | Needs a full refresh |
on_schema_change='fail' | Columns are added or removed | The first incremental run after merge fails |
| Snapshot tracking change | check_cols or strategy changes on an existing snapshot | .sql and dbt 1.9+ yml snapshots |
| Load-timestamp dedup (suggestion) | row_number() ordered only by loaded_at / ingestion_timestamp | Loader sync columns (_fivetran_synced, _airbyte_*, _sdc_*) are not flagged |
Per-slice unique_key (opt-in: incrementalSliceVars) | A delete+insert/merge model filtered by a listed var (e.g. a tenant) lacks that column in unique_key | Otherwise one slice's run deletes or overwrites another's rows |
| Trino non-atomic rebuild (suggestion) | on_table_exists='replace' with an enforced contract, or drop | replace + contract runs create or replace then insert; drop removes the table during the build. Checked against the dbt-trino 1.10.6 source |
unique_key and strategy changes are left to the engine's DBT011/DBT012.
Also in this PR
- Dialect-pack rules (
BigQuery: …,Snowflake: …,Redshift: …,Databricks: …) fire only on their own engine once the project's engine is known. A DuckDB project was getting BigQuery partition advice. - Column-set rules now see literal and function-only columns (
current_date as load_date). Column lineage only reports columns that read a source column.select-list.tsadds the final SELECT's own names: an item counts only when it's aliased after a complete expression or is a bare column, and*defers to lineage.
Changed after an adversarial generality review, before opening. The goal was rules that fit any dbt project, not one monorepo:
- the history rule no longer fires on every incremental edit
- the slice rule is opt-in
- a Trino/Athena microbatch-concurrency rule was removed: dbt-trino's concurrency default is changing, and the table format can't be known from the adapter
- loader sync columns are no longer treated as load timestamps
How did you verify your code works?
- New tests:
test/altimate/review-incremental-semantics.test.tscovers the generic rules on two adapters, formatting-only edits, the opt-ins, yml snapshots and orchestrator wiring.test/altimate/review-select-list.test.tscovers the select-list names. - Suites: 495 tests across the review files pass,
bun typecheckis clean, and the real-world corpus is unchanged (15/15 caught, 0/5 false positives). - Local end-to-end run. A real dbt 1.10 DuckDB project, released 0.12.10 vs this stack. A snapshot
check_colschange and adding a column underon_schema_change='fail'are reported; 0.12.10 reports neither. A model reading a tenant var with noincrementalSliceVarsconfigured stays quiet. - Replay of 7 real PRs from a private dbt monorepo. 0 off-diff findings, no earlier catch lost. One new true positive: a contracted mart on Trino with project-level
on_table_exists: replace.
Cost side effects
None. No model calls or warehouse queries.
Screenshots / recordings
N/A (CLI).
Checklist
- I have tested my changes locally
- I have not included unrelated changes in this PR
- I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"
🤖 Generated with Claude Code
Summary by cubic
Adds dbt incremental and snapshot semantics to PR review as advisory findings, following dbt-core behavior that is the same on every adapter. Config is read from the manifest (project, yml, inline, as dbt merges them) with the head's inline {{ config() }} on top; formatting-only edits never count as changes.
Rules fire when an incremental model gains a column or its is_incremental() filter changes without a stated plan for existing rows, partition_by changes, on_schema_change='fail' meets added or removed columns, a snapshot's check_cols/strategy or query changes, or dedup orders only by a load timestamp. Per-slice runs are opt-in via incrementalSliceVars, and each must include the slice column in unique_key. On Trino, on_table_exists='replace' with an enforced contract and drop get a suggestion because readers can see an empty or missing table. Adapter-dependent advice reads the capability table and stays silent where behavior isn't established, dialect pack rules fire only on their own engine, and column-set rules now see literal and function-only columns (current_date as load_date) via the final SELECT's own names. Also fixes a false "table not found" warning against a manifest-only schema (category → kind).