feat(agent): add dbt-optimizer agent — 6-lane dbt project optimization with evals
Issue for this PR
Closes #1091
Type of change
- Bug fix
- New feature
- Refactor / code improvement
- Documentation
What does this PR do?
Adds dbt-optimizer, a 5th native primary agent that scans a dbt project for fixable issues and proposes targeted fixes with cost/impact reporting (the agent behind the planned Optimize workflow). Four parts:
1. Agent registration (agent.ts). Deny-by-default permission allowlist: read/analysis/finops tools allowed, edit and bash prompt per action, sql_execute_write denied. The deny is re-applied after the global user-config merge AND after the per-agent config merge — permission evaluation is last-match-wins, so without the second re-application agent."dbt-optimizer".permission.sql_execute_write: "allow" would have silently won (regression tests cover both paths). Named dbt-optimizer rather than optimizer because more optimizer agents are planned; no alias shim needed since nothing shipped under the old name.
2. Prompt (prompts/dbt-optimizer.txt). Encodes the taxonomy from docs/internal/2026-08-12-dbt-optimization-taxonomy-research.md: 6 detection lanes, an evidence-attribution ladder (invocation-ID/query-tag down to lineage-match, with confidence labeling), ROI-ranked triage, cost-honesty rules ("not estimable" is a valid answer; never invent dollar figures), a 4-phase loop that stops after scan for candidate selection, and an auto-fix vs propose-only boundary. Builder's self-review gains a dbt-scoped "Optimization handoff" nudge — primary agents are excluded from the task tool (task.ts filters mode !== "primary"), so the nudge is the only build-time bridge and auto-delegation of cost-incurring scans is deliberately NOT wired.
3. Verification-tool fixes. Three latent bugs in existing tools the agent's core promises depend on, found during review: the rewrite verify gate trusted equivalent: true even when the engine said decidable: false (now UNDECIDABLE = unproven, everywhere it surfaces); sql_explain analyze:true executes the statement on Postgres/MySQL/DuckDB/Trino and had no statement-class guard (now blocked for anything non-read-only, including SELECT ... INTO); the sql_diff wrapper read response fields the native handler never returns, so every comparison reported "identical".
4. Evals. Tier 1 (CI): 20 deterministic tests asserting the prompt's non-negotiable invariants (whitespace-normalized so reflow doesn't break them) plus an evidence-chain suite proving each planted fixture issue is genuinely detectable. Tier 2 (opt-in): a live eval that runs the compiled binary against a 7-model DuckDB fixture with 6 planted issues (incremental candidate, dead model, SELECT * propagation, ORDER BY, verbatim-duplicated CTE ×3, untested model) and grades deterministically — ≥4/6 recall with directional signal phrases, exit-code check, and a tree-snapshot proving the scan modified/removed/added nothing. Answer key lives outside the scanned directory.
How did you verify your code works?
- 230 tests green across the 9 affected files (agent permissions incl. both override-bypass regressions, carry-forward guards, tool fixes, prompt contract, fixture evidence chain);
tsgo --noEmitclean; upstream marker check clean (--markers --base main --strict); oxlint 0 errors on changed files. - Four external review rounds (Codex): plan review, full-diff review, and two focused verification passes — findings (permission bypass, undecidable gate, EXPLAIN ANALYZE execution, sql_diff contract, eval grading false-positives, contradictory planted issue) were each fixed with regression tests.
- NOT verified: the tier-2 live eval has not been run against a live model yet (it is opt-in and needs a compiled binary + API key); the finops-dependent scan lanes are untested against a real warehouse — the fixture exercises the static lanes only.
Screenshots / recordings
Not a UI change — the agent appears in the existing Tab ring/agent list.
Checklist
- I have tested my changes locally
- I have not included unrelated changes in this PR
🤖 Generated with Claude Code
Summary by cubic
Adds the dbt-optimizer agent for evidence-backed dbt project scans and tightens permission, SQL-safety, and path-gating so read-scoped scans cannot write or exfiltrate. Old: stored “always allow” approvals and tool exposure could bypass agent denies, EXPLAIN ANALYZE could execute writes, and out-of-project paths weren’t uniformly gated; New: non‑overridable denies hold across sessions, EXPLAIN ANALYZE requires write approval, and all analysis paths gate through a canonicalized external-directory check.
- Registers
dbt-optimizerwith deny-by-default;edit/bashask;sql_execute_writeis non‑overridable. Built-in tool exposure now honors denies viaPermission.disabled, with edit-tool remaps to keepapply_patch/patch/multiedit underedit. - Permission asks: persisted “always allow” approvals never flip a configured deny. Session rules merge safely, and the engine re-applies only the agent’s effective permission‑specific denies (not the
*catch‑all), so ceilings hold without clobbering allowlists. - SQL safety: a single-pass lexer masks literals/comments/identifiers; read-shaped calls to side‑effecting functions (incl. quoted identifiers) escalate to write; CR/CRLF normalize.
sql_explainblocks multi‑statement input and now requiressql_execute_writeapproval foranalyze:true, falling back to estimated plans on denial. sql_diff: LCS line diffs with hunk context; forwardsschema_context/dialect; returnsequivalence_assessed,decidable, and never treats UNDECIDABLE as equivalent. Engine failures no longer erase the text diff.- Path gating: dbt readers,
parse-dbt, and allschema_pathwrappers resolve paths relative to the project, canonicalize symlinks before gating, pass the same real path to reads, and propagate permission rejections. - Impact analysis: true multi‑seed BFS by dbt
unique_id; target matching acceptsunique_id; affected tests counted byunique_id. - Evals/docs: deterministic prompt/evidence tests and an isolated, opt‑in DuckDB live eval; docs list
revieweranddbt-optimizer. Equivalence abstentions (decidable:false) report UNDECIDABLE, not ERROR, even with validation errors.
Summary by CodeRabbit
-
New Features
- Added a
dbt-optimizeragent for evidence-based analysis and approval-gated fixes. - Added optimization guidance for materializations, SQL patterns, costs, testing, DAGs, and warehouse design.
- Added support for SQL dialect context in comparison and rewrite workflows.
- Added a
-
Bug Fixes
- SQL comparisons now distinguish proven, different, and undecidable results.
- Analyze operations reject unsafe SQL and require appropriate approval.
- dbt tools now validate project and schema paths.
- Protected warehouse writes and destructive operations from unauthorized access.
-
Documentation
- Added guidance for agent scaling and dbt optimization workflows.
[!NOTE]
High Risk
Touches permission evaluation (deny vs stored “always” approvals), SQL write classification, EXPLAIN ANALYZE execution, and external path gating—security-critical warehouse and filesystem surfaces.
Overview
Adds dbt-optimizer, a fifth primary agent that scans a dbt project in four phases (read-only scan → user-selected fixes → impact report → optional PR) with a deny-by-default allowlist: analysis/finops tools allowed, edit/bash ask, and sql_execute_write denied non-overridably even after user/agent config merge. Builder’s self-review now hands off optimizer-shaped issues instead of fixing them out of scope.
Hardens the permission and tool surface the agent depends on. Stored “always” approvals no longer override a configured deny. Path-taking dbt/core tools go through a shared external_directory gate (symlink-canonicalized). schema_index and training save/remove prompt before mutating persistent state.
SQL safety: a lexer-based masker drives write classification (side-effect functions, comment/literal bypasses, CR line endings). sql_explain analyze:true is limited to a single read-only statement and requires sql_execute_write. Equivalence/rewrite/sql_diff now distinguish UNDECIDABLE from proven equivalent, use LCS diffs, and no longer treat engine abstention as a false safety claim. Impact analysis traverses by dbt unique_id so package-colliding names are not dropped. Docs list Reviewer and dbt-Optimizer alongside the existing modes.
Reviewed by Cursor Bugbot for commit 54d2df677d85fbf4772f2d1cb7100e08140bc9fe. Bugbot is set up for automated code reviews on this repo. Configure here.