feat: dbt_cte_profile lists a model's CTEs and times one CTE with a count query

HK
1d ago
+3172-0

Issue for this PR

None (Power User feature: Profile CTEs, altimate-code side).

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Replaces the dbt_cte_profile stub with a working tool, plus a CTE finder (tools/cte-detect.ts).

  • cte-detect.ts scans the model's raw text (Jinja included) with string offsets. It skips SQL, block and Jinja comments and string literals, and handles quoted names, CTE column lists and several WITH clauses (a WITH clause that contains a nested WITH is skipped). It also builds the count query: the model's text up to the end of the chosen CTE (so dbt configs, variables, Jinja between CTEs and every earlier CTE stay as written), then SELECT COUNT(*) AS _profile_count FROM <cte>. When that cut would leave a Jinja block open (a WITH inside {% if %}), the tool refuses with a message instead of sending a query dbt cannot compile.
  • dbt_cte_profile { model, cte? }: without cte it returns the CTE list (name, line, index) and runs nothing, so a read-only call works. With cte it runs the count query through the dbt adapter (immediatelyExecuteSQLWithLimit(query, "cte_profiler_<name>", 1)) and returns { cte, queryMs, rows } in metadata, reading the count by position. A read-only call is refused with a message asking for --allow-write. model is a name, unique_id or project-relative .sql path; an ambiguous name asks for the path.
  • The plugin loops over the CTEs and computes marginal times and tiers (separate PR in altimate-claude-plugin).
  • dbt-adapter-access.test.ts (foundation) asserted every feature tool is a stub; it now skips dbt_cte_profile, which has its own test file.

This branch contains the foundation commits (feat/pu-mods-foundation); they drop out of the diff when the foundation merges.

How did you verify your code works?

bun test test/altimate/cte-detect.test.ts test/altimate/dbt-cte-profile.test.ts test/altimate/dbt-adapter-access.test.ts (fixtures and a fake adapter, no warehouse; all pass) and bun typecheck (clean). Also run for real with altimate tool run dbt_cte_profile on a DuckDB jaffle_shop project: the CTE list, the read-only refusal, one timed CTE per position (6 CTEs, about 2.4 s each), and the Jinja refusal. Demo video: /Users/harish/.quikim/artifacts/pu-mods/videos/P-03.mp4 (local file; the plugin pane part is the plugin PR's tests against a mocked tool).

Review: two independent reviewers (Claude and Codex) on the tool and detector. Fixed after review: count query kept an unclosed Jinja block, quoted $/non-ASCII names wrongly, column lists with ) in quotes or comments, comments between AS and (, nested set blocks, ambiguous manifest names, $$ strings, a query timeout now reported as timedOut. Not fixed (dialect-specific, the warehouse reports an error rather than a wrong number): backslash-escaped quotes, # comments, nested block comments, [..] inside bodies.

Facts relied on: Tool.define shape as tools/dbt-manifest.ts; adapter gate tools/dbt-adapter-access.ts dbtAdapterFor; immediatelyExecuteSQLWithLimit(query, modelName, limit) in dbt-integration 0.3.17 index.d.ts; loadRawManifest/findModel in native/dbt/helpers.ts. Port of Power User's detectCtes and buildCountQuery.

Cost side effects

A timed run is one warehouse query per CTE, and each holds the earlier CTEs of its WITH clause (a warehouse may skip those the CTE does not use), so a model of N CTEs costs N queries of growing size. It runs only on an explicit request (dbt_cte_profile with cte, from /profile-ctes or Claude), never on edit, and needs write permission. The CTE list runs no query.

Changelog

  • New dbt_cte_profile tool: list a dbt model's CTEs, or time one CTE with a row-count query.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

🤖 Generated with Claude Code

Summary by cubic

Replaces the dbt_cte_profile stub with a working tool that lists a dbt model's CTEs, or times one CTE with a count query.

  • A CTE finder scans the model's raw text (Jinja included), skipping comments and string literals and handling quoted names, column lists, dollar-quoted strings, nested WITH clauses, WITH RECURSIVE, and $-prefixed or non-ASCII names. It parses only the top-level WITH clause in order, so CTE-like text inside strings or CTE bodies no longer misdetects.
  • Without cte, the tool returns the CTE list (name, line, index) and runs nothing. With cte, it runs SELECT COUNT(*) over that CTE and the earlier ones of its WITH clause through the dbt adapter, returning elapsed milliseconds and the row count. The count query is cut from the model's own text, keeping Jinja between CTEs, and a cut that leaves a Jinja block open or inside a Jinja block tag is refused. model is a name, a unique_id (resolved through the manifest), or a project-relative .sql path; a CTE is chosen by name or position, and a model from another package or shared by several manifest entries is refused.
  • Timing a CTE costs one warehouse query per CTE, each recomputing its earlier CTEs. It runs only on an explicit request, needs write permission (a read-only timed request is refused with a --allow-write message), and respects the call's abort and timeout. Failures are reported as tool errors.
  • The branch also carries the foundation's tool stubs, AltimateApi.apiRequest, the shared dbtAdapterFor adapter gate, and the allow-list entries; those drop out of the diff when the foundation merges.

Summary by CodeRabbit

  • New Features
    • Added dbt CTE inspection to list model CTEs and profile a selected CTE with a row count and elapsed time.
    • Added several dbt and documentation tools to the available tool list; most currently return a “not implemented” response.