feat: dbt lineage pane that follows the model Claude reads or edits

Merged
HK
2d ago
+5560-43

Summary

Adds a dbt lineage pane to the plugin's hooks module. It follows the model Claude reads or edits, and it checks a change's downstream reach before the edit lands. Stacked on #12 (its branch is merged in); needs AltimateAI/altimate-code#1637 (structured dbt_lineage) and AltimateAI/altimate-code#1635 (stacked on it: manifest fallback fields, column reach, large manifests). The plugin's calls are read-only, so altimate-code answers them from the compiled manifest and never starts the project's dbt adapter; the pane also reads the integration's model-lineage shape when a result carries one.

  • Pane (/lineage <model>[.column]): follows the model Claude reads, edits, or names in its own dbt_lineage call.
    • Graph view (default), like dbt-lineage-viewer: the model's whole lineage as boxes and arrows (every level up and down, every model, every name in full, dashboards marked ◆) drawn once as a canvas the pane is a window onto. Scroll it with the mouse, touchpad or arrows; pan with h/l (or ◀ ▶); f brings the model back into view. The pane opens on the model. Nothing is folded or cut. With a column picked, the models and dashboards that read it are highlighted and the rest dimmed.
    • What the answer is about: once the session has seen a dbt project, after each answer the pane lists the dbt models Claude named (IN THE ANSWER), each a press away; when Claude answered without reading a model that turn, the pane follows the first model it named, so the lineage matches the answer.
    • Views for big projects (v): ends only (the sources and dashboards the model sits between, with "via N"); folders (sources → staging → intermediate → marts → dashboards, top to bottom with counts and level spans: a source-to-mart chain on one screen); list (the earlier text view, also used in a pane under 48 columns).
    • The pane keeps the model's whole upstream (up to 500 models), so a 12-hop chain shows back to its sources.
    • No manifest: a project without target/manifest.json still gets its graph, read from the models' ref() and source() calls the way dbt renders them (Jinja only, SQL comments included, versions, snapshots, Python models). Columns, dashboards and column reach then need dbt parse.
    • Follows indirect reads: a shell command that finds the model (find … -name x.sql, a cd first) moves the pane; a listing of many files does not. From a monorepo's root, /lineage <model> finds the dbt project below it that has the model.
    • Each output column with its sources and lens type.
    • The column box: "read by N models · M dashboards". It lists models that may read the column and models that could not be traced, and the count becomes "at least N" when there are any.
    • Keys: v switches view, c picks the next column, r refreshes, a asks Claude; in the ends and list views u/d unfold upstream/downstream. Pressing a model follows it.
  • Same model only: columns from Claude's own dbt_lineage call are reused only when the answer's unique_id is the followed model's, never for a same-named model of another project.
  • Graph data: read from target/manifest.json with no model calls. A manifest over 4 MiB is read through dbt_lineage include_graph (CLI) and cached per revision. If altimate-code fails or its answer is too large to read, the pane says so rather than reporting that the model is missing.
  • Impact first: before an Edit, Write or MultiEdit of a model's SQL runs, the plugin traces every column the edit touches (up to 3, in parallel) and merges their reach, holding the edit for at most 8.5 s. A column whose check fails or times out is named, and the count becomes a lower bound. No trace starts once the hold is over, the CLI run stops at the deadline, and an edit made while the same column's trace runs shares it. It shows a toast, adds the lineage to Claude's context once per turn per agent (each project name quoted and capped, the note at most 4 KB and framed as data), and leaves an impact band above the prompt. The band shows impact_analysis severity and the direct, transitive, test and reader counts.
  • Claude's own dbt_lineage results: drawn as a list of edges colored by lens. ctrl+o keeps the raw output under it.
  • Desktop: an SVG graph.
  • Warehouse lineage: when a connected MCP server offers table_lineage or databricks_table_lineage, the pane adds what the warehouse has observed for the model's table. The plugin calls it itself, without a prompt, and sends the table's name, only for a model the person asked for (/lineage, a press, r), never on a follow of Claude's reads. The README says so.
  • Calls: go to the plugin's MCP server with no permission prompt. A result the server cut for size, or a disconnected server, falls back to altimate tool run --json.
  • Composition: the engine allows one hooks module per plugin and one unmatched hook per event, so integrations.tsx registers the lineage command, hooks and band line. runAltimate, parseJson and later move to hooks/altimate-cli.ts, shared by both halves. The CLI falls through to altimate-code only when the altimate binary is missing, not when it times out.

Test plan

  • claude plugin validate plugins/altimate-code passes.
  • claude plugin test plugins/altimate-code passes 157 tests across the graph module, the layout module, the lineage mod and the integrations pane. They cover:
    • following, the large-manifest path, recompiles, stale-focus guards;
    • impact-first ordering, context de-duplication, lower bounds;
    • CLI fallback, capped results, column switching, warehouse lineage, the result redraw;
    • the dbt integration's model lineage, same-named models of other projects, unreadable and failed CLI answers;
    • the layout: layering, whole names on an unbounded canvas, panning slices, folder order, a 500-model lineage in under 300 ms, and that no drawn line joins two models that are not linked;
    • the SQL-file graph (refs in comments, quoted text, versions, snapshots, Python, YAML forms, model-paths outside the project), shell reads, monorepo lookup, and presses staying in the project shown;
    • the impact note's quoting and size cap, no trace after the deadline, shared traces, an MCP call that never answers, an unreadable dashboards section, warehouse lineage only for models the person asked for (presses included), model-paths: ['.'], and no folder walk in a session that has seen no dbt project.
  • Reviewed by Claude and Codex as separate reviewers, each run on its own, until both explicitly approved.
  • Shell tests (6/6) and pytest (17/17) pass.
  • Ran live in Claude Code 2.1.316, using the dev altimate-code build from #1635, on a compiled dbt-duckdb demo project (shopwise: 20 models, 3 dashboards; synthetic seed data):
    • the pane follows Read, shell reads and Claude's own dbt_lineage calls;
    • /lineage fct_orders.revenue shows 7 models and 3 dashboards;
    • an edit to revenue shows the toast before it lands, and Claude's answer used the added lineage (it named the 7 readers and the 3 dashboards);
    • Claude's dbt_lineage result is redrawn.
  • Ran live on a large real project (altimate-ingestion's Snowflake dbt project: 230 models, 60 sources, chains of 12 hops, a 7.6 MB manifest read through altimate-code in about 2 s). For a mart 12 hops from its sources, folders reads sources (26) → staging (18) → intermediate (49) · 7 levels → marts (9) → mart, and ends only lists the real sources first.
  • Ran live from altimate-ingestion's repo root on a project with no manifest (transform-dbt, 50 models): /lineage stg_dbt_artifact_records draws its graph from the SQL files.
  • Warehouse lineage against a real Altimate SaaS MCP server. It is covered only by a stub, since no SaaS server is connected in this environment.

🤖 Generated with Claude Code