fix: bubble real dbt show error instead of generic "Could not parse"

Merged
SA
5mo ago
+680-86

PINEAPPLE

Closes #932

(Supersedes #931 — same commits, branch renamed to drop the stale Jira key.)

Summary

altimate-dbt execute --query "..." (which calls execDbtShow in packages/dbt-tools/src/dbt-cli.ts) swallows the real error from dbt show and surfaces a misleading generic message:

{
  "error": "Could not parse dbt show output in any format (JSON, heuristic, or plain text). Got 0 JSON lines.",
  "fix": "Run: altimate-dbt doctor"
}

But running dbt show directly produces a clear actionable error, e.g.:

Runtime Error: Failed to read package: No dbt_project.yml found at expected path dbt_packages/dbt_utils/dbt_project.yml

Agents read "could not parse" as transient and retry alternate commands instead of bailing out — burns budget on a project that is structurally broken.

Root cause

packages/dbt-tools/src/dbt-cli.ts has two adjacent catch blocks in execDbtShow that swallow the run() rejection silently:

// ~line 224 — Tier 1 JSON attempt
try { const { stdout } = await run(args); lines = parseJsonLines(stdout) }
catch { lines = [] }            // ← discards err.stderr

// ~line 298 — Tier 3 plain-text fallback
try { ... await run(plainArgs) ... }
catch { /* fall through */ }    // ← also discards err.stderr

throw new Error("Could not parse dbt show output in any format (JSON, heuristic, or plain text). ...")

When execFile rejects with a non-zero exit, the error object carries .stderr and .stdout. Both catches discard them. The generic "Could not parse" message was designed for the case "dbt exited 0 but the output is unparseable" — but it currently fires for the "dbt actually crashed" case as well, conflating two very different failure modes.

Fix

  • Capture the execFile rejection on both tiers (primaryRunError, plainRunError).
  • Recover any partial JSON log lines from the failed run's stdout — dbt with --log-format json emits structured level: "error" events before exit.
  • Before the generic throw, call extractDbtError(...) which picks (in order): structured error event > primary stderr > plain stderr > exception message.
  • If extractDbtError returns anything, surface it as dbt show failed: <real msg>. Fall through to the generic message only when both runs exited 0.

Net: ~45 lines in dbt-cli.ts (helper + 2 small guard sites), 6 new test cases.

Scope

Limited to execDbtShow. execDbtCompile and execDbtCompileInline share the same catch { lines = [] } pattern but have additional fallback paths (manifest.json for compile, no-args plain-text retry for inline) that reduce caller impact. Worth addressing in a follow-up PR if telemetry shows masking there too — kept out of this PR to keep the diff focused.

Test plan

  • bun test test/dbt-cli.test.ts → 24/24 pass (18 pre-existing + 6 new)
  • Generic "Could not parse" message still surfaces when dbt show exits 0 but emits unparseable output (regression guard preserved)
  • Real stderr from a crashing dbt show surfaces in the thrown error
  • Structured level: "error" event preferred over raw stderr when both present
  • spawn ENOENT (no dbt binary) surfaces clearly
  • Reviewer: smoke against a real project with a corrupted dbt_packages/<pkg>/dbt_project.yml; expect to see the real "Failed to read package" message, not "Could not parse"

Links

  • Issue: #932
  • Catalog entry: 03-issues-and-fixes.md Issue #14
  • Discovered while investigating the auto-deps clobber on a sister repo (altimate-dbt-integration#120); the masked error happened to be a corrupted dbt_packages/ fixture

Summary by cubic

Bubble real dbt errors across execDbtShow, execDbtCompile, and execDbtCompileInline, while redacting inline SQL and separating parser failures from dbt execution issues. Fixes misleading “Could not parse” cases and makes failures actionable. Addresses #932; closes #943, #944, #945.

  • Bug Fixes
    • run() now attaches stdout/stderr on execFile rejection; shared helpers extractDbtError/fallbackExitMessage/bubbleDbtError prefer last structured JSON error > primary stderr > plain stderr > a redacted exit-status, strip ANSI, and avoid doubling dbt category prefixes.
    • execDbtShow: distinguish failure modes; if JSON-mode crashes, bubble the real dbt error; if JSON-mode succeeds but is unparseable and plain-text then fails, throw a parser error. Skip Tier 1/2 when the primary run failed, never feed crashed stdout into heuristics; still try Tier 3 and return the table if it succeeds.
    • execDbtCompile and execDbtCompileInline: adopt the same error bubbling and SQL redaction (covers exit codes and signal/timeouts); also bubble errors from manifest/plain fallbacks.
    • Guard data shape: treat truthy non-array data.preview as empty rows to avoid malformed results.
    • Tests expanded for JSON/stderr preference, parser-vs-dbt attribution, ANSI strip, SQL redaction, Buffer I/O, Tier 3 recovery, and non-array preview; 41 passing tests.

Summary by CodeRabbit

  • Bug Fixes

    • Improved error handling for dbt show to surface actual dbt failure details, preferring structured JSON error events when available and avoiding generic “Could not parse …” messages on real crashes.
    • Made dbt compile and dbt compile --inline report consistent, safer failure messages (ANSI cleaned and inline SQL not leaked).

  • Tests

    • Expanded coverage for dbt show/compile failure modes, including fallback behavior when process output is missing, correct error-event selection, retry recovery, and message sanitization.