#849 feat(validators): dbt-tests-pass validator + dbt-schema-verify hardening

What does this PR do?

Completes the harness-side completion-gate validator framework. The registry and dbt-schema-verify were shipped in earlier PRs; this adds dbt-tests-pass and hardens the existing validator.

New: dbt-tests-pass validator

Fires after the agent declares done (finishReason === "stop"). Finds every dbt model .sql file the session modified (by mtime since sessionStartMs), runs altimate-dbt test --model <name> against each, and injects a synthetic user message if any tests fail — forcing one more agent turn to fix the SQL.

What this catches that schema-verify doesn't: row-data correctness errors. A model whose column names all match schema.yml can still fail relationships, unique, not_null, accepted_values, or AUTO_*_equality tests because the SELECT logic produces wrong values or wrong row counts.

Hardened: dbt-schema-verify

• parseSchemaVerifyOutput() — scans the stdout blob for the last balanced {...} block. altimate-dbt emits dbt log noise (ANSI codes, parser warnings) before the JSON verdict; JSON.parse(stdout) was failing on it. This fix was validated on airbnb006 where the validator correctly triggered a schema-mismatch retry.
• Debug logging (spawn errors, close events, stdout/stderr heads) gated behind ALTIMATE_VALIDATORS_DEBUG=1.

prompt.ts: diagnostic cleanup

• All console.error calls gated behind ALTIMATE_VALIDATORS_DEBUG=1 — previously they were always on, producing noise in normal sessions.
• Added missing validatorsEnabled && guard on dispatch condition.
• Added dispatch_enter, dispatch_result, dispatch_error debug events.

build-local-tarball.sh: altimate-dbt on PATH

Added "altimate-dbt": "./dbt-tools/bin/altimate-dbt" to bin entries in the benchmark npm package. Validators spawn altimate-dbt and it wasn't in PATH inside the Docker containers used by ade-bench.

system.ts: Marker Guard fix

Moved // altimate_change end from inside the skills() function body (after the unreachable return) to outside it. The function's closing } was appearing as unguarded new code in the diff, tripping Marker Guard CI.

Type of change

• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)

Issue for this PR

Closes #848

How did you verify your code works?

End-to-end on DGX with the ade-bench Docker harness:

• dbt-schema-verify: airbnb006 — agent triggered a schema-mismatch retry, fixed 5 column mismatches, all tests passed
• dbt-tests-pass: confirmed validator detects test failures and injects the correct fix hint
• altimate-dbt test on PATH: which altimate-dbt → /usr/bin/altimate-dbt inside container
• TypeScript: bun run typecheck passes
• Marker Guard: bun run script/upstream/analyze.ts --markers --base main --strict passes

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I ran bun run script/upstream/analyze.ts --markers --base main --strict and it passes

🤖 Generated with Claude Code

---

Summary by cubic

Adds a harness-side dbt-tests-pass validator and hardens dbt-schema-verify, with an opt-in completion gate that blocks session finish when dbt tests fail or schema diffs are found. Validators run only when enabled; ADE-Bench setup turns them on by default per trial (opt-out available), with docs and telemetry updated.

• New Features

  • dbt-tests-pass: after finish, runs altimate-dbt test --model <name> on modified models; injects a user message on failures to force another turn; names failing models.
  • Validator framework: explicit registration, retries via ALTIMATE_VALIDATORS_MAX_RETRIES, timeouts and concurrency controls; enriched details (dbt_root, session_id, elapsed_ms, concurrency_limit).
  • altimate-dbt schema-verify: CLI and auto-run in build responses; validator blocks completion on mismatch and returns structured diffs.
  • ADE-Bench: enable validators by default via ALTIMATE_VALIDATORS_ENABLED=1 (opt-out ALTIMATE_VALIDATORS_BENCH_DISABLE=1); docs and telemetry updated.

• Bug Fixes

  • Performance/behavior: dispatch gated by ALTIMATE_VALIDATORS_ENABLED=1 or ALTIMATE_VALIDATORS_SHADOW=1; no scans/spawns by default; telemetry only in these modes.
  • Robustness: last-wins JSON parsing; optional WARN/SKIP/NO-OP; ANSI stripping; bounded failing-test capture; symlink/case-insensitive model discovery; Windows paths; strict JSON-envelope guard; spawn-failure accounting; fail-closed on schema-verify errors; XML-escape skill names; mark altimate-backend provider IDs with markers in packages/opencode/src/provider/transform.ts to prevent upstream overwrite.
  • Docs: clarified validator telemetry (removed “session rollup”; marked checked/concurrency_limit as optional), noted both opt-in flags and zero-overhead default in dbt-tools docs, and corrected skill auto-load placement (prepended before <available_skills>); minor README code-fence fix.

Summary by CodeRabbit

• New Features

  • CLI schema-verify and auto-run after successful builds; validators that enforce schema and dbt unit-test checks in sessions; auto-loading skills via applyPaths/alwaysApply; new Altimate Code ADE‑bench agent and local tarball tooling; ADE‑bench agent log parsing and benchmarking support.

• Documentation

  • Expanded dbt development guidance, schema-verify/unit-test workflows, skills configuration, ADE‑bench README and experiment reports.

• Tests

  • Broadened test coverage for schema-verify, build auto-verification, validators, validator utilities, and dbt test parsing.