feat: pre-execution critic gate for side-effecting tools

Draft
AN
2mo ago
+750-0

What does this PR do?

Adds a flag-gated (ALTIMATE_CRITIC_GATE, default OFF) pre-execution "critic gate". Before a side-effecting tool (bash/write/edit/sql_execute/dbt_run/patch) runs, a pluggable Verifier checks the proposed args; on a hard verdict the call is skipped and the reason is fed back so the model can fix-and-retry, instead of executing a bad action. No behavior change when the flag is unset.

  • tool/critic.ts — pure, testable gate + pluggable Verifier interface.
    • ALLOW_ALL — ungated (opt-out / tests).
    • basicSafetyVerifier — the wired default: a conservative, dependency-free heuristic that blocks catastrophic, unambiguous host-destructive bash (rm -rf / incl. system-path / glob / compound / fully-qualified / ${HOME} variants, fork bombs, mkfs/dd on a raw device, recursive chmod of /). Best-effort safety net, not a security boundary — documented as such; defense-in-depth stays with the OS sandbox, the permission system, and a richer verifier a product may inject.
    • gate() is fail-open on verifier error AND on a verifier timeout, so a broken or hung verifier never breaks/hangs the agent.
  • session/prompt.ts — wired into the native-tool execute wrapper just before item.execute, marker-wrapped, emits tool.execute.after on a block so observability sees it. MCP tools use a separate wrapper and are intentionally not gated (documented on DEFAULT_GATED).

Split out of #857 / PR #858, where the critic was previously an unwired no-op flag. This PR makes the flag actually do something.

Type of change

  • New feature (non-breaking change which adds functionality)

Issue for this PR

Closes #862

How did you verify your code works?

  • 100 tests across three files: unit (critic.test.ts), adversarial bypass probes that document caught-vs-known-limits (critic-adversarial.test.ts), and e2e driving the REAL BashTool through the gate with real filesystem effects (critic-e2e.test.ts) — no mocked tool calls. All green; 448 tool-suite tests pass; tsgo typecheck clean; altimate_change markers in prompt.ts balanced (37/37); default-off path unchanged.
  • Adversarial testing found + fixed real bugs: compound commands where the fatal rm wasn't last (rm -rf / && rm -rf ./safe), and a separator glued to the target (rm -rf /;).
  • Reviewed by a multi-model panel (Gemini 3.1 Pro, GLM-5, MiniMax M2.7) before opening. Their findings were applied: removed false positives (rm -rf *, rm -rf ., and rm buried in an argument like git commit -m "...rm -rf /..."), closed false negatives (glob wipes rm -rf /var/*, /.//.., fully-qualified /bin/rm, ${HOME}, long-form/glob chmod), and added an input length cap + verifier timeout.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have added tests that prove my feature works
  • New and existing unit tests pass locally with my changes

Summary by cubic

Adds a flag-gated pre-execution critic for side-effecting tools that blocks clearly dangerous bash before it runs. Default is off; when enabled, blocked calls return feedback so the model can fix-and-retry.

  • New Features

    • Controlled by ALTIMATE_CRITIC_GATE; pluggable Verifier with ALLOW_ALL and default basicSafetyVerifier.
    • Blocks catastrophic bash (e.g., rm -rf / variants, fork bombs, mkfs/dd to devices, recursive chmod/chown of /); fail-open on verifier error or 5s timeout.
    • Wired into session/prompt.ts just before execution; on block emits tool.execute.after and skips execution. MCP tools are not gated.
    • Gated tools: bash, write, edit, sql_execute, apply_patch. Tests: 104 (unit, adversarial, e2e with real BashTool).

  • Bug Fixes

    • Corrected gated tool id to apply_patch; removed non-existent dbt_run.
    • Closed bypass: inline env assignments (FOO=1 rm -rf /, IFS=x ... rm) now resolve command position correctly.
    • Fixed timer leak by clearing the verifier timeout after resolution; tightened types to Record<string, unknown> in Verifier.verify and gate, and updated the prompt wrapper cast (no behavior change).

Summary by CodeRabbit

  • New Features
    • Optional pre-execution safety gate for side-effecting tools (e.g., bash) that can block dangerous commands, return informative feedback when blocked, and otherwise allow normal execution.
  • Tests
    • Added comprehensive unit, adversarial, and end-to-end tests covering detection heuristics, gating behavior, timeouts, robustness, and fail-open guarantees.