PR #1215 - AltimateAI/altimate-code

Issue for this PR

Type of change

Bug fix
New feature
Refactor / code improvement
Documentation

What does this PR do?

Reworked onto #1217 (2026-09-02). #1217 merged to main and deleted the monolithic builder.txt this PR originally edited, splitting it into builder/core.txt + builder/packs/*.txt fragments assembled by altimate/prompts/profiles.ts. The Pre-Execution Protocol is now the sql-guard pack, and it ships statically inside the default byte-pinned PROMPT_BUILDER. This PR was rebased onto that architecture and its mechanism re-expressed as conditional pack exclusion instead of conditional text injection — the intent, the classifier, and the safety property (unknown keeps the protocol) are unchanged; only the "how" moved. See "The scoping mechanism" and "How did you verify" below for the reworked details, and the description of the original evidence/ablation immediately below is unchanged.

## Pre-Execution Protocol sat statically in packages/opencode/src/altimate/prompts/builder.txt (now builder/packs/sql-guard.txt), making sql_analyze + altimate_core_validate mandatory before every sql_execute. builder is a PRIMARY agent, so that one section governed every builder surface at once — dbt authoring, interactive chat, and headless question-answering runs.

The evidence. An internal pre-registered paired prompt ablation, 540 trials on a public data-question benchmark, one binary across both arms, every trial's resolved system prompt verified by sha256 against a pre-registered artifact:

	macro Pass@1	micro Pass@1
control	0.6667	0.6778
treatment (protocol removed, among other changes)	0.6807	0.7148
delta	+0.0140	+0.0370

query-blocked sign-flip permutation, 20,000 resamples: p = 0.7358
cluster-bootstrap 95% CI: [-0.0400, +0.0674]
per-query direction: treatment better 16, control better 11, tied 27 (exact sign test p = 0.4421)

The honest claim is "no score effect, materially cheaper." The +0.0140 is not distinguishable from zero and is not presented here as an improvement — the interval comfortably contains zero in both directions.

What did move:

	control	treatment	delta
wall clock per trial (restricted mean)	440.9s	319.4s	-121.5s (-27.6%)
model generations per trial	11.9	8.6	-27.7%
generation seconds per trial	187.6	127.2	-32.2%
`altimate_core_validate` calls (arm total)	1,476	0	-1,476
`sql_analyze` calls (arm total)	1,329	0	-1,329
`sql_execute` calls (arm total)	1,716	2,554	+838 (+49%)
trials hitting the 900s timeout	27	16	-11

The 2,805 ritual tool calls going to zero — not reduced, zero — is the one number directly attributable to this text. The ritual is prompt-ordered: remove the order and it stops completely. The freed budget went into the benchmark's actual work (sql_execute +49%).

Why this scopes rather than deletes, both taken from the experiment's own caveats:

The latency win is not attributable to this section alone. That treatment arm bundled five coupled changes across two arms; the experiment declined to attribute and no factorial was run.
The measurement covers data questions only. dbt authoring and interactive chat are unmeasured builder surfaces where a pre-execution discipline may genuinely earn its place.

The scoping mechanism (reworked onto #1217's pack architecture)

agent.ts registers builder with prompt: PromptProfiles.PROMPT_BUILDER — a plain string, assembled ONCE at module load from core.txt + all packs including sql-guard, and byte-pinned by test/altimate/prompt-profiles.test.ts. That default registration is never touched by this PR — no change to agent.ts, profiles.ts's existing exports, or the pin.

The mechanism this PR adds: profiles.ts gets one additive export, PROMPT_BUILDER_SCOPED — the same profile, same fragment order, with only the sql-guard pack filtered out. session/pre-execution.ts's gate (SessionPreExecution.scopedBuilderPrompt, formerly preExecutionInstruction) now returns that override string, or undefined to mean "use the agent's default prompt, unchanged." session/prompt.ts — at the point in the per-step loop where a fully-resolved Agent.Info is already in scope, right before the model call — clones it with .prompt swapped only when the gate returns an override, and passes that clone into processor.process (which is what actually reaches LLM.stream). The registered agent object is never mutated.

This follows the precedent already in the tree in spirit — SessionTermination.completionInstruction also scopes a run-mode-only instruction to builder — but the action is now exclude a pack from an already-assembled prompt rather than inject additive text into the turn's system array, because #1217 baked the protocol into the static default and there is no longer a builder.txt to edit or an absence to inject text into.

Design fork considered and rejected: making Agent.Info.prompt a function (computed per-session) instead of a plain string, or excluding the pack at agent.ts registration time. Rejected because agent.prompt is read as a plain string directly downstream in llm.ts, llm/request.ts, and compaction.ts, and agent.ts registration happens once at config load with no access to a session's cwd/worktree — either option would touch the agent registration shape and multiple downstream consumers for no behavioral gain over cloning the already-resolved agent at the one call site that needs the override.

How the condition is decided. The protocol is dropped only when all three hold:

run mode (Flag.ALTIMATE_RUN_MODE — the run CLI, CI, headless), and
the agent is builder (the only profile that ever carried the sql-guard pack — analyst.txt and reviewer.txt never did), and
the workspace is confidently classified as having no dbt project.

That is exactly the cell the ablation measured — unchanged from before the rework. Every other case returns undefined (default prompt, protocol included), so a kept case resolves to the exact same prompt as the default builder profile.

Where it looks. A dbt_project.yml (or .yaml) file at the candidate directory (realpathed first, so a symlinked cwd is followed), at any ancestor up to the filesystem root, or one level below — the last matching findDbtProjectRoot's existing rule and skip list, which is how benchmark and monorepo layouts nest a project. The ancestor walk matters because a session is routinely started inside models/, and on a non-git project the worktree candidate is the same directory, so nothing else would find the project. The walk is unbounded on purpose: a depth limit would have to report "I stopped early" as unknown to stay honest, which on any deep tree turns the gate off entirely, and two stat calls per level in run mode is not worth that.

The ambiguous case keeps the protocol. Classification returns a tri-state — dbt / non-dbt / unknown — and only non-dbt drops. non-dbt requires at least one candidate the scan examined completely: symlinks resolved, every ancestor probe answered up to the root, and its own children enumerated and probed. ENOENT/ENOTDIR are real answers ("nothing there"); every other failure — EACCES, EIO, a flaky mount — is unknown. The filesystem root never qualifies on its own, because its children are deliberately not scanned. An unrelated ancestor project is a false positive that keeps the protocol, which is the safe direction. The asymmetry throughout is deliberate: the cost of wrongly keeping it is 27% latency on one workload, the cost of wrongly dropping it is unmeasured.

One complete answer is enough — an incomplete partner candidate does not veto it. That matters: worktree is the filesystem root on a non-git project, so a veto rule returned unknown for every headless run in exactly the configuration the ablation measured, and the gate would have shipped as a no-op.

Interactive chat is deliberately left alone. Run mode is not itself a task-shape signal — it is the surface the evidence covers, and widening the gate to interactive sessions needs its own measurement.

Not touched

main also carries ## Finish Protocol (shipped in #1171), a second mandatory ritual in the same family that was added after the binary the ablation measured was built. The shipping prompt is therefore heavier than what was measured, and the -27.6% understates the current cost. No measurement covers that section, so this PR leaves it alone; a test asserts it survives.

How did you verify your code works?

Carried over unchanged by the rework (the classifier itself was not touched):

Unit suite packages/opencode/test/session/pre-execution.test.ts's workspace classification describe (14 tests): each classification outcome against real temp directories (project at the candidate, one level down, above it, .yaml as well as .yml, no project, a missing directory, a directory that stats but cannot be enumerated, no candidates, the filesystem root, dbt_project.yml as a directory rather than a file, the unbounded ancestor walk, a symlinked candidate, a symlinked child project, and a complete candidate not vetoed by an unreadable partner or by the filesystem root) — untouched by the rework, still 14/14 green.
Reviewed by a second model across two rounds pre-rework. Round one found three bugs — two silent-drop paths in the classifier and an injection-order issue. Round two found four more, including a veto rule that would have made the gate a no-op on every non-git workspace, a depth limit whose exhaustion was recorded as a complete answer, a lexical (symlink-blind) ancestor walk, and a child scan that silently skipped symlinked directories. All seven fixed, each with a test that survived the rebase.

Rework-specific verification (2026-09-02):

Rewrote the pre-execution protocol gate and prompt text fidelity describes in pre-execution.test.ts (they asserted the old text-injection mechanism, including one test that grepped the now-deleted builder.txt) to assert the new override/exclusion behavior: each gate arm now checks the returned override is === PromptProfiles.PROMPT_BUILDER_SCOPED and !== PROMPT_BUILDER (not just "truthy"/"a string containing X"), and every non-firing arm asserts undefined — a test that fails if the gate silently no-ops. Added PROMPT_BUILDER_SCOPED omits sql-guard and nothing else, verifying the override is BUILDER_PROFILE minus exactly the sql-guard fragment and that neighbouring packs (## dbt Verification Workflow, ## Finish Protocol) survive. 24/24 green.
Fixed the one sql-validation-e2e.test.ts test that called the renamed preExecutionInstruction; whole file green (56/56).
Byte-identity gate (test/altimate/prompt-profiles.test.ts, #1217's guarantee that the default builder profile is unchanged): green — PROMPT_BUILDER still hashes to the pinned sha256 (17663410dd9accc527b4cbd84558fc577ccc36d33d0428c5c5205d5df25400d7, 14,773 bytes). This PR's only edit to profiles.ts is one additive export block; BUILDER_PROFILE, FRAGMENTS, assemble, PROMPT_BUILDER, DATA_QA_PROFILE, and PROMPT_DATA_QA are byte-for-byte untouched.
bun install then bun run typecheck — clean on every touched file.
bun run script/upstream/analyze.ts --markers --base origin/main --strict — ok: 1 upstream-shared file checked (session/prompt.ts), all custom code properly marked.
bun test test/session/pre-execution.test.ts test/altimate/prompt-profiles.test.ts test/altimate/sql-validation-e2e.test.ts test/agent/ — 141 pass, 0 fail.
test/session/prompt.test.ts's loop calls LLM and returns assistant message / loop surfaces content-filter finishes as session errors time out locally — reproduced identically by stashing this rework's changes and re-running on the pre-rework branch tip, so this is pre-existing local flakiness (not something the rework introduced or should be scoped as fixing).

Not verified: this change has not been run end to end against a live warehouse, and no benchmark re-run was performed on this build. The behavioural claim rests on the ablation cited above, which measured a pre-#1171 binary; the gate reproduces that binary's treatment only for the cell it measured. The rework changes HOW the exclusion is expressed, not the classifier or the decision boundary, so no new ablation is claimed or needed for the rework itself.

Screenshots / recordings

Not a UI change.

Checklist

I have tested my changes locally
I have not included unrelated changes in this PR

Note: the auto-generated bot summaries below (Cursor Bugbot, CodeRabbit) describe the pre-rework, text-injection mechanism against the pinned commit noted inline — they predate the 2026-09-02 rework onto #1217's pack architecture described above and have not been regenerated.

[!NOTE]
Medium Risk
Changes when the builder must run sql_analyze/altimate_core_validate before sql_execute in CI/run mode; misclassification could drop guards on dbt work, though unknown and dbt paths intentionally keep the protocol.

Overview
Headless builder runs in workspaces with no dbt project can now skip the mandatory Pre-Execution Protocol (sql-guard pack) without changing the default registered builder prompt.

Adds PROMPT_BUILDER_SCOPED in profiles.ts (same builder fragments minus sql-guard) and a new session/pre-execution.ts module that classifies cwd/worktree as dbt, non-dbt, or unknown via ancestor walks, bounded downward search, and symlink-aware probes. scopedBuilderPrompt returns that scoped prompt only when run mode, registry key builder, prompt is still byte-identical PROMPT_BUILDER, and classification is confidently non-dbt; every other case keeps the stock prompt.

session/prompt.ts memoizes the gate once per loop() invocation and passes a cloned effectiveAgent into processor.process when an override applies. Tests cover classification edge cases, gate arms, cache behavior, and e2e prompt expectations.

^{Reviewed by Cursor Bugbot for commit 8e93080ea5667a57f3b16ab59b3f5924b0ecfd9d. Bugbot is set up for automated code reviews on this repo. Configure here.}

Summary by CodeRabbit

New Features
- Added workspace-aware prompt behavior for builder sessions.
- Run-mode builder sessions in confirmed non-dbt workspaces now receive a streamlined prompt without SQL guard instructions.
- dbt workspaces, interactive sessions, uncertain workspace classifications, and other agents retain their standard prompts.
Tests
- Added coverage for workspace detection across nested, ancestor, YAML, symlinked, missing, and unreadable project layouts.
- Added validation for prompt selection across supported session types and workspace conditions.