gather: deterministic context-pack tool (aft_gather)

What

One new tool — aft_gather — assembles a bounded "context pack" (ranked, deduped, budgeted verbatim code evidence) in a single call. It replaces the multi-turn search → outline → zoom → callgraph read chain an agent otherwise runs to build context around a question or a symbol.

Two modes (mutually exclusive):

• question: "how does X work?" — seeds from handle_semantic_search (same pipeline as aft_search, all lanes/fallbacks)
• symbol + filePath — seeds from the callgraph (impact depth-1 callers + call_tree depth-1 callees)

Seeds expand one hop through the callgraph, dedupe by canonicalized (file, symbol) with seeds winning, and render via render_symbol_within_budget until a hard line budget (default 400, cap 800) is spent. Everything past the cut appears as one-line stubs under ## Beyond budget (zoom to expand) — nothing is silently dropped.

Why

Agents burn serial turns assembling context: search, then outline the hits, then zoom the symbols, then chase callers. Each turn round-trips through the model. A pack returns evidence (verbatim bodies with file:line headers), not conclusions — the agent reasons over it directly, ready to attach to a subagent dispatch.

Measured on a real config repo (3 questions, one call each vs. the manual chain):

• "how does the lane-verdict cache flow end-to-end" → complete 4-file chain (cache shapes, verdict logic, write/read, plugin deny hook) in one pack, used=283/400. Manual baseline: 5-6 tool calls.
• "how does score-tap decide when to write an executor row" → the full decision chain (event filter → dedup → context sources → verdict parse → DB insert) in one pack at budget=200.
• An unrehearsed cross-file question → correct core symbols plus their docstrings, first try.

Independently reproduced on the aft codebase itself: question: "how does bash output compression dispatch pick a compressor" → seeds=15, used=226/400, one pack assembling the full dispatch chain (compress → gate → compress_with_registry_exit_code 20-compressor array → Compressor trait → install path → subc mirror) that otherwise takes a 4–5-call search→zoom chain.

Honest degradation

The pack never lies about its own quality:

• While the semantic index is building, long NL queries degrade to lexical-only file-level hits. The pack renders them as visible file:line (no containing symbol) stubs and flags the header with degraded=semantic-index-building (partial results — retry when index ready) — detected via the response's semantic_status field, cleared as soon as one real seed resolves. No blocking or retry inside the tool.
• Grep-fallback hits ({file, line_text, line}, no symbol name) resolve to their containing symbol by line containment — definitions, call sites, and comment hits all upgrade to the enclosing symbol. Hits with no containing symbol stay visible as stubs.
• Unresolved external/stdlib callees collapse to one summary line ((N unresolved external calls omitted)) instead of drowning the stub list; unresolved seeds and callers are never suppressed.

Implementation

• crates/aft/src/commands/gather.rs — Rust-side composition: calls handle_semantic_search / impact_result / call_tree_result / render_symbol_within_budget directly (shared &AppContext, no bridge round-trips, no parallel reimplementation of search).
• Wiring: main.rs dispatch arm, subc_translate.rs mapping, TS factory packages/opencode-plugin/src/tools/gather.ts + registration (same tier as aft_callgraph — depends on the callgraph store).
• No new dependencies, no config surface beyond the tool args, no LLM calls, no caching.
• Follows the tri-state honest-reporting convention (protocol.rs Response doc-comment): success:false+code for un-performable calls (e.g. invalid_request on a bad mode combo), success:true with a visible degraded/stub pack for partial results — never a bare empty success.

Tests

22 unit tests in gather.rs, including red-checked regressions (each confirmed to fail against pre-fix code): mid-codepoint truncation panic, duplicate-symbol line-anchored resolution, abs/rel path dedupe, containing-symbol resolution via a real TreeSitterProvider, callee-only stub suppression driven through the production build_pack path, and degradation-flag presence/absence/mixed cases.

Limitations (deliberate scope)

• 1-hop expansion only — multi-hop was deliberately excluded: the budget math and ranking get harder and the packs get noisier.
• Callgraph-dependent by design: neighbor expansion quality follows the callgraph store's freshness, same as aft_callgraph.
• Budget counted in lines, not tokens — matches aft's existing budget idiom across zoom/outline.

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

Summary by cubic

Adds a deterministic context-pack builder: gather in Rust and aft_gather_context in @opencode. It replaces the multi-step search→outline→zoom→callgraph chain with one call that returns ranked, deduped, verbatim code within a fixed line budget.

• Two exclusive modes: question (semantic seeds) or symbol + path (impact callers + call-tree callees); 1-hop expansion; dedupe by (file, symbol) with seeds winning.
• Hard line budget (default 400, max 800) with per-symbol balancing; overflow becomes visible stubs under “Beyond budget”; exact used lines computed post-render.
• Honest degradation: grep-fallback hits resolve to containing symbols; no-symbol hits render as stubs; header flags degraded=semantic-index-building and neighbors=skipped(callgraph-unavailable).
• Optional includeTests to include test-file neighbors (default off).
• Transport and surface: wired into subc manifest/schemas/translate; exposed only on the “all” surface; public path maps to internal filePath (translator accepts both). Schema inventory updated (22 bare tools).
• Path normalization/dedupe: absolute vs. relative paths dedupe on Windows; separators normalize before compare; Unix backslash in filenames normalizes to / and renders a clear not-found stub.
• Tests: 22 unit tests plus transport-invariance and an integration test covering includeTests. No new dependencies.

Confidence Score: 4/5

• Safe to merge with the callgraph path inconsistency investigated or accepted — the failure mode is silent degradation (fewer neighbors), not corruption or incorrect output.
• The core budget accounting, dedup, degradation flagging, and stub rendering are well-covered by 22 unit tests and the transport-invariance test. The schema/path dual-spelling fix (gather_file_path) and the include_tests thread-through are correct. The one open question is whether collect_callgraph_neighbors silently produces empty neighbor sets in question mode: it calls impact_result and call_tree_result with normalized-relative seed.file paths while every other call site in the codebase uses absolute validated paths, and errors are swallowed with if let Ok. If the callgraph store is keyed on absolute paths, question-mode neighbor expansion returns nothing with no visible indicator. The integration test covers only symbol mode, so this path is untested end-to-end.
• crates/aft/src/commands/gather.rs — specifically collect_callgraph_neighbors lines 421–447 and the absence of a question-mode callgraph expansion integration test.

sequenceDiagram
    participant Agent
    participant gather.ts
    participant translate_gather
    participant handle_gather
    participant handle_semantic_search
    participant collect_callgraph_neighbors
    participant build_pack

    Agent->>gather.ts: "aft_gather_context({question|symbol+path})"
    gather.ts->>gather.ts: rename path→filePath, validate mode
    gather.ts->>translate_gather: gather_file_path() accepts "path" or "filePath"
    translate_gather->>handle_gather: "{command:"gather", filePath:...}"

    alt question mode
        handle_gather->>handle_semantic_search: "top_k=15 seeds"
        handle_semantic_search-->>handle_gather: results + semantic_status
        handle_gather->>collect_callgraph_neighbors: seeds (normalized-relative file paths)
        Note over collect_callgraph_neighbors: impact_result(seed_path) where seed_path is relative<br/>symbol mode uses ctx.validate_path (absolute) — inconsistency
        collect_callgraph_neighbors-->>handle_gather: neighbors (or silently empty)
    else symbol mode
        handle_gather->>handle_gather: ctx.validate_path → absolute file_path
        handle_gather->>handle_gather: "impact_result(&file_path) — absolute ✓"
    end

    handle_gather->>build_pack: seeds + neighbors + budget
    build_pack->>build_pack: dedup, render within budget, stub overflow
    build_pack-->>Agent: gather pack text with file:line headers

_{Reviews (29): Last reviewed commit: "gather: deterministic context-pack tool ..." | Re-trigger Greptile}