Add harness model registry and four streaming adapters

Merged
VA
3d ago
+17169-857

Problem

GraphCoder needs shared streaming providers that execute through the stock native harness. Tool and reasoning continuity must survive persisted history, replay and provider changes, and context admission must use verified token counts.

Design

Resolve immutable provider/model selections and options from ProviderSpec catalog data: endpoints, authentication policy, exact capacities and token-counter routes. Anthropic Messages uses API keys; OpenAI Responses, compatible Chat Completions and Gemini stream text, reasoning, tool calls and usage. Native callbacks retain the issued allocation custody and cancellation. Credentials resolve keychain → environment → config; Linux uses Secret Service.

Verified admission counts are separate from usage-calibrated compaction weights. Unknown capacities/counters require provider or config data. The openai/gpt-6-luna fixture configures them explicitly and exercises API-key reasoning/tool follow-ups on ordinary and native paths.

Canonical ordered reasoning lives in existing task-volume transcript artifacts. Native signed payloads borrow their original RawValue bytes; foreign readable reasoning becomes labelled context, while opaque foreign continuity is omitted. Harness-minted call IDs encode operation UUID, step and call ordinal. Exact provider IDs remain metadata bindings; a request-local bijection maps calls and results for each target, including reused provider IDs across alloy turns.

Gemini preserves signed visible parts at their original boundaries, including empty signed parts. Their canonical continuity is marked suppressed-display and owns no second readable text copy. Unsigned fragments coalesce between signature boundaries; call-free unsigned completions use the ordinary answer. Projection consumes duplicate display text across chunks and tool steps while emitting the exact native parts. Foreign targets receive the ordinary answer once. Chat text parts concatenate without invented newlines.

Compaction derives dependency groups directly from canonical reasoning, calls and results. Planning never cuts through a group; mandatory selection and spare-slot refill retain or drop every member together. Nonempty native continuity is kept without provider-specific parsing, including Gemini thought signatures, Anthropic signed thinking and Responses encrypted reasoning. Unrelated text remains independently selectable.

Complete RecordedInvocation envelopes are admitted against both file and history-render bounds before tools execute. Invocation metadata travels with its indexed call. Retained stream state has aggregate byte/item limits and incremental accounting; HTTP and SSE errors retain sanitized retry classes.

Deletions

Against merged main 913b0a082: +17,169/-857, net +16,312 across 180 files; the standalone fuzz lock accounts for 5,085 added lines. Latest compaction fix: +436/-102, net +334 across 6 files. The prior Gemini signed-answer fix remains unchanged.

Remove estimate-based admission, provider-name auth/capacity matches, first-call reasoning attachment, bare invocation artifacts, repeated metadata searches, per-frame retained-state recounts, duplicate signed-answer projection and artificial Chat text separators. harness-codex is deleted on main. The next provider follow-up is #508: registry WASM exports, TS caller repointing, openai.ts/openai.test.ts deletion and browser qualification. Existing Codex-login work remains separately reviewed in #498.

Tests run

At signed head 93f7af0b5fc07233152daa47f8cde8cdd8743801, on the #686-merged provider head:

  • 97 provider tests passed in 0.59 s, including the unchanged stock native workflow for all four adapters. No filtered or ignored tests.
  • 40 focused context/executor tests passed: context 27 (0.28 s), executor compaction 13 (0.01 s). The 64-case shrinking property generates mixed-origin native reasoning, packed/separate messages, interleaved calls, reversed parallel results, arbitrary cuts and tight budgets. It checks planning, direct application, refill and byte-exact continuity. A call/result-only negative control detects missing reasoning. Item limits 0..512 cover successful and rejected selection with zero retained temporary custody.
  • Reused unchanged-code evidence from c152ea0c9: 54 focused harness/history tests passed: JSON codecs 18 (0.59 s), projection 20 (0.11 s), metadata indexing 3 (0.01 s), admission 4 (<0.01 s), model identities/codecs 8 (0.22 s), saved-history replay 1 (0.76 s). These reuse built test binaries.
  • Shrinking properties cover signed/unsigned Unicode fragments, independent wire and answer chunk boundaries, empty signed parts, tool steps, all four target projections, byte-exact native continuity, retained-balance cleanup and cross-turn call-ID collisions. Checked-in SSE and shrunk split-answer fixtures exercise the same paths. Existing negative controls remain enabled.
  • Reused unchanged-parser evidence from c152ea0c9: pinned nightly-2026-07-01 GNU/ASan provider_stream fuzz: 256 runs, under 1 s. Its oracle checks exact native part bytes plus one copy of each visible byte.
  • Final Windows lb.ps1 cargo check -p acyclic-harness --lib --tests --features filesystem,test-support passed in 53.81 s (690 s queued). The previous providers/filesystem Windows checks remain unchanged-code evidence.

Stock gate, original 65,536-item allowance:

Adapteritems_examinedRemaining
Chat65,305231
Anthropic64,4841,052
Responses64,5171,019
Gemini64,3091,227

Latest Linux log: /home/var/log/buildbox/2026-10-16T111809980Z-3656582.log. Provider fuzz and non-compaction history evidence is reused from /home/var/log/buildbox/2026-10-16T104834190Z-3593142.log. Prior Windows logs: Q:/lanes/gc-providers-496-landed-decoder-windows.log and Q:/lanes/gc-providers-496-gemini-final-windows.log; the final harness check completed through lb.ps1 in the lane terminal. All model execution uses scripted fixtures; the coordinator owns the live API-key acceptance run.

Decisions

Merged origin/main through 913b0a082 (#686), including its complete nested-array decoder fix. The unchanged stock gate now passes; its original quotas and assertions remain intact.

Catalogs are config-volume JSON; reasoning and provider bindings use existing transcript files. Gemini's native visible continuity uses the existing opaque/signature IR and suppressed-display outcome, with no new transcript store or protocol. Unsigned parts may coalesce; signed parts retain their exact boundaries and bytes (GEM-THK-01/04). The model's final answer is the visible-content fold across the tool loop, so request projection consumes matching native display coverage across those steps and clears it at a new user turn.

Canonical call identity uses an injective operation/step/ordinal encoding. Retried dispatches mint the same ID; distinct dispatches or ordinals mint different IDs. The ID/binding graph is admitted before allocation. Supported history uses RecordedInvocation with mandatory metadata; former bare artifacts require migration rather than a compatibility path.

Compaction uses the canonical IR continuity field as its dependency contract instead of interpreting provider JSON. Groups are computed per projection from transcript data; no new durable store, event or global state is added. Required members stay byte-exact. A group that cannot fit spare slots is dropped whole; unrelated text may use the remaining budget. The former repeated edge-closure scan is removed in favor of flattened components and two linear selection passes.