Pin harness instructions, skills and tool schemas in one context manifest

Merged
VA
3d ago
+5662-225

Problem

Instructions, selected skills and model-visible tool schemas have separate identities, while GraphCoder owns a mutable discovery/reload copy. Resumed compositions need one pinned selection. Sampling also cloned the entire registry and materialized schemas before dropping unauthorized tools.

Design

ContextManifest records ordered AGENTS.md/CLAUDE.md references, the pinned skill catalog and invocations, and exact tool definitions/revisions. Publish it in the caller volume through ContentPublisher; authenticate and decode its FileRef through ContentResidencyVerifier with bounded lexical JSON admission. Validate file/render/message limits, every instruction and skill path-byte bound, duplicate selections, and tool schemas/revisions before selected skill body reads. Bind selected skills once and cache the context stage; context.manifest metadata and the immutable contract retain the exact manifest identity.

Registry snapshots share one Arc-backed map set and Arc payloads. Ordinary executor cloning is O(1), including hidden revisions. Mutations copy metadata while sharing schemas. Request assembly and request identity select authorized names before copying definitions; admitted names are counted first, with both traversals charged, then one funded vector reservation serves the entire selection; tool-free sampling retains the pinned catalog without materializing any schema.

Deletions

Delete GraphCoder's /context FILE, WorkflowContext, mutable RwLock and duplicate discovery/reload/attribute assembly. Its config contains an optional manifest FileRef; resume reuses the harness pipeline. Replace wrapper assertions with in-process, filesystem-backed harness scenarios. Delete arbitrary executor registry replacement and repoint MCP sampling to grant attenuation. Delete registration's full name scan; the existing complete logical-name index answers membership.

Decisions

  • Started from main at 1fa9fff61bb6; merge main at 63e67312ff15 signed, preserving #572 installation, #594 models, and #601 allocation-free hiding. No pushed rebase.
  • Volume manifest is authoritative. Shared registry maps cache native executor bindings; they add no store, protocol, journal, relay, polling, or synchronization algorithm.
  • Instruction array order is broad-to-specific precedence; selected skill order is invocation order. Tool definitions must exactly match retained executors. Caller grants still attenuate exposure; manifests grant nothing.
  • Pinning changes visibility atomically while preserving every logical-name index entry as selected or hidden. New revisions of a manifest-hidden name stay hidden. Admitted pinned revisions remain callable by exact identity. Hiding an unshared registry keeps #601's zero-allocation lookup.
  • Every skill catalog path, selected or lazy, obeys path_bytes before any selected body read. Exact-bound paths are valid.
  • Native explicit ownership copies remain funded by the original caller ledger, including Arc backing/header bytes. Shared native registration rejects implicit unfunded copying; the existing funded copy path remains available.
  • Sampling removes fresh tool:call grants, retains all other grants and the pinned catalog, and emits no tool definitions. The allocation property compares complete real callbacks with 10 and 10,000 logical tools, two retained revisions each and half hidden. Its deliberate negative control restores request assembly that deep-copies definitions before discarding tools; the same measurement loop and independence detector must reject it. A separate shrinking funded-I/O property compares nonempty admitted selections of 1–128 tools with an explicit single-reservation reference, including hidden, absent and unauthorized names.
  • No new locks. Shrinking properties cover selection, authority, immutable sharing and the complete name index after all mutations, including manifest pinning. Parser coverage uses actual production decoding, ASan fuzzing and checked-in seeds.
  • LocalHarness remains the sole composition root; default wiring follows #495.

Tests run

Signed head: 271f79c596e65722e1f63e175d8b0e01ccab9009. This correction: +114/-32, net +82. Full PR: +5662/-225, net +5437; 4081 added lines are the standalone fuzz lockfile (net excluding it: +1356). GraphCoder wrapper shrinks by 72 lines and its qualification script by 34.

All Linux builds and execution via Q:/lanes/bs.ps1, warm target and incremental defaults. One native feature closure: filesystem-local,native-execution,terminal,mcp-http,test-support (includes hooks, mcp-stdio and defaults).

  • Final touched-feature compile: cargo test --no-run --lib --test context-discovery --test mcp-durable passed, 206.19 s build. Built test binaries run directly afterward.
  • Sampling allocation regression and its deliberately broken variant: 1/1, 2.09 s wall. Correct variant: both catalog sizes exactly 581 allocations / 101,215 bytes. Negative control: 2,066 / 272,879 versus 1,485,581 / 179,933,039; the same detector rejects it. This expanded harness measures a per-call ToolContext clone too (one allocation / six bytes); the production callback remains unchanged.
  • Registry shrinking properties and allocation/funding checks: 7/7, 1.66 s. Includes 1–128 admitted tools matching the single-reservation reference in exact allocation count and bytes, complete name-index invariants through manifest pinning and other mutations, shared schema snapshots, native funding and allocation-free hiding.
  • Sampling attenuation: 1/1, 0.09 s; manifest properties/negative cases: 4/4, 0.08 s; tool/revision scenarios: 6/6, 0.01 s.
  • MCP transport/install/schema/cancellation scenarios and properties: 57/57, 0.42 s.
  • Filesystem-backed scripted discovery/manifest/restart scenarios: 13/13, 0.12 s. Covers source replacement, pinned generations, precedence, lazy/selected skills, one body read, cached context, exact revisions, replay, and bounds. An oversized skill path at a short manifest location rejects with zero skill reads; the exact path limit succeeds.
  • Durable MCP callbacks/reopen: 8/8, 0.67 s, including sampling lost-ack/reopen, cancellation, media and form/endpoint controls.
  • Parser unchanged by the registry corrections: previously updated ASan context_manifest target passed five seeds plus 512 runs in 0.64 s. Actual ContextManifest::decode/encode, shrinking canonical round-trip and path-byte properties; fuzz varies path_bytes across 1/16/64/4096. Rust 1.98.1, RUSTC_BOOTSTRAP=1 only for sanitizer instrumentation. Newer nightly fails existing filesystem Unicode/lifetime checks; no checks weakened.
  • rustfmt --check and changed-file git diff --check passed. node --check scripts/test-graphcoder.mjs passed at the initial implementation. Full wrapper qualification was not rerun; moved behavior is covered by harness scenarios.

Final gate log: /home/var/log/buildbox/2026-10-16T000254705Z-3622394.log (rc=0). The preceding merged-head gate also passed, including 57 MCP tests: /home/var/log/buildbox/2026-10-15T235018911Z-3518110.log. Prior ASan gate: /home/var/log/buildbox/2026-10-15T230153583Z-3232305.log (rc=0). Each execution suite remains under 10 s; no sleeps or polling added.