Give checkout turns Codex-framed environment facts and discovered skills

Closed
VA
9h ago
+2877-33

Summary

F10: checkout turns now carry Codex-framed environment facts (cwd, shell, OS, date, timezone, git status) and a catalog of discovered skills from .agents/skills/ and .claude/skills/.

Skill catalog, projected per request

  • The manifest carries data, not text. Discovery records each valid skill (name, description, file) in the checkout manifest's skill_catalog. Invalid entries are skipped and the scan keeps going. Discovery pages to exhaustion and refreshes after reopen.
  • Rendered when each request is projected. context/manifest/catalog.rs renders the block for the model selected for the request.
    • The allowance is 5% of usable input: (context_tokens - response_reserve_tokens) / 20.
    • The allowance is bounded in bytes. The implementation assumes a byte-to-token upper bound and avoids count_tokens here. Blocking design gap: the provider proptest uses a byte-counting fixture, so it does not establish that tokenizer assumption for the generic ModelProvider contract. See the tokenizer finding below.
  • Deterministic longest prefix. The block lists the longest prefix of entries that fits, plus an exact count of the entries omitted.
  • The pointer line is always present. Whenever at least one skill exists, the block includes N skills available (list .agents/skills/ and .claude/skills/ to see all), even when no entry fits. The line counts toward the allowance. If even the framed block does not fit, the catalog falls back to the minimal form .agents/skills/ and .claude/skills/ have N skills.
  • What this replaces. It replaces the capture-time byte budget (local/context/budget.rs, deleted). That budget sized the catalog to the manifest render bound and could exceed a small model's capacity.

Tests

  • context::skill_catalog:
    • Property rendered_catalog_fits_any_budget, with budgets in 0..64, 300..700 and 0..8192, so it covers budgets below one entry. Rendered bytes stay within the budget, the output matches the first candidate that fits (full prefix, shorter prefixes, then the minimal form), and the total count is always present.
    • Negative control: a renderer that drops the pointer when no entry fits fails check.
  • context::manifest::catalog: property test that projection never fails and never exceeds the allowance.
  • local::tests::environment::oversized_skill_catalog_fits_the_model_allowance:
    • Uses the shipping config (automatic compaction), 300 skills with 1000-byte descriptions plus one broken skill, and a 200K/16K model. The turn completes.
    • The block fits the allowance, equals render(allowance), lists a strict prefix in order, states the omitted count, and the request needs no compaction.
    • Negative control: the same harness reopened with a manifest whose catalog uses the old byte-only budget. The turn is rejected, no request is dispatched, and the cause is mandatory content leaves no compaction source.
  • graphcoder cli:
    • The existing tests keep the 65536-token fixture, which leaves enough allowance to list the fixture skill, as their exact transcripts expect.
    • New small_models_still_learn_that_skills_exist at 8192 tokens. It asserts that the request carries exactly render((8192-1024)/20), that the pointer is present with no entries listed, and that the request fits.
  • The fuzz target checkout_context and scripts/verify-checkout-context.py cover the new shared sources.

Expected-error change in the negative control (accepted by the coordinator)

The negative control originally expected mandatory content exceeds selected model capacity, which is raised at executor/compaction.rs:65. That check runs only when compaction has an older source to summarize.

  • Each local turn starts its own context, so a turn whose catalog alone exceeds capacity holds only mandatory content: system messages plus the current prompt.
  • Compaction planning therefore rejects it earlier, at context/compaction/selection.rs, with mandatory content leaves no compaction source.
  • That is the real production failure for an oversized catalog. The test asserts that message, along with the absence of any dispatched request.

Merged prerequisite: co-st-outcomes (#1020)

On this branch's current base, the rejected negative-control turn is settled Indeterminate: filesystem/task_runtime/dispatch.rs maps every command Err to an indeterminate outcome even when nothing was dispatched. Consequently LocalHarness::close refuses to close it. The test currently drops the harness after its assertions.

#1020 merged into main during this review (2026-10-18 18:33:45 UTC). GitHub still reports this branch as conflicting with main. The author must move these commits to a rebased branch/PR that includes #1020 and then close the negative-control harness normally. Per lane rules, this review does not merge main or rewrite the published branch. Only the integrated version goes to the merge gate.

Gate

  • bb2: local::tests::environment 3/3 (run 6), graphcoder cli 9/9, context:: suites green. The Windows proof is cargo check --target x86_64-pc-windows-gnu for harness and providers.
  • Known failures that are not from this PR, matching main-gate 129c0bf0f: providers codex_stock_execution_counts_locally_without_a_backend_count_route, and the wasm acyclic_test_support unresolved imports.

🤖 Generated with Claude Code

Review boundary fixes

Catalog data is also accepted through published manifests, independently of checkout discovery. Every field (including the skill name) now uses the shared context-value encoder; a decoded catalog cannot author tags or catalog lines. One checked total is shared by manifest validation and rendering: an unrepresentable skills.len() + unlisted is rejected before projection, and direct rendering remains total. No new public API was added.

The checkout-context verifier now checks nonzero test selection, runs the provider byte-bound property, and includes source mutants for these two defects. Its test binaries and fuzz run each have a ten-second deadline.

Guarantee

  1. Class: K (untrusted metadata framing and integer representation), B (stale checkout facts), H (catalog size exceeding file/request/model bounds), and I (verification disconnected from shipping paths). The shared context-value encoder handles every catalog field and environment value. A single checked catalog total is enforced at both manifest admission and rendering. A durable per-operation manifest pins environment and catalog data; subsequent operations rediscover checkout skills. Catalog publication bounds a counted prefix to the manifest file, and model projection derives a counted prefix from the selected model allowance and the complete request byte bound.
  2. Proof: Shrinking properties cover arbitrary Unicode in every catalog field, arbitrary counts including the full u64 boundary, manifest decode/round-trip admission, longest-prefix rendering and exact JSON sizes, projection fit, frontmatter round trips and ignored-field nesting, Gregorian dates and Git porcelain records. checkout_context compiles the private production frontmatter, catalog and porcelain sources directly; its checked-in corpus includes malformed YAML, aliases, duplicate fields, deep ignored values, truncated Git records, renames, hostile JSON catalog fields and overflowing counts. scripts/verify-checkout-context.py compiles deliberately broken production-source variants alongside the original: unchecked count addition, missing value escaping, Unicode separators, capped discovery, invalid-name acceptance, incomplete/rename-source Git counting, shifted dates and removed parser stack guards. Each is detected by the corresponding invariant. The oversized-catalog test compares shipping automatic compaction with the old byte-only catalog and requires the old variant to fail before model dispatch. The pointer-dropping renderer fails the longest-prefix oracle. The real graphcoder binary uses a current-thread runtime and loopback HTTP provider to verify ordered environment/skill/instruction framing, reading the actual skill file through bash, metadata refresh after reopen, replay without redispatch, and the small-model pointer. The live F10 failure on main is recorded in Q:/lanes/live-pass/20261018T001232Z-074cddc92.md; that shipping-path evidence is reused as the regression-first baseline.
  3. Entry points: Checkout LocalHarness capture enters stage_context_manifest_with on turn admission; retries/resume consume the retained operation manifest. Native filesystem model commands bind via model_context; direct stock executors bind via with_context_manifest. Both use the same StageCatalog projection, including compaction projection, and the shared renderer. Public ContextManifest::encode, decode and observed binding validate the checked count; public SkillCatalog::render also guards it independently. Model-free ContextManifest::bind intentionally leaves catalog projection to the executor, as documented. Headless, JSONL and TUI use these same harness paths.
  4. Gaps: The generic provider byte-to-token upper bound is unproven: ModelProvider does not advertise a non-expanding tokenizer contract, and the provider fixture returns request-body bytes plus 512 instead of running a tokenizer. This blocks merge under the zero-gap contract. Closing the class requires choosing a certified provider-specific bound or provider-counted catalog budgeting; either changes the provider/budgeting design and is outside this review pass. #1020 merged during this review, but its deterministic pre-dispatch settlement fix is not on this conflicting branch. A rebase must incorporate it before the oversized-catalog negative control can close normally. The current test drops its indeterminate harness, as described above. This dependency blocks merge. GitHub also reports conflicts with main; the author must move this work to a rebased branch/PR under the lane rules. The ENV parser/fuzz and shipping-HTTP artifacts requested by the guarantee matrix are present; the matrix should be reconciled with this evidence by the coordinator.

The earlier verification tier lacked the shipping environment/catalog assertions and modeled only checkout-validated names and small omitted counts. This pass extends the shared boundary properties to the decoded-manifest entry point.

Coordinator: add to merge gate

The new frontmatter/catalog/porcelain fuzz target is selected by scripts/qualify-graphcoder-client.sh. Run that checkout-context target and the source-mutant verifier on the build box; no GitHub-hosted Rust job is added. Harness/provider runtime properties are already included in the full crate merge gate.

Review validation at 8835cc3f59f0131adcfbfde57e693c8cdb128b27

  • 55/55 focused tests passed; combined runtime 2.098s. Local context and production-source mutants: 31/31 in 0.239s; decoded count admission: 1/1 in 0.012s; shared value encoder: 2/2 in 0.009s; provider projected-byte fixture (not a tokenizer proof): 1/1 in 0.021s; frontmatter depth/round-trip: 1/1 in 0.045s; catalog properties: 6/6 in 0.146s; request projection: 1/1 in 0.044s; native environment/catalog scenarios: 3/3 in 0.746s; real CLI/HTTP scenarios: 9/9 in 0.836s. Every suite was subject to a ten-second deadline and required a nonzero test count.
  • x86_64-pc-windows-gnu lib/tests compilation for Harness, providers and GraphCoder passed in 172.745s with the same native feature closure.
  • The pinned-nightly checkout_context fuzz target compiled in 1.306s and completed 1,000 corpus/mutation runs in 2.421s.
  • Linux test build: 433.332s. Build-slot reported 759.7s for the complete gate; admission reported 95s queued. Evidence: Q:/lanes/passes/1003/pr-1003-review-gate-2.log.
  • The pure-Python nonzero-selection check accepts a populated test result and rejects a zero-test negative control. Caller scans cover stage_context_manifest_with, ContextManifest::bind_observed and model_context. git diff --check passed.

Fixes are signed and pushed. This PR is ready for re-review of those fixes; merge remains blocked by the required rebase and the provider-tokenizer design gap. No approval or merge was issued by this pass.

Blocking tokenizer-contract finding

harness-providers/src/counting/admission_tests.rs:63 calls the common test transport; harness-providers/src/tests.rs:254 responds with request.body.len() + 512. Consequently the passing property tests adapter projection/JSON escaping, not actual tokenizer behavior. The generic ModelProvider trait (harness/src/model/provider.rs:134) exposes verified token counts but no UTF-8-byte upper-bound capability, while harness/src/context/skill_catalog.rs:105 uses the assumed bound for any provider. Recorded examples alone would not establish the class-wide bound for arbitrary registered models.

This is a blocking design decision: establish and enforce the model/tokenizer capability required for byte budgeting, or budget using the provider's verified count. This review leaves production behavior unchanged here and keeps review thread PRRT_kwDOUMoiI86ttIqV open. The 55 passing tests and fuzz results remain valid for their actual scope and are not presented as tokenizer certification.