Observe consented shadow policies before promotion
Observe a candidate alongside the active policy on explicitly consented inputs without giving the candidate tool or mutation authority. The typed SDK and pm rl shadow register/observe/compare pin policy artifacts, consent, a frozen cohort and both-arm allowances; existing generation promotion and the recursive controller require a validated shadow comparison throughout online-bound lineages.
Correctly hashed original records previously could pass history checks while claiming inconsistent outcomes, and manifest reload omitted registration-time cohort/budget checks. The promotion gate now revalidates the evidence with the same validators used by registration and live observation.
| Blocking finding | Fix |
|---|---|
| Original individual receipts could claim rewards 0 and 1 for identical active/candidate actions, or falsify disagreement. | One shared receipt validator resolves the consented input through the pinned policy table, verifies both actions, finite bounded rewards, same-action reward equality, disagreement/failure flags and the selected-outcome digest. Observation, retry/resume, comparison and promotion all use it. |
| An original consented manifest with zero allowances or an expanded cohort could bypass registration checks on reload. | One shared manifest validator rechecks the exact generation pair, consent, frozen cohort and both allowances on registration and every reload. Certified SDK inventory counts actual unique observation slots, including unexpected extras; aggregate charges must match validated evidence. |
Every shadow validation mismatch refuses with the exact receipt in context.item_id, the invalid context.field, and an actionable message. Invalid receipts never authorize promotion. Verification remains read-only inside the existing authoritative promotion transaction; contamination, score, approval and promotion-budget checks remain required. No parallel promotion path or autonomous switching API is added. SDK/dev dependency pins and all gate thresholds are unchanged.
Four requested real-SDK reproductions cover identical-action rewards, an original zero-budget manifest, a tampered disagreement flag, and a manifest cohort expanded after plan registration. Each fails 1/1 when only the corresponding shared validator's behavior is reverted; failures are Missing expected rejection, proving prior admission rather than a test/setup error. Restoring validation passes 4/4 with Git absent from PATH and an explicit discovery ceiling. The complete focused suite passes 25/25 at 100% shadow lines/branches/functions. Additional original-record tests cover action identity, reward bounds, failure/digest fields, slot identity, actual extra charges, both aggregate charge fields, post-registration edits, prototype-like action names and native SDK workflow-policy refusal. No SDK methods are mocked and no history records are rewritten. Typecheck, test typecheck, lint, zero duplication, docstrings, changelog checks and linked PM execution (1/1) pass.
Repacked public acceptance reuses the existing read-only, non-Git MIT unbraind/pm-context tracker copy at commit f55cdbbcf2447dae29df789e133fdbe5521a47b6 and reproduces docs/shadow-evidence.json byte-for-byte: 84 certified items, first 20 sorted IDs selected without outcome filtering (17 closed, 3 other), 20/20 active actions preserved, 20 disagreements, 0 candidate failures, 20 active + 20 candidate calls, and all 175 source files unchanged. The single repository is one independent unit, so promotion correctly refuses despite empirical means 0.15 active and 0.85 candidate. Observation JSON is 548 bytes; comparison JSON 1,278 bytes, TOON 1,127 bytes. This measures observation/gating, not learned-policy improvement. See docs/shadow-policy.md and test/shadow.acceptance.ts for reproduction.
Current-head complete CI passes at 3d6e1f8ec6487a5e949c61c93170688cd0533d00:
| Complete release gate | Runtime | Tests | Statements / lines / branches / functions |
|---|---|---|---|
npm run release:check | Node 22 | 744/744 | 100/100/100/100 |
npm run release:check | Node 26 | 744/744 | 100/100/100/100 |
bun run release:check | Node 26, Bun 1.3.14 | 744/744 | 100/100/100/100 |
All three completed job logs were independently downloaded and checked: zero failures, skips, cancellations or todos; all 50 authored source files measured; 358 declarations documented. Fresh strict tracker health, typecheck/build, lint, zero duplication, isolated identity, production audit, publish attestation, packaging, changelog and Bun graph installation pass. Current-head CodeQL passes. Previous-head gate evidence is historical only. No local full gate is claimed on the saturated host.
PM: pm-rl-shadow, under pm-rl-apvf. Revert proofs are recorded on the item; its claim is released and it remains in progress for orchestrator verification. Drafts #95 and #98 remain unchanged. No review-thread replies, mentions, merge or release. Automated reviewer skips/unavailable access are not approval.
Summary by Sourcery
Require consented, fail-closed shadow observation evidence before promoting online policy successors.
New Features:
- Add bounded shadow deployment support through typed SDK APIs and
pm rl shadow register/observe/comparecommands. - Allow candidates to observe explicitly consented inputs while returning only the active action and withholding mutation or tool authority.
Bug Fixes:
- Prevent candidate failures, forged or edited receipts, incomplete cohorts, budget violations, and unreserved online successors from affecting active actions or bypassing promotion controls.
Enhancements:
- Require sealed, independently validated shadow outcome comparisons within the existing transactional generation promotion gate.
- Bind consent, policy artifacts, input identities, generation lineage, and separate active/candidate call budgets to immutable receipts.
- Preserve retry-safe observation semantics and reuse the existing paired evaluation and statistical promotion criteria.
Build:
- Package the new shadow SDK artifacts, documentation, example configuration, and acceptance evidence.
CI:
- Extend the test suite with shadow observation, receipt integrity, budget, concurrency, lineage, CLI, and non-Git promotion coverage.
Documentation:
- Document the shadow consent, budget, bounded policy, receipt, comparison, and promotion contract.
Tests:
- Add public-tracker acceptance validation and comprehensive fail-closed regression coverage for shadow deployment and promotion.
Chores:
- Record the shadow feature specification and implementation history in project metadata.
Summary by cubic
Adds a shadow observation mode that lets a candidate policy see the same explicitly consented inputs as the active policy without gaining tool or tracker mutation authority. Online candidates now need a sealed, validated shadow outcome comparison, revalidated against evidence and budgets, through the existing promotion transaction before they can replace the incumbent in that lineage.
New Features
- Adds
pm rl shadow register/observe/compareand typed SDK exportsregisterShadowDeployment,observeShadow, andcompareShadowfor bounded declarative lookup policies. - Consent pins the exact generation pair, both policy digests, minimized input hashes, and separate active/candidate call allowances.
Behavior
- Candidate failures preserve the active action and consume the reserved candidate attempt; identical retries add no calls.
- Conflicting outcomes, edited evidence, incomplete cohorts, and forged aggregate receipts refuse with stable codes and remedies.
- The shadow requirement follows the lineage: any successor of an online-bound incumbent refuses without shadow evidence, blocking recursive-controller bypasses.
- Offline recipe selection keeps its existing contract; promotion still uses the existing generation gate.