SDK evidence traceability and context-integrity primitives

Merged
UN
3mo ago
+2963-134

Outcome

Ships a cohesive SDK/CLI context-integrity tranche: indexed source-to-item evidence lookup, atomic evidence replacement, truthful no-op and telemetry receipts, reliable linked-test classification, duplicate identity preservation, Windows portability, and transactional history-redaction cache refresh.

This advances the project philosophy that project management = context management by making evidence discoverable and safe to maintain through the same typed primitives across CLI, SDK, PmClient, one-shot actions, and MCP.

Major changes

  • Adds bounded pm files lookup <path...> reverse traceability with cursor-bound SQLite projection, incremental refresh, pagination, strict/source-scan completeness receipts, absolute-path canonicalization, and CLI/SDK/MCP parity.
  • Adds atomic --replace-files and --replace-docs update contracts, including bulk preview, completions, guarded argv vocabulary, rollback behavior, and generated schemas.
  • Makes duplicate linked file/doc additions true no-ops without item or history rewrites.
  • Separates telemetry queue progress from completion with queue_entries_drained, queue_progressed, queue_empty, and completion-correct queue_drained.
  • Classifies narrow linked-test lock acquisition contention as infra_collision while retaining real assertions as assertion_failure.
  • Preserves meaningful suffixes in external issue-code duplicate scoring.
  • Makes workspace/global path regressions portable across Windows, macOS, and Linux.
  • Serializes history-redact with metadata-index refresh and drift-cache invalidation on commit and rollback.
  • Documents the public evidence/integrity contracts and regenerates contracts, SDK surface, and changelog through pm-changelog 2026.8.2.

Canonical PM lineage

Verification

  • Full sandboxed coverage: 418 files / 6,703 tests, exact 100% statements (47,530/47,530), branches (34,246/34,246), functions (9,673/9,673), and lines (46,120/46,120).
  • Strict lint, typecheck, zero-clone duplication, 100% docstring/static quality, package boundary, SDK parity/surface, token-budget, context-intent, compatibility, secret scan, and dependency audit gates pass.
  • Exact-head nightly matrix run 30747290823 passes Windows Node 24, macOS Node 24, and Ubuntu Node 22/24/25.
  • pnpm smoke:npx passes the generated tarball across all 15 packaged modules.
  • Independent unrelated-root npm and Bun consumers installed the tarball, initialized isolated trackers, created work, linked and looked up evidence; npm also proved atomic replacement and repeated-add changed=false.
  • pnpm changelog:pm:check, history verification, and graph audit pass with zero drift and zero active isolates. Existing historical resolution debt remains unchanged at 295 items.

Release boundary

The immutable automatic release v2026.8.2 already exists for 2026-08-02, so this PR must not create or move a second same-day auto release. The merged changes are eligible for the next daily auto release; packed npm/Bun consumer acceptance supplies pre-publication proof.

Closes #779
Closes #861
Closes #865
Closes #866
Closes #867
Closes #868
Closes #870
Closes #871

Summary by Sourcery

Introduce reverse source-to-item evidence lookup, atomic linked-file/doc replacement, and more precise telemetry, test, and similarity governance across CLI, SDK, PmClient, one-shot actions, and MCP.

New Features:

  • Add bounded pm files lookup for reverse linked-file traceability with pagination, index-backed reads, and source-scan completeness receipts.
  • Expose files lookup as a first-class SDK primitive, in PmClient, one-shot helpers, MCP actions, and CLI subcommands with guarded contracts and shell completions.
  • Extend update and update-many to support atomic replacement of linked files and docs alongside existing dependency and test replacement semantics.
  • Publish an SDK evidence traceability and integrity guide documenting reverse lookup, atomic replacement, no-op behavior, telemetry receipts, and similarity contracts.

Bug Fixes:

  • Ensure history-redact rewrites refresh item metadata and history drift projections atomically, invalidating drift caches on both success and rollback.
  • Treat repeated linked file/doc additions as true no-ops that report changed=false and avoid emitting phantom history entries.
  • Classify linked-test lock acquisition contention as infrastructure collisions while preserving genuine assertion failures as assertion_failure.
  • Preserve letter-suffixed external issue codes (for example BD-30-A vs BD-30-B) in duplicate similarity scoring to avoid collapsing sibling work.
  • Correct telemetry flush receipts so queue_drained only reports full drain, with explicit fields for entries drained, progress, and queue emptiness.
  • Harden workspace path assertions and snapshot expectations to behave consistently on Windows, macOS, and Linux.
  • Prevent corrupt or stale metadata-query indexes from serving reverse lookup results by falling back to authoritative source reads when needed.

Enhancements:

  • Extend the item metadata derived index and SQLite projection to store linked-file records separately for efficient reverse traceability queries.
  • Tighten linked-artifact mutation semantics so change detection respects stable value equality and skips item writes for true no-op updates.
  • Relax MCP and tool schema contracts for files so actions can target either an item id or reverse lookup paths with clear parameter metadata.
  • Improve CLI flag contracts, option normalization, and shell completions to cover new files lookup and replace-files/replace-docs capabilities.
  • Adjust test failure classification patterns to recognize compound lock-contention messages without over-matching generic lock-related assertions.

Documentation:

  • Add SDK Evidence Traceability and Integrity documentation and link it from the main README and docs index.
  • Update the changelog with fixes and contracts for history-redact projection integrity, Windows portability, suffixed issue-code identity, linked-test classification, telemetry receipts, linked evidence no-op behavior, and atomic replacement.

Tests:

  • Introduce comprehensive context-management SDK primitive tests covering files lookup across transports, atomic evidence replacement, no-op behavior, index-backed reverse queries, telemetry flush receipts, and lock-collision classification.
  • Extend existing validation, similarity, runtime, and action-schema parity tests to cover new index refresh behavior, issue-code patterns, files lookup contracts, and Windows-safe path expectations.
  • Add TypeScript surface tests to guarantee availability of filesLookup, runFilesLookup, and telemetry flush receipt types from the public SDK entrypoint.

Chores:

  • Bump the metadata-query-index schema version and adjust caching thresholds and manifest handling to support the new linked-file projection.

Summary by cubic

Adds reverse source-to-item lookup and hardens evidence integrity across the SDK and CLI. Enables atomic evidence updates, precise telemetry receipts, and consistent indexing/history across platforms.

  • New Features

    • pm files lookup <path...>: indexed reverse traceability with pagination, strict/source-scan receipts, scope validation, authoritative index joins with source-read fallback, and absolute-path canonicalization; available via CLI, filesLookup, runFilesLookup, and PmClient.filesLookup.
    • Atomic replacement for linked files/docs with --replace-files and --replace-docs; duplicate adds are true no-ops and do not write history or migration receipts.
    • Updated schemas, flags, and shell completions; new docs at docs/SDK_EVIDENCE_TRACEABILITY.md.

  • Bug Fixes

    • Telemetry flush now reports queue_entries_drained, queue_progressed, and queue_empty; queue_drained is true only when empty; exposes TelemetryStatusSummary and TelemetryFlushResult.
    • Linked-test lock acquisition contention classified as infra_collision (not assertion_failure).
    • Preserves letter-suffixed issue codes (e.g., BD-30-A) in similarity scoring.
    • Windows portability: native path assertions and absolute-path lookup canonicalization.
    • history-redact refreshes the derived metadata index and invalidates drift cache atomically on commit/rollback, and preserves error precedence.