Native list pagination and bounded output contracts

Merged
UN
16d ago
+5314-100

Native JSON list reads previously required explicit unbounded policies and could not continue a page. This adds producer limits/offsets, semantic query cursors, snapshot output continuations, triage token budgets, output ceilings, compaction, omission receipts, and the documented Workspace::list_page SDK API against published PM CLI 2026.10.5.

Both real executables run over shared fixtures and compare complete stdout/stderr bytes and exit codes with a fixed read clock. Tests cover multi-page walks, empty/final pages, filter mismatch, separate-process mutation and stale snapshot refusal, producer cursor rebasing, bounded receipts, and 96 randomized walks with no missing or repeated rows. The original nine unbounded golden envelopes remain unchanged.

Validation: just release-check passed all 183 tests, formatting, deny-warning Clippy, public/private rustdoc, dependency audit and generated changelog verification. Coverage is 100%: 2,892 lines, 251 functions, 4,383 regions, and 604 branches. pm test pm-rust-ufzy --run --progress passed all 15 linked differential tests. The CLI/dev/CI pin is 2026.10.5; gate thresholds remain unchanged, and stale coverage output is invalidated before instrumentation.

This PR remains draft. UTF-16 string compaction that splits a surrogate pair still differs from the reference and requires a JSON representation change. An unchanged legacy lock-replacement test also failed intermittently once, then passed isolated and aggregate retries; that fixture needs investigation. Numeric controls and native selection retain the documented slice boundaries. Changelog regeneration through the pinned pm-changelog flow produced no new entry because it selects closed items; all tracker items remain open for orchestrator verification.

Tracker: owner pm-rust-ufzy, epic pm-rust-u4al, Unicode follow-up pm-rust-8hkb, fixture follow-up pm-rust-ax0o.

Summary by Sourcery

Implement the PM CLI 2026.10.5 native list pagination and bounded-output contracts while preserving existing unbounded compatibility.

New Features:

  • Add native JSON list pagination through producer limits, offsets, semantic cursors, snapshot continuations, and the public Workspace::list_page SDK API.
  • Add bounded list and triage output policies with token budgets, row ceilings, compaction, omission receipts, and continuation recovery contracts.

Bug Fixes:

  • Align query fingerprints with published CLI path resolution across nested, parent-relative, and Windows path inputs.
  • Invalidate stale coverage artifacts before instrumentation and preserve existing unbounded list envelope compatibility.

Enhancements:

  • Expose machine-readable cursor refusals and preserve legacy list SDK methods while routing CLI JSON reads through the paginated API.
  • Document the 2026.10.5 list compatibility and pagination contracts, including supported controls and remaining Unicode limitations.

Build:

  • Update the project compatibility and tooling pins to PM CLI 2026.10.5 and add the base64 dependency required for cursor handling.

CI:

  • Update CI and release workflow invocations to PM CLI 2026.10.5 and retain byte-exact differential validation across both executables.

Documentation:

  • Document native pagination, bounded output, triage behavior, continuation semantics, and compatibility boundaries.

Tests:

  • Add differential, SDK, unit, randomized pagination, mutation, stale-snapshot, cursor, compaction, and output-budget coverage for the new read contracts.

Chores:

  • Record follow-up tracker items for the remaining Unicode compaction discrepancy and intermittent lock-replacement fixture failure.

Follow-up items opened during review

  • pm-rust-8qbo: mirror the upstream --output-limit continuation fix (pm-cli#1420) once it ships.
  • pm-rust-rnex: install the published-CLI oracle from a committed lockfile in CI (pm-cli#1417).
  • pm-rust-otel: transient OpenTelemetry 2.12.0 registry propagation that failed one CI run.
  • pm-rust-cwlf: Windows CRLF checkout broke changelog-check; fixed here with LF-pinned .gitattributes.

Summary by cubic

Native JSON list reads no longer require explicit unbounded policies and can now continue across pages. They match the published PM CLI 2026.10.5 pagination and bounded-output contracts: producer limits and offsets, query cursors, snapshot output continuations, triage token budgets, output ceilings, compaction, omission receipts, and the Workspace::list_page SDK API. list --json accepts default bounds, and the original nine unbounded golden envelopes remain unchanged.

Also included

  • History hashes, snapshot fingerprints, wire JSON, and token estimates share one JavaScript number renderer; recovery arguments strip --workspace=/--timestamp= spellings.
  • Windows sharing-violation lock contention is classified as contention, denied-listing fixtures use real ACL denials, and release gates run on Ubuntu, macOS, and Windows with coverage-gap reporting; .gitattributes forces LF so changelog-check compares byte-identical files.

Remaining gaps

  • UTF-16 string compaction that splits a surrogate pair still differs, .toon extra metadata keys sort alphabetically, and a cursor without after_index publishes the query fingerprint instead of opaque_cursor.
  • A page cut by both --output-limit and --output-budget carries an output cursor the published CLI refuses as stale on an unchanged workspace; upstream fix tracked as unbraind/pm-cli#1411.
  • Following next_cursor after an --output-limit trim skips rows; the published CLI does the same, with the upstream fix tracked as pm-rust-8qbo.
  • The pinned CLI install can fail transiently while telemetry dependencies propagate on the registry; a committed-lockfile install is tracked as pm-rust-rnex.