fix(read-output): keep annotation rows, nested empties and continuations faithful

Merged
UN
4d ago
+1876-560

Four read-surface defects, fixed together because each one let a read deliver less than its own contract claims. All four were found while reviewing #1681, #1685 and #1687.

Changes

pm-e3hfv8: TOON annotation listings keep null rows

A malformed null row in comments, notes or learnings was dropped by the TOON sparse policy while count still reported it. The rendered rows then no longer matched the positions that --edit/--delete address.

  • src/sdk/read-output/positional-rows.ts declares the position-addressed collections (PM_READ_OUTPUT_POSITIONAL_ROW_PATHS).
  • The core renderer re-delivers every row of a declared path.
  • Well-formed listings render byte-for-byte as before. The general null-dropping contract is unchanged for every other payload; a negative control in the spec pins this.

pm-fkhdy8: TOON get --fields keeps nested empty collections

Empty collections were restored only at the top level of item, so linked.tests: [], linked.docs: [], schedule.reminders: [] and schedule.events: [] disappeared from TOON while JSON kept them.

  • The get facet roots move to the import-free src/sdk/query/get-facet-roots.ts.
  • Restoration is now recursive inside item and every selected facet. Nulls stay dropped.

pm-rrwh2o: typed refusal for a mutation given output controls

The refusal used to be command_failed with pm --help examples. It is now read_output_mutation_conflict, exit 2, which is catalogued and listed in the refusal-closure census. It carries:

  • details.output_controls: the CLI flags;
  • details.output_option_keys: the keys an SDK or MCP caller supplied;
  • details.mutation_options;
  • a required line and runnable per-surface examples.

src/cli/runtime/read-output-mutation-retry.ts rewrites the CLI recovery as the caller's own argv without the controls and the values they consumed. Flag arity comes from the shared scanOptionTokens (in projection-retry.ts), and namespaced paths such as item comments resolve.

pm-sbqn4h: MCP pm_comments continuations reach the oldest comment

Listings applied an implicit 20-row producer page that predates the universal output budget, so the backward cursor stopped at that page's edge while has_more stayed true.

  • Listings now leave the bound to the default budget, as pm_notes, pm_learnings and the CLI already do.
  • options.limit=N still selects the newest N.
  • The tool description and contract snapshot are updated.

Verification

  • New specs:
    • tests/unit/core/output/annotation-read-fidelity.spec.ts (unit and CLI subprocess, both items, negative controls)
    • tests/unit/cli/runtime/read-output-mutation-retry.spec.ts (refusal shape, argv repair, CLI retry that runs)
    • tests/unit/mcp/comments-continuation.spec.ts (tools/call from index 30 to index 1 following next_cursor alone; has_more equals cursor presence on every page)
  • Updated specs: tail-retention.spec.ts (new message) and mcp-dynamic-package-actions.spec.ts (new MCP default, plus the explicit limit page).
  • Full suite: 12310 passed. The two local failures were the old MCP expectation (fixed) and a load timeout that passes alone.
  • Coverage of every added line in the changed src files was intersected with the diff: complete.
  • Local gates pass: contracts, agent surfaces, error catalog, SDK surface, refusal closure, flag lexicon/parity, command grammar, graph-composition, record-integrity, defect-evidence and changelog:pm:check.
  • Run locally under host load average 12–25: the SDK import-cost and CLI transport-floor benches exceeded their budgets. Hosted CI is the authority there.

Tracker

  • The four items are closed with structured resolution/expected/actual, review_caught_late escape class and gate evidence.
  • A scope note on pm-yo46im says pm validate should report non-object annotation entries by index.
  • CHANGELOG.md is regenerated by pm-changelog.

Summary by Sourcery

Restore faithful read results across TOON, mutation recovery, and MCP comment pagination while preserving safe, contract-driven output behavior.

Bug Fixes:

  • Preserve position-addressed annotation rows in TOON output, including malformed null entries, so rendered positions and counts remain faithful.
  • Restore nested empty collections in explicitly selected TOON get fields and facets to match JSON output.
  • Return a typed refusal when mutations are combined with universal output controls, including structured recovery guidance and safe CLI retry handling.
  • Allow MCP comment continuations to page from the newest comments through to the oldest without an implicit producer-page boundary.

Enhancements:

  • Improve read-output contracts, error cataloguing, recovery documentation, and coverage for the corrected CLI, SDK, TOON, and MCP behavior.

Build:

  • Exclude the ecosystem planning review from the npm package to restore publication size headroom.

Documentation:

  • Document positional TOON rows, recursive empty-collection restoration, mutation/output-control conflicts, and MCP comment pagination semantics.

Tests:

  • Add unit, CLI, and MCP coverage for annotation row fidelity, nested empty collections, mutation recovery, and comment continuations.

Chores:

  • Regenerate changelog and update tracker records for the resolved read-output defects and related package publication follow-up.

Summary by cubic

Fixes four read-output defects where a read delivered less than its own contract claimed, and restores npm publication headroom.

  • TOON annotation listings keep malformed null rows, so rendered rows match the positions --edit/--delete address.
  • TOON get --fields restores empty child collections recursively inside selected facets, matching JSON, even when the output projection removed item.
  • Mutations combined with output controls are refused with read_output_mutation_conflict (exit 2) instead of generic command_failed; the CLI retries the caller's invocation without those controls.
  • The retry is withheld when a control consumed a flag-like token, directly follows a value-taking option, runs inside an enclosing SDK revision scope whose guards never reached argv, the argv carries URL credentials, or any control spelling remains in the rebuilt argv. Credential detection is now a single conservative rule — any token containing both @ and : withholds the retry — so a credentialed package source never produces a retry that repeats the secret, at the cost that scoped package names and @mentions beside a colon also lose the convenience retry.
  • MCP pm_comments listings no longer apply an implicit 20-row producer page, so the backward cursor reaches the oldest comment. options.limit=N still selects the newest N, and a next_cursor pages through budget-trimmed rows inside that window; outputBudget "unbounded" returns the complete listing only without options.limit/outputLimit, and a budget too small for one comment omits the listing with output_budget_exceeded.
  • The shared error-catalog recovery line now tells callers to retry the suggested command only when one is present, since a withheld retry has none.
  • docs/ECOSYSTEM_PLANNING_REVIEW.md is excluded from the npm tarball to bring the package back under its 20 MB ceiling.
  • Known gaps: pm contracts has no entry for files discover (tracked as pm-9ke7h9), and refusal attempted_command/normalized_args echo URL credentials verbatim for every refusal (tracked as pm-nhauur).