Release encoder buffers at exactly their retained capacity

Merged
VA
2d ago
+154-14

Cause

Breaking commit: #821 be953d15c (Batch filesystem staging and reuse request encoding).

canonical_json_bytes_bounded and json_bytes_bounded hand back their output buffer with its whole capacity() still retained through charge. Every caller releases exactly capacity(). Before #821, the encoder sized its output in a separate pass and allocated it exactly, so capacity() == len(). #821 made the output grow geometrically (reserve_output), so the returned buffer now has slack.

Two sites charged that slack a second time:

  • store::history::canonical_bytes_for_bounds_with_work
  • the attachment-manifest check in fork::grants

As a result, every measured canonical encoding leaked its slack as owned bytes.

Fix

  • Delete the second slack charge at both sites.
  • Document the returned-capacity contract on both encoders.

No limits, tolerances or expectations change.

Main-gate failures fixed (all introduced at be953d15c, all passing at its parent d6e6422ca)

  • core::authority::effect_root_tests::exact_work_oracle_detects_restored_decimal_serialization (owned 22 vs 0)
  • distributed::tests::frame_cancellation::properties::drop_at_every_await_balances_ordinary_charges (956 left)
  • store::operations::native_publication_custody_tests::original_provider_publication_releases_exact_source_after_receipt_drop (3 vs 0)
  • store::tests::interactions::admission_content_keeps_original_published_bytes_until_last_alias ("history exceeds live logical byte allowance")
  • store::tests::measured_fork_tests::roots::native_root_binding_keeps_original_actor_scope_fence_and_indexed_retry (268435373 vs 268435456)

Regression guard: one exact-ownership oracle per adapter

  • History adapter (store::history::canonical_bytes_with_work ā†’ canonical_bytes_for_bounds_with_work): history_encoding_owns_exactly_its_capacity_at_every_length (every length 0..=1100) and ā€¦_for_any_text (proptest, 64 cases, shrinking). Each asserts the ledger owns exactly capacity() on return and exactly 0 after the caller's release.
  • Fork-grants adapter (fork::decode_fork_manifest): manifest_check_owns_exactly_its_schema_at_every_length (caption lengths 0..=600) and ā€¦_for_any_labels (proptest, 0ā€“3 items with any valid captions, including escapes and multi-byte text). Each asserts that on return the quota and the ledger own exactly the schema reservation, and 0 after the quota drops.
  • Both sweeps assert that encoder slack occurred, so neither oracle passes vacuously.
  • Negative controls, run as real production mutants on bb2:
    • Re-adding the deleted slack charge in fork/grants.rs fails only the two grants tests (e.g. owned 5198 vs schema 5136, minimal case labels=[]). Every history test still passes.
    • Re-adding it in canonical_bytes_for_bounds_with_work fails only the history tests and the five measured store/fork/distributed oracles (e.g. capacity+slack vs capacity). Both grants tests still pass.
    • So each restored charge is caught independently.

Why the merge gate missed it

PR CI (.github/workflows/static.yml and the others) runs no harness Rust test target. #821 merged with these lib tests red.

Verification (bb1/bb2)

  • At 14fabbf98, canonical lib set: the 4 new adapter tests, the effect-root oracle, publication custody, admission content and root binding all pass (10/10, 0.05 s).
  • cargo test --locked -p acyclic-harness --all-features --lib over the accounting cluster, fork::grants and executor::json:
    • Every failure above passes, and the new test passes.
    • The 6 remaining failures belong to two other #821 root causes, fixed in separate PRs (trace typing; array-index scan item charge).
  • Bisect: cluster at d6e6422ca gives 58/58 passed; at be953d15c it gives the same 11 failures as main 66ddfd542, with default features and with --all-features.

Guarantee

Class: F, detached accounting lifetime: two owners charged the same slack. Structural fix: the encoder's returned buffer carries its whole capacity() charge, and the caller releases exactly capacity(). That contract is documented on both encoders, and both extra charges are deleted, so no caller holds a second claim on the slack.
Proof: one exact-ownership oracle per adapter: an exhaustive length sweep plus a shrinking property over arbitrary text and captions, each with a non-vacuity check that slack occurred. Each adapter has its own production-mutant negative control, and both were run (see above).
Entry points: canonical_json_bytes_bounded and json_bytes_bounded had two callers that re-charged slack, the history adapter and fork::grants; both are now covered directly. The other callers release capacity() only and are covered by the five measured store, fork and distributed oracles listed above.
Gaps: none.

šŸ¤– Generated with Claude Code