Resolve the default compaction policy from the bound task

Merged
VA
2d ago
+779-101

Supersedes #1012 (rebased onto main).

Problem

bundle::tests::builder_rejects_missing_or_conflicting_bindings fails on main in the gate run cargo test --locked -p acyclic-harness --all-features --no-fail-fast (baseline 66ddfd542). It fails the same way under harness-only features and with acyclic-harness-providers, so feature unification is ruled out.

Breaking commit: #898 (b66d74738). It rejects threshold compaction at turn entry unless the task can publish private summaries. It switched only filesystem/task_runtime/execution.rs to Disabled where the task cannot. Every other stock composition, HarnessBuilder among them, still defaulted to Threshold. Those turns now fail with Unauthorized("summaries require the task's private volume") before any model work.

Why the gate missed it: PR CI runs no harness Rust test target, so this lib test never ran before merge.

Fix

  • StockExecutor and HarnessBuilder hold Option<CompactionPolicy>.
  • When no policy is set, StockExecution::compaction_policy() resolves it from the bound task: threshold where the bound task has a private summary writer, disabled for a bound task without one; an unbound standalone executor retains threshold. That method is the single place the default is decided.
  • An explicit policy is unchanged and still verified at turn entry.
  • The task runtime's own special case is removed, because the executor now owns the default.

Review round: single resolution and exact accounting

  • One compaction resolution per turn. StockExecution holds compaction: once_cell::sync::OnceCell<ResolvedCompaction>, which is the policy plus the task's typed private summary destination. compaction() resolves it on first use. The entry check (binding::verify_summary_destination), the policy read and the summary path (executor/summary.rs) all read that one value, so the writer is no longer re-resolved three times on the original native IO.
  • The exact preparation table no longer depends on features. Under --features filesystem, preparation_differential differed from --all-features by 32 bytes. An allocation trace shows exactly one allocation differs: the box BoundedContentFuture::new makes for SourceStage's projection future (472 vs 440 bytes).
    • That future holds the source's BoundedContentFuture across its await. Inside it is a NativeAllocation → NativeMaterializationWork, which carries a field gated on acyclic-fs/native-mount.
    • The future is now named: context::source::project_source is a plain fn returning the same async move block, so its upvar layout and size are unchanged. An async fn was measured at +136 bytes and rejected.
    • source_projection_future_bytes() (cfg(test)) takes size_of of that function's output type without calling it. preparation_expected.json stores the feature-independent base, and the test adds the derived term. Every comparison stays exact.
    • The test future is pinned on the stack (std::pin::pin!), so tokio's boxing of large block_on futures no longer enters the measurement.

Author validation (before review fixes)

  • bb1, this branch fd5e2c3d8 (#1012's two commits rebased onto main 4b9c84541): --lib -- executor:: bundle::
    • --all-features: 108 passed, 0 failed.
    • --features filesystem,hooks: 108 passed, 0 failed.
  • bb1, pre-merge commit 96dd6aac6, --features filesystem --lib executor::: all preparation, compaction and bundle tests pass. The one failure there is listed under the issues found while verifying this PR.
  • git diff 44f024e83 fd5e2c3d8 is byte-identical to git diff 7c05f9cdc 4b9c84541, so the tree equals #1012's verified tip plus main's changes alone. The negative control below was run at 44f024e83.
  • Formatting: rustfmt --check on every file this PR touches is clean.

Guarantee

  • Class: M: parallel mechanisms, in three places, each replaced by one derivation point.
    • Compaction default. It was decided in two places. Now StockExecution::compaction_policy() derives it from the bound task, and the old default is no longer representable.
    • Writer resolution. It was re-run per consumer. Now one once_cell::sync::OnceCell<ResolvedCompaction> serves the entry check, the policy read and the summary path.
    • Allocation table. It hardcoded values for one feature set. Now it holds a feature-independent base plus size_of of the one allocation whose type varies by cfg.
  • Proof:
    • Default branches. The bundle test covers both. filesystem::task_runtime::tests::canonical_compaction covers the private-volume threshold branch. compaction_policy_matrix_resolves_through_the_task_writer runs every writer Ɨ override Ɨ session combination through the real resolver.
    • Single resolution. compaction_resolves_once_with_original_work checks exactly one writer resolution on the original ledger. Negative control: repeated_resolution_negative_control_is_detected restores the pre-change repeats and asserts both detectors trip (allocations and ledger receipt). It runs in the suite.
    • Exact table. request_preparation_has_exact_physical_allocations passes under --all-features, --features filesystem (96dd6aac6) and --features filesystem,hooks (this branch). Negative control, actually run under --features filesystem,hooks: replacing the derived term with the old hardcoded 472 fails with actual=[178,48,73,24148,4695,8728], expected=[178,48,73,24180,4695,8728]. A mutant using an async fn was caught by the exact table (+136 bytes) before it was rejected.
  • Entry points:
    • Every StockExecution construction (ordinary, native context, native stock composition, summary execution, native session) reads the policy and writer only through compaction() / compaction_policy().
    • Policies arrive from HarnessBuilder, the filesystem task runtime, local::bind, task_commands::model and StockExecutor::new.
    • SourceStage::apply is the only constructor of the projection future.
  • Gaps: none.

Issues found while verifying this PR (separate root causes, not changed here)

  • Main's lib tests don't compile with --features filesystem alone (7c05f9cdc). runtime/tools.rs:216 is gated cfg(all(test, not(wasm32))) but reads the hooks-gated crate::hooks::mcp_identity_tests, which fails with E0433. Introduced by #982 (32a9ae569) and reported to the coordinator. This is why the tip was verified with filesystem,hooks. #1028 fixes it.
  • --features native-execution alone fails to compile with a mounts_mut error on MountTable. This exists without this PR.
  • executor::json::differential_properties::exact::encoding_has_exact_production_work_physical_allocations_and_original_custody failed under --features filesystem at base 66ddfd542 (ledger/trace differs). It passes on this branch after #998.

šŸ¤– Generated with Claude Code


Summary by cubic

Fixes compaction policy defaults so stock compositions no longer reject turns at entry when the bound task can't publish private summaries. An unset policy is now resolved from the bound task — threshold where it has a private summary writer, none elsewhere — while explicit policies stay unchanged and are still verified; the filesystem task runtime's hardcoded default is removed.

  • StockExecution resolves the policy and private summary destination once per turn via a fallible OnceCell, shared by the entry check, policy read, and summary path.
  • The exact preparation allocation table is feature-independent: the source projection future is a named function and its box size is derived from the type instead of hardcoded per feature set.

Codex review pass 1

  • Preserve execution failures. The original .ok() erased every writer-resolution failure, including native budget exhaustion and cancellation, and let the default select Disabled. A fallible resolver now returns the original cause and custody through every consumer. Explicit Threshold rejects that first error, without a second lookup in the entry verifier.
  • Resolve unavailable authority without discarded errors. SummaryDestination separates validated private authority from an unavailable destination with a static reason. The default can select Disabled without allocating and dropping an Unauthorized diagnostic. Manual summaries reuse that destination and construct exactly their one returned diagnostic. The existing private_summaries API delegates to the same resolver, and volume-write authorization shares the original grant-check function.
  • Make the successful reference actually succeed. The old native allocation probes funded only accounting-state memory, so their supposed successful writer check failed. The old .ok() concealed that failure. The probes now use the existing production boundary allowance and assert that the writer is authorized; the old state-only allowance remains an explicit failure injection. No production limits were raised.
  • Successful resolutions use once_cell::sync::OnceCell::get_or_try_init, the stable fallible initializer, to return owned errors without cloning, stringifying or discarding native custody. Subsequent successful reads allocate and charge nothing. Failed admission is terminal and does not need an error cache.

Guarantee amendment

  • Class: cause erasure and redundant work at optional-feature admission. One fallible resolver distinguishes typed unavailable authority from execution failure. Every policy consumer propagates its Result. A validated writer token can only be constructed by the shared authority resolver; unavailable authority has no discarded error allocation.
  • Proof:
    • compaction_policy_matrix_resolves_through_the_task_writer enumerates all 12 writer Ɨ override Ɨ ordinary/summary states through the production resolver.
    • native_resolution_failures_reject_before_journal_or_model_work injects hash allowance, allocation-operation allowance, item allowance, cancellation, and native allocation-memory failures into the real private-writer check on its original ledger, for default and explicit Threshold. It checks Work/Cancelled/NativeFilesystem at entry and rejection without journal effects. Its negative control applies the original .ok() fallback to the same failed check and demonstrates successful Disabled admission.
    • compaction_resolves_once_with_original_work compares entry verification with exactly one independent writer check, requires actual writer authorization, and compares exact physical allocation counts/bytes and native receipts. Later reads must have zero allocations and unchanged receipts. repeated_resolution_negative_control_is_detected restores repeated authorized lookups and trips both detectors.
    • writerless_summary_reports_one_error_without_repeating_resolution runs the real manual-summary path for all three overrides before/after policy resolution. Physical allocation accounting must equal exactly one returned Unauthorized diagnostic. Restoring the discarded diagnostic plus repeated lookup fails that same allocation bound.
    • The existing preparation properties and hardcoded-size negative control remain, along with summary-authorization shrinking properties and their deferred-admission control.
  • Entry points: task binding/entry validation, request digest, model capacity, output budget, response accounting/projection/replay, automatic compaction, and explicit summary admission all propagate the same resolver result. Original and differential preparation implementations propagate the same failures. Native context and native stock execution enter through the same binding verifier. TaskContext::private_summaries and all volume publication retain the shared grant check.
  • Gaps: none.

Review validation

All commands ran through Q:/lanes/bs.ps1 (routed to bb2), with one feature set: filesystem,hooks, debug profiles only.

  • Production-final focused run: cargo test --locked -p acyclic-harness --features filesystem,hooks --lib -- executor:: bundle:: filesystem::task_runtime::tests::canonical_compaction runtime::content_write::. 112 passed in 3.80 s; the additional bundle negative-control fixture failed because it wrongly required an empty journal for Disabled. Only that assertion changed afterward: Disabled must record exactly one Started event, whereas default/Threshold must record none. Build 4m10s, wall 409.68 s including 100 s queue. Log: Q:/lanes/passes/1032/pr-1032-gate-2.log.
  • Corrected fixture: cargo test --locked -p acyclic-harness --features filesystem,hooks --lib -- bundle::tests::private_writer_defaults_require_capacity_before_bundle_turn_effects. 1 passed, 0 failed, 0.00 s. The 112 unchanged production proofs were reused; no redundant rebuild or suite rerun was made to reconfirm them.
  • Windows compile proof on the final source: cargo check --locked -p acyclic-harness --lib --tests --features filesystem,hooks --target x86_64-pc-windows-gnu. Passed in 3m06s. Corrected-test build 1m17s; combined final test/cross-check wall 426.28 s, including 70 s admission queue and snapshot overhead. Log: Q:/lanes/passes/1032/pr-1032-gate-3.log.
  • The initial review gate exposed the underfunded successful-reference fixtures and an invalid ancestor-scope fixture; those failures were diagnosed and corrected before the production-final run. No production budget or limit was raised.
  • git diff --check, rustfmt on all touched Rust files, LF/line-count checks, and all renamed/changed private API caller scans passed. No unresolved review threads.

Guarantee matrix G01 / GREEN-capacity

The baseline row diagnosed the changed error as a missing capacity check. The actual cause was Threshold selected for a task unable to publish summaries. Writerless defaults now run Disabled; explicitly armed writerless tasks reject Unauthorized before capacity lookup. private_writer_defaults_require_capacity_before_bundle_turn_effects runs the shipping builder/run entry point with private content and tests default, explicit Threshold, and explicit Disabled: the first two retain the exact missing-capacity error before effects; Disabled is the negative control reaching the journal instead. The existing resolver matrix detects incorrectly disabled writer-backed defaults. No catalog/parser code changes are involved.

Standalone executors without any TaskContext retain their existing Threshold policy and journal-backed summaries; impossible_default_capacity_fails_before_turn_admission covers their capacity gate. The private-volume default applies when an actual task context is bound.