Profile summary history runtime and evaluate directory lookup savings

Draft
VA
3d ago
+386-41

Closing this follow-up because the candidate does not establish the required runtime headroom. All 32 generated histories, shrinking, authenticated production paths and assertions remain. The production candidate has a small measured gain, but its final profile includes an updated run above the 10s gate; this is not an acceptable performance result to merge.

Profile

Temporary per-case timing of all 32 histories at the original published head fafa0e483 took 6.705s on bb1. Combined stage time:

StageSecondsShare of run
Parent history creation and compaction3.97559.3%
Capture1.19217.8%
Publication0.5458.1%
Preparation lease setup0.4146.2%
Authenticated summary-context preparation0.1702.5%
Authenticated child read0.1241.9%
Other setup, payload reads and rejection checks0.2854.2%

The two real parent turns alone took 3.674s. The second turn (compaction) was consistently slower. Thirty stack samples show substantial work in real filesystem path resolution, accounting, page writes, hashing and JSON processing. Runtime construction is not the dominant cost.

Candidate and Decisions

The signed candidate e48986b03d45afc51a7fa371c9151c6b0a01b5b4 retains the originally requested shared runtime and regression-only proof load/source clone. It additionally changes production Transaction::create_dirs: an already existing nested directory is resolved by one complete no-follow lookup, instead of resolving every prefix from the root. Missing directories retain the original creation loop. It preserves native work validation, accounting, cancellation and observed namespace identities.

A new input-tree property checks success/rejection, exact directory kinds, existing identities and whether the transaction changed state. It covers 64 shrinking histories, five observation faults and nine bounded two-transaction rebase histories. Every ancestor replacement must conflict; an unrelated peer write must rebase. Omitting the directory observation is a separate negative control. This is a namespace/dependency invariant check; it is not a new claim about arbitrary filesystem thread interleavings.

The first paired probe used production code from 9286b1135, identical to the final candidate's filesystem code. The later commit only fixes the new model's handling of empty transactions. Both binaries retain identical temporary timing instrumentation, all 32 cases and the same seed; order alternates:

SeedOrderBeforeAfter
750before then after6.649s6.461s
751after then before6.554s6.459s
752before then after6.711s6.568s

Median 6.649→6.461s, about 2.84%. Parent-history time fell in every pair, while capture/publication were roughly unchanged. This is a small gain, not a solution to the dominant pipeline cost.

Final same-head paired validation at e48986b03 was highly variable:

SeedOrderBeforeAfter
750before then after8.957s7.902s
751after then before14.336s13.101s
752before then after9.733s7.649s

The 13.101s updated run fails the budget. Another direct binary pair for seed751 was 8.704→6.873s. A subsequent fixed-CPU diagnostic did not produce a valid comparison because its output parser failed after the baseline; it is excluded from the evidence. CPU time was close to wall time; no specific cause for the large variation is established. These results do not prove reliable sub-10s headroom, so the PR is closed rather than marked ready.

Validation

All 16 functional tests pass: the original 12 summary tests plus four directory/model tests. The directory suite took .080s, its two assertion-deletion copies .078s each, existing summary controls .260s, original full regression1.097s, and metadata/invariant tests .015s. All six new fault observations reach their required assertions and complete after deletion of the targeted assertion. The runtime gate remains failed as described above.

Windows lb.ps1 checks pass for both touched crates: Filesystem --lib --tests in1m15s plus270s queue; final Harness --lib --tests in8.01s plus105s queue. The Filesystem source is unchanged between the candidate's two commits. Rustfmt, LF, caller search and git diff --check pass. No diagnostic modules, timing assertions, preemption caps or reduced case counts are committed.

bb1 profile log: /home/var/log/buildbox/2026-10-16T203131122Z-738849.log. Final validation log: /home/var/log/buildbox/2026-10-16T205647692Z-777939.log. Per-case profiles and paired reports are retained in Q:/lanes and the worktree target directory. The parent execution/compaction pipeline remains the performance finding to pursue separately.


Summary by cubic

Reuses a single Tokio runtime across all 32 generated summary-history property cases, drops the redundant proof load and source-context clone from the shared fixture, and adds targeted property coverage for recursive directory creation.

  • The generated property previously built a new current-thread runtime per case; it now creates one runtime and still runs 32 cases with default shrinking, file persistence, and the same test metadata.
  • SummaryFixture no longer keeps an unused source context, pushing that clone and the extra proof assert into published_summary, where their results are asserted.
  • create_dir_all verifies the full target with a non-following lookup before walking ancestors, skipping the walk for existing directories and rejecting non-directory targets; empty directory transactions are modeled as unchanged.
  • The new directory_creation_properties.rs covers existing-node preservation, obstacle rejection via files and symlinks, five negative controls, and observing each replaced ancestor across a rebase.
  • Paired before/after timings did not show a consistent speedup, so treat the runtime reuse as redundant-work removal rather than a measured acceleration.