#867 Resolve directory prefixes in one authenticated walk

What

Transaction::create_dir_all (and restore, which reaches the same create_dirs entry) now resolves the whole requested path in one authenticated walk. The walk keeps the file-record and directory-name observations for every ancestor (CaptureRecords::Prefixes). It then creates only the missing suffix, and the final creation reuses the path that was already parsed. Before this change, each prefix of a depth-N path ran its own no-follow lookup from the root, which costs O(N²) page reads and component copies.

• Windows-profile, case-folded and single-component paths keep the original per-prefix loop (create_dirs_individually). The Portable, POSIX and Browser profiles take the single walk.
• When a mutation is already pending, the walk does what the separate lookups did: the candidate lookup runs on the transaction root and observations come from the base generation. A non-directory prefix stops the walk before it descends, so NotDirectory is returned with the same observations.
• Missing directories are created in the same order with the same generation identities. No public API, parser or synchronization change.

Class

H, repeated work. Every create_dir_all / restore caller goes through create_dirs, so there is one place to fix. Exact work and allocation pins at that entry fail if the per-prefix walk comes back (see the restore control below).

Verification (Perf rule)

rust/crates/filesystem/src/tests/workspace/directories.rs uses the shared acyclic_test_support::differential::pair helper:

• Independent baseline: the original per-prefix loop runs on a separately created memory filesystem with fixed operation IDs. It never shares the candidate's checkout, cache or journal.
• Complete trace equality: both sides must match on full results and errors, all checkout observations, and a later concurrent commit. They must also match on the commit outcome, generation IDs, stat of every prefix, and the bytes of untouched and later files.
• Cases covered:
  • depths 0, 1, 4 and 8, with existing, missing and partial prefixes;
  • file and symlink collisions, and concurrent ancestor modification;
  • maximum_paths = 1, and invalid spellings;
  • a mutation pending before the call (new): a pending write, and a pending removal of a non-directory collision that the base still binds;
  • all four filesystem profiles, native custody, and cancellation with no work dispatched;
  • a 24-case shrinking proptest over depth, existing, collision, concurrency, pending and batch size.
• Exact pins: directories/work.json pins every work counter (==) and six physical-allocation fields for four POSIX workloads. Page reads and parsed components are also checked against counts derived independently of the fixture (Σ(1+2k) vs 1+2N).
• Negative controls:
  • restored_prefix_walks_fail_the_same_work_and_allocation_assertions measures the original traversal against the pinned candidate values, for all four workloads, and must fail both the work and allocation assertions.
  • missing_prefix_observation_fails_the_same_trace_comparison keeps only the terminal observation and must fail the complete-trace assertion.

POSIX fixture

The fixture was earlier measured on the Portable profile. Native Linux volumes use POSIX. Only itemsExamined changed (existing4 old 160→162, existing8 509→518 / 142→145, missing4 597→595 / 502→501, partial4 469→455 / 370→359), on both sides. Pages, components, bytes and allocations are identical. The cause is that authored FileIds hash the component name-encoding tag (Utf8 = 1 vs PosixBytes = 2). The file-table key order therefore changes, and the binary search compares a different number of keys.

POSIX case	Pages	Parsed components	Heap allocations	Allocated bytes
existing depth 4	24 → 9	10 → 4	122 → 68	71,466 → 42,348
existing depth 8	80 → 17	36 → 8	306 → 97	134,224 → 49,956
missing depth 4	66 → 42	10 → 10	768 → 732	278,323 → 235,369
partial depth 4	60 → 38	10 → 7	504 → 449	177,931 → 142,078

Some peak/current allocation fields do not decrease. The fixture records them as measured.

Paired call phase, bb1, 10 runs, µs median [min, max], original → candidate:

• existing8: 354.7 [305.0, 430.0] → 158.3 [136.5, 183.4]
• existing4: 212.1 [162.2, 256.5] → 114.8 [93.0, 133.2]
• partial4: 453.5 [394.7, 722.2] → 427.2 [350.5, 452.8]
• missing4: 654.7 [579.3, 965.3] → 651.2 [496.2, 888.4]

Paired measurement command (reruns the built binary):

GC_LOCAL_DIRECTORY_TIMINGS=1 target/debug/deps/acyclic_fs-<hash> workspace::tests::directories::directory_work_and_physical_allocations_are_exact --exact --nocapture

Full LocalHarness matrix vs clean main

local::admission::poll_sweep::run_cancellation_at_every_poll_recovers_head_and_admission was compared against clean main 98bbbaed8, which differs from current main only in harness/src/wasm. Both binaries were built with cargo test --locked -p acyclic-harness --lib --features filesystem-local,native-execution,test-support,terminal,tui,mcp-http,model-checking. Each side carried the same measurement-only patch, which was never committed. With GC_LOCAL_CUT_SET=1 the patch prints each scenario's enumerated cut sequence, and it reports the clean-main coverage drift described below instead of failing on it. Every cut still runs.

• Cut sets: clean main and the candidate enumerate identical scenario/index/name sequences, Text 352 + EffectAndContinuation 856 + Compaction 684 = 1,892. All three per-cut recovery, repeated-resume, reopen and audit phases ran for each cut on both sides.
• Six alternating pairs (wall s / CPU s):

Run	clean main	candidate
0	28.7 / 578.7	14.0 / 535.3
1	22.0 / 560.3	16.2 / 550.3
2	63.4 / 656.0	74.6 / 648.3
3	44.3 / 606.5	91.6 / 672.5
4	46.2 / 654.4	17.7 / 536.7
5	9.9 / 502.8	9.8 / 497.8
median	36.5 / 592.6	17.0 / 543.5

bb1 was heavily oversubscribed during these runs, so wall times ranged from 9.8 s to 91.6 s on the same binary. These runs do not establish a full-matrix wall-time gain. The quietest pair (run 5) differs by about 1%. Median CPU time is lower for the candidate, but the spread overlaps. In the matrix, directory creation happens only on shallow /.system/... paths. The proven benefit is the exact per-call work above, which is O(N) instead of O(N²) pages for deep paths.

Matrix command (reruns the built binary on each side):

GC_LOCAL_CUT_SET=1 target/debug/deps/acyclic_harness-<hash> local::admission::poll_sweep::run_cancellation_at_every_poll_recovers_head_and_admission --exact --nocapture

Validation (bb1, head 0689760)

Built once: cargo test --locked -p acyclic-fs -p acyclic-test-support --lib --features model-checking --no-run. Tests were run directly from that binary:

Suite	Result	Process time
workspace::tests::directories	9 passed	0.72 s
workspace::tests::sibling_directories	2 passed	0.04 s
workspace::tests::transactions	6 passed	0.07 s
workspace::tests::restore	5 passed	0.90 s
kernel::path_access::tests	27 passed	0.78 s
git_compat::	48 passed	0.36 s
lazy_workspace::	32 passed, 1 skipped (pre-existing failure, below)	0.42 s
multi_root::	passed	0.02 s
test-support	passed	0.03 s

Windows GNU compile-only proof (cargo check --locked -p acyclic-fs --lib --tests --features model-checking --target x86_64-pc-windows-gnu) passed in 2 m 14 s.

Harness local::admission::poll_sweep (13 tests): 10 passed. The 3 failures (indeterminate_admission_…, coverage::omitted_actual_hook_…, compaction_summary_is_readable_…) are the same clean-main coverage drift listed below, with identical counts on both sides.

Pre-existing failures on clean main (not caused by this PR)

• lazy_workspace::tests::mount::source_parent_promotion_reads_only_the_record fails the same way on clean main 98bbbaed8 (left: 0, right: 2, published case). It is deterministic, also fails single-threaded, and does not touch any path changed here.

• local::admission::poll_sweep::run_cancellation_at_every_poll_recovers_head_and_admission is red on clean main on bb1:

  • effect_goal.json still pins 870 non-ready boundaries, but the sweep records 856 (#806 removed 14 boundaries without updating the file);
  • stream:read:ready drifted: Text 2718→2821, Effect 7124→7367, Compaction 4758→4868.

  The candidate produces identical counts. Neither count was changed here. Both need attribution in their own follow-up.

Decisions

• Windows-profile and case-folded volumes keep the per-prefix loop. Folded-name canonicalisation retries per lookup and is not covered by the single walk. This leaves the Windows matrix unchanged.
• The old loop stays only as that fallback and as the independent baseline. Net production deletion is roughly zero: the change removes repeated work, not code.

🤖 Generated with Claude Code

Review fixes (1b53018c3)

1. [P1] A failed walk keeps completed-prefix observations. Each prefix group (its directory name, plus its record when bound) is checked against the dependency limit in order. The groups that fit are admitted, and the first rejected group raises the same error the per-prefix loop did. For example, with existing /a/b and a limit of 3, /a's two observations are retained.
2. [P2] Base observations are bounded by the candidate's stopping prefix. Only prefixes up to the first non-directory, or the first missing prefix, are admitted. A pending file replacing a base directory now reports NotDirectory at a two-dependency limit.
3. [P2] Windows and folded volumes use the shared walk. The fallback loop is deleted, and the tests keep it as the differential oracle.
   • An existing depth-N path is walked once on every profile.
   • On folded volumes, the case-folded retry from the first missing prefix onward still uses the separate lookup, because that retry observes the parent's listing.
   • Base directories that a pending removal left below the first missing prefix are also looked up separately, preserving the per-prefix interleaving of observations and creations.

New coverage

• Dependency-limit sweep: the limit is swept across every boundary of eight shapes, both sensitive and folded. Shapes include a pending file over a base directory and pending removals of base directories.
• Proptest: now also covers the Windows and Portable profiles, folded volumes, mis-cased requests and dependency limits.
• Profile test: 4 profiles × sensitive/folded × 7 shapes, including mis-cased requests.
• Negative control: atomic full-path admission fails the same trace comparison.
• New exact work/allocation pins (windows4, windows8, folded4, folded8), with analytic page reads and parsed components equal to POSIX. The POSIX pins are unchanged.

Workload	Page reads, separate lookups → shared walk	Items examined	Allocation count
windows4 / folded4	24 → 9	155 → 78	122 → 68
windows8 / folded8	80 → 17	500 → 143	306 → 97

Validation (bb1)

Suite	Result
workspace::tests::directories	13 passed, 0.78 s
sibling_directories	2 passed
transactions	6 passed
restore	5 passed
path_access	33 passed
rebase::	14 passed

• Build: 676 s.
• Windows GNU compile proof: exit 0, 61.6 s.
• Paired command: acyclic_fs-<hash> workspace::tests::directories::directory_work_and_physical_allocations_are_exact --nocapture. The restore control is restored_prefix_walks_fail_the_same_work_and_allocation_assertions.

🤖 Generated with Claude Code