#821 Batch filesystem staging and reuse request encoding

Summary publication repeatedly prepares identical model requests, serializes source values for separate sizing/accounting passes, and resolves staging paths in separate transactions. Reuse the prepared request, serialize each bounded source value once, batch staging, and retain the memory host's immutable workspace handle. All 32 generated independent parent histories and shrinking remain.

Review r0001 fixes: current head 356e9ba658f502f7a7a7174fc37d6681d4a02cdb

All three findings are fixed in signed commits 675ceb3ab and 356e9ba65.

• Generic native batching helpers and their transitive private-candidate/replica/proof chain now use the native caller's cfg, without requiring native-watch or cfg(test). Workspace binding likewise uses the native memory caller's cfg, independent of filesystem-local and native-execution; custody checks remain.
• scripts/check-storage-feature-sets.sh compiles production libraries in three separate Cargo invocations: Filesystem memory, Harness filesystem, and Harness filesystem-local, each with --locked --lib --no-default-features. CI runs this script in the existing aarch64 job. All three passed on bb1 in 207.477s total. Test/dev feature unification cannot hide these gates.
• Replace all prior-path scans with one funded borrowed-path index, using the existing accounted adaptive/heapsort. Keep original ordinals and write order. Validate kinds in input order before sorting; equal-path groups preserve directory/file conflict and duplicate semantics. General unsorted inputs are O(N log N); sorted inputs are linear. The old batch_contains path is deleted.

The frozen old classifier initializes separate paths and kinds through the shared differential runner. Thirty-two shrinking semantic cases include mandatory accepted duplicates and both error classes. Another 32 shrinking cases check N and 2N exactly. Static cases compare the whole work receipt and all six physical allocation fields, using ==:

Paths	Inspections	Path comparisons	Index allocations	Index bytes
1,024	3,070	2,046	1	16,384
2,048	6,142	4,094	1	32,768

Inspections are exactly 3*N-2: N kind checks and two adjacent-pair passes. The negative control restores the old N*(N-1)/2 scans after classification, without changing semantics or allocating. It fails the same exact terminal guard at both sizes; deleting that guard permits both to complete (2/2 detected; 2/2 deletion completions). The staging controls and their deletion copies also pass. Normal tests never rewrite expectations.

Accounting fixture changes are explicit: healthy staging adds one 112-byte scratch allocation (7 entries); the binding fixture adds two scratch allocations totaling 256 bytes. Healthy staging inspections are 993 instead of 974 and bytes copied 23,478 instead of 22,822; rejected staging inspections fall from 231 to 223 with unchanged allocations. Other full work/allocation fields remain exact. Independent full bytes, errors, generation and ordered publication traces pass. No charge, resource limit, tolerance, case count or shrinking is weakened.

Focused suite	Passed	Seconds
Index semantic/scaling/exact/control/measurement	5	0.122
Index assertion-deletion copy	5	0.120
Staging differential/controls/deletion	6	0.190
Batch properties	4	0.193
Transactions	7	0.147
Kernel batch/cached helpers	5	0.013
Shared differential runner	11	0.004
Binding differential/controls	4	0.161
Namespace binding	2	0.113
Request preparation/controls	5	0.014
JSON exact guards + generated differential	3	0.016
Single-pass properties/controls	4	0.008
Existing JSON	18	1.027
Replay / context / ownership / compaction	30	0.072
Journal observations/controls	8	1.840
Pinned reads	2	0.340

One canonical three-crate native test build passed in 439.077s. Its Rust source is 675ceb3ab; final 356e9ba65 changes only two runtime JSON expectation files, verified by the retained binaries. The final Windows GNU cross-check of all three crates passed in 52.622s. The workflow runner gate passed. No local Cargo builds were used.

Eight alternating paired publication runs compare independent retained 6ac9a183d and final-candidate binaries/processes, with all three tests, all 32 independent histories and shrinking retained:

Seed	Before 6ac (s)	After 356 (s)
750	4.121	4.029
751	3.920	4.140
752	3.946	4.058
753	4.041	4.068
754	4.046	4.070
755	4.041	4.348
756	4.019	4.128
757	3.967	4.104

Median 4.030 -> 4.087s; maximum 4.121 -> 4.348s. All are under five seconds in this block; this delta establishes the classifier's exact scaling, not an aggregate speed-up. Earlier noisy blocks below remain part of the evidence. This review comparison uses the pre-review candidate as baseline, not origin/main.

Exact gates and paired rerun command (retained independent binaries, no rebuild):

Q:/lanes/bs.ps1 bash scripts/check-storage-feature-sets.sh
Q:/lanes/bs.ps1 cargo +nightly-2026-07-01 test --locked -p acyclic-harness -p acyclic-fs -p acyclic-test-support --no-default-features --lib --no-run --features acyclic-harness/terminal,acyclic-harness/filesystem-local,acyclic-harness/native-execution,acyclic-harness/grpc,acyclic-harness/test-support,acyclic-harness/search-http --message-format=json-render-diagnostics
Q:/lanes/bs.ps1 cargo check --locked -p acyclic-harness -p acyclic-fs -p acyclic-test-support --lib --tests --no-default-features --features acyclic-harness/terminal,acyclic-harness/filesystem-local,acyclic-harness/native-execution,acyclic-harness/grpc,acyclic-harness/test-support,acyclic-harness/search-http --target x86_64-pc-windows-gnu
$paired = @'
import json, os, re, subprocess, time
paths = {"before": "/work/target/gc-verify-821-final-harness", "after": "/work/target/gc-verify-821-review1-acyclic_harness"}
for seed in range(750, 758):
    row = {"seed": seed}
    for arm in (["before", "after"] if seed % 2 == 0 else ["after", "before"]):
        start = time.monotonic()
        p = subprocess.run([paths[arm], "filesystem::memory::summary_tests::publication::", "--test-threads=1"], env=dict(os.environ, PROPTEST_RNG_SEED=str(seed)), stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, check=True)
        assert re.search(r"test result: ok[.] 3 passed;", p.stdout), p.stdout
        row[arm] = time.monotonic()-start
    print(json.dumps(row), flush=True)
'@
Q:/lanes/bs.ps1 python3 -u -c $paired

Artifacts: target/gc-verify-821-review1-final.json (completed:true); lane logs gc-verify-821-review1.log, gc-verify-821-review1-finish.log, and gc-verify-821-review1-finish-resumed.log. The driver initially expected three kernel tests; five passed, including two cached submodule tests. The corrected count resumed direct-binary selection without rebuilding or repeating prior suites.

macOS retry at this head was refused by the pre-Cargo capacity guard: free space fell from 10.4 GiB at inspection to 8.6 GiB at launch (9,218,371,584 bytes in the report). No compile or test ran; no Mac pass is claimed. Only the existing ~/mac-src/gc-verify and CARGO_TARGET_DIR=~/mac-target/gc-verify exist, with no old lane directories or per-commit sources/tarfiles to clean. No own Mac job is running. Resume only after capacity recovers above the user threshold.

Decisions

• Reuse prepared request bytes for accounting and dispatch. Recovered projections still validate at storage entry. Existing ownership and native-custody checks remain.
• Fund old/new JSON storage before growth and admit actual digest chunks; canonical identities and Value-only one-allocation/linear-work guarantees remain.
• Resolve paths once per eligible staging batch and transfer path ownership. Existing files still truncate; duplicate writes retain order. Pending/folded namespaces use the funded serial candidate and adopt only on success. Raw metadata invalidation/removal uses the ordinary path.
• Workspace binding caches only the immutable handle. Live heads, exact-generation reads, authority and foreign-reference fallback remain dynamic.
• Integrate main 2eb8a99a3 (including #934) with signed merge 0c4316bb4. Lane rules prohibit rebasing pushed branches.
• Main integration changes allocation layouts. Lower all six binding expectations; preserve exact journal-frame count/byte observations with checked-in byte expectations; update the native pinned-read exact byte delta from 1840 to 1824. Outputs, errors, ordered traces, receipts and count assertions remain. No resource limit or tolerance changed.
• The requested aggregate below five seconds is not established. This PR is marked ready at the coordinator's request to unblock st-codec(c). Current absolute timings below do not establish a fresh speed-up over current main.

Verification

Use the shared acyclic-test-support::differential runner. Every old/candidate arm initializes independent state. Frozen original factories, codecs and workspace constructor come from 3d2f72e2a; no candidate decoder or journal state is shared with the reference.

Compare full output bytes, exact errors and ordered operation traces; require exact whole work receipts and all six physical allocation fields. Original native accounting/custody remains checked. Negative controls add back actual redundant encoding, serial staging, request factories or workspace opens. Every control fails its terminal observation, and deleting that observation permits completion. Normal tests never rewrite expected data.

Previous integrated head 6ac9a183d9b62bc43f4a0052adb2163a2808f906 (historical)

Suite	Passed	Seconds
JSON exact work/allocation guards	2	0.009
Request differential/controls	5	0.014
Workspace binding differential/control	4	0.193
Existing JSON	18	1.943
Journal observations/controls	8	2.510
Pinned reads and controls	2	0.418

Unchanged Filesystem staging controls/deletion (6), generated batch properties (4), transaction tests (7), shared differential tests (11), and Harness assertion-deletion copies passed in the earlier integrated build; their source is unchanged. Replay (12), context (4), request ownership (1) and compaction (13) also passed.

Full three-test publication filter, serial execution, all 32 generated histories with shrinking, seeds 750–757: median 7.494s, maximum 9.360s. Runs: 6.941s, 6.058s, 5.318s, 5.594s, 8.047s, 9.339s, 8.465s, 9.360s. These are absolute current-head timings.

Windows GNU cross-check for Harness, Filesystem and test support: passed, 335.48s, on bb1. This compiles Windows paths; it is not Windows runtime/performance validation.

Q:/lanes/bs.ps1 cargo +nightly-2026-07-01 test --locked -p acyclic-harness --no-default-features --lib --no-run --features acyclic-harness/terminal,acyclic-harness/filesystem-local,acyclic-harness/native-execution,acyclic-harness/grpc,acyclic-harness/test-support,acyclic-harness/search-http --message-format=json-render-diagnostics
Q:/lanes/bs.ps1 cargo check --locked -p acyclic-harness -p acyclic-fs -p acyclic-test-support --lib --tests --no-default-features --features acyclic-harness/terminal,acyclic-harness/filesystem-local,acyclic-harness/native-execution,acyclic-harness/grpc,acyclic-harness/test-support,acyclic-harness/search-http --target x86_64-pc-windows-gnu

Focused tests and timing reruns use retained binaries directly. bb1 log: Q:/lanes/gc-verify-821-finish.log. macOS native compilation was attempted at this head but interrupted before tests when ivar fell below the user disk threshold (5.9GB free). No Mac test/performance result is claimed. The single lane target is preserved at ~/mac-target/gc-verify; no old lane data exists to clear. Resume after capacity is restored; see the Mac follow-up comment.

Historical paired measurements

Independent baseline origin/main 3d2f72e2a plus the identical summary-path compatibility fix; separate builds and per-process histories, all three tests/all 32 cases. Original eight pairs: median 6.684→5.074s, maximum 12.720→8.444s, 7/8 wins.

After integration with independent main 142c97f68, paired results were noisy and did not establish the requested gain:

Seed	Main (s)	Candidate (s)
750	6.752	5.696
751	6.447	4.966
752	6.755	11.305
753	7.112	9.295
754	7.042	5.157
755	8.585	5.514
756	10.228	9.896
757	12.424	9.072

Median 7.077→7.384s; maximum 12.424→11.305s, 6/8 wins. Another block measured median 7.038→5.343s and maximum 11.390→9.715s. Neither erases the slower block. Main's unrelated improvements receive no credit here.

Paired command for the independently built historical baseline/candidate (retained binaries; these paths do not denote current head):

$paired = @'
import json, os, re, subprocess, time
paths = {
    "before": "/work/target/gc-verify-summary-main-142c97-before",
    "after": "/work/target/gc-verify-summary-perf-rule-main-after",
}
for seed in range(750, 758):
    row = {"seed": seed}
    for arm in (["before", "after"] if seed % 2 == 0 else ["after", "before"]):
        start = time.monotonic()
        result = subprocess.run(
            [paths[arm], "filesystem::memory::summary_tests::publication::", "--test-threads=1"],
            env=dict(os.environ, PROPTEST_RNG_SEED=str(seed)),
            stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, check=True,
        )
        assert re.search(r"test result: ok[.] 3 passed;", result.stdout), result.stdout
        row[arm] = time.monotonic() - start
    print(json.dumps(row), flush=True)
'@
Q:/lanes/bs.ps1 python3 -u -c $paired

Additional experiments were removed when paired runs did not establish gains: plain-string lexing (5/8 wins, 6.721→7.068s median), canonical-Value re-admission (3/8, 5.343→5.723s), dependency capture (3/8, 4.816→4.913s), repeated-proof/context fixture (3/8, 8.236→9.838s; duplicate-baseline noise median 8.124s). No experimental source remains. Historical full tables, profiles and logs are preserved in lane artifacts. No test case count, shrinking, invariant or charge was weakened.