PR #961 - acyclic-labs/sdk

Native staged publication issued a provider put and fsync per object even though each drain already selected a bounded batch. Native and ordinary drains now use one put_many call. Failed batches stay staged until the provider acknowledges the complete batch, preserving retry of a partial durable prefix.

Duplicate detection retains the first body in a private identity-keyed fixed slot index. Every occupied or empty slot probe is charged inside the private index before access, including collisions and duplicate resolution. The index replaces quadratic duplicate scans without hiding collision work. Its single backing array is admitted using its exact checked Layout before try_reserve_exact; insertion cannot grow it. Storage is freed before allocation custody is released. Opaque HashMap capacity estimates and later reconciliation are removed.

For 57 entries, the old constructor claims 5,618 bytes, physically allocates 6,288 on Linux, then reconciles to 11,008. The replacement allocates and charges exactly 6,144 bytes: a 6,000-byte allowance rejects before allocation, and 7,000 bytes admits the affordable table.

Verification

At b3e87c826, both matrix gaps OBJ-probe and OBJ-wasm are closed (R961-085344.1/.2). The retained physical-allocation properties also cover R961-014542.1.

Linux: 72 focused tests passed. Logical objects 12/0.063 s; staged publication 17/4.024 s; distributed fixtures 8/0.021 s; merge 15/0.024 s; multi-root 14/0.079 s; workspace publication 6/1.684 s. Build 549.975 s at 50b7e992e; the final four target-gate edits leave the compiled native paths identical, so these tests reuse that binary (SHA-256 89bd9771cb5935ab471e69294644b8849a2d9cc71ba27af537f00694cf634c7b).
Wasm memory library-test compilation passed in 225.078 s: cargo test --locked -p acyclic-fs --lib --no-run --no-default-features --features memory --target wasm32-unknown-unknown.
Windows GNU library/tests cross-check passed at b3e87c826 in 20.513 s.

At filesystem 1f4e63304 plus merged native-runtime fc3a577d7, macOS debug compilation took 482.554 s; logical-object tests passed in 1.473 s and staged-object tests in 2.955 s. The run used F_FULLFSYNC, then deleted its single lane target and source copy; 48 GiB remained free. The later target-closure changes adjust portable fixtures and module locations.

At 6634a5e7dd: Linux 27/27 passed in 1.40 s (release build 16m 24s); Darwin 27/27 passed in 2.99 s (release build 16m 02s); Windows GNU --lib --tests cross-check passed in 3m 00s. Seven paired native-publication reruns passed on each OS. Windows native release retry built in 14m 10s. The first Windows attempt hit an internal LLVM LTO panic; the retry quarantined only this worktree's filesystem incremental cache, preserving dependencies and source.

Shrinking quota properties generate counts 1–128 at B−1, B and B+extra in standalone and native accounting. Admission requires exactly one allocation of B bytes; denial requires zero allocations. Charged peak must match backing storage and release must restore live custody. Native fixture setup initializes the test lock-order checker before measuring index storage.
An independently initialized copy of the old constructor fails the same exact allocation/admission predicate. The explicit 57-entry test includes both 6,000- and 7,000-byte allowances.
A separate property compares the complete first-value trace with an independent HashMap and requires zero insertion allocations.
N/2N properties pin exact probe counts against an independent occupancy-only oracle and exactly 3 + 3N allocations. The restored quadratic-scan control adds exactly N(N-1)/2 examinations and fails the same predicate. Complete readback is checked.
Forced-collision properties check exact triangular counts, arbitrary quota cuts and below/at/above boundaries, typed denial before mutation, duplicate first-value traces and retry after denial. A private mutation of the real loop restores the old one-charge-per-insert bug and fails the same predicate. Cancellation denies before any probe or mutation. These proofs remain compiled on Wasm; only allocation measurement is gated to its native-only dependency.
Independently initialized individual-write, direct-batch and staged providers compare complete readback bytes, exact cancellation errors and logical write/read traces, including durable reopen and partial-batch retry.
Absolute flush pins: staged batch == 1, direct batch == 1, individual == N. An extra real metered flush produces exactly 2, preserves output/traces, and fails the same batch-work predicate.
Exact staging pins remain direct-batch work plus four allocations and 3N examined objects. Codec work is pinned relative to physical journal output bytes; object bytes and backend writes match exactly.

Measurements

Seven paired runs of each already-built release binary, 32-byte objects. Each run checks full outputs, errors, observer traces and absolute counters. Initialization/readback are outside the timed section. Medians:

OS	objects	individual writes	staged batch	flushes	backend writes	native allocations
linux	4	5.398 ms	0.917 ms	4 -> 1	8 -> 2	113 -> 101
linux	8	9.488 ms	0.846 ms	8 -> 1	16 -> 2	225 -> 179
windows	4	3.425 ms	1.024 ms	4 -> 1	12 -> 3	113 -> 101
windows	8	5.634 ms	0.833 ms	8 -> 1	24 -> 3	225 -> 179
darwin	4	13.887 ms	4.372 ms	4 -> 1	8 -> 2	113 -> 101
darwin	8	29.257 ms	4.257 ms	8 -> 1	16 -> 2	225 -> 179

Staged ranges: linux: N=4 0.578–2.922 ms, N=8 0.717–3.713 ms; windows: N=4 0.862–1.434 ms, N=8 0.821–1.165 ms; darwin: N=4 3.486–5.464 ms, N=8 3.334–7.804 ms.

Paired measurement command, after the focused release test build:

for run in 1 2 3 4 5 6 7
do
  "$binary" native_publication_batches_exact_work_with_individual_fsync_negative_control --nocapture --test-threads 1
done

Linux builds/tests use bs.ps1; Windows native I/O runs use lb.ps1; Darwin runs use ssh ivar with CARGO_TARGET_DIR=~/mac-target/co-stream-c. Subsequent runs invoke the built binary directly.

Guarantee

Class: F (unfunded work/allocation), H (redundant work), J (target capability closure), I (blind accounting oracle). One provider-batch choke point removes divergent native/ordinary dispatch. The private fixed-layout index admits exact storage before allocation, cannot grow on insertion, and admits every probe before slot access. Only metered insertion is visible to production callers. Allocation instrumentation is confined to a module gated identically to its dependency.
Proof: standalone/native exact-layout quota properties with an independent former-constructor control; portable collision and duplicate/denial/retry properties over the real loop with reverted-accounting control; independent occupancy oracle and restored quadratic-scan control; zero insertion-allocation and full first-value differential; independent individual/direct/staged providers comparing full output, errors, observer traces, reopen and partial retry, plus an actual extra flush that fails the same exact work predicate. Native, GNU and Wasm library-test compile proofs cover the dependency boundary. The review's original unresolved native-only import and the two recorded closure failures are compiler negative controls for mismatched target gates; portable properties remain included. Each focused suite stays under 10 s. No new parser or concurrency primitive is introduced.
Entry points: LogicalObjectStore::put_many with ordinary or original native custody, and ::put_many_hashed, converge on publish_batch and the same private Index::insert; the hashed-native unsupported boundary is preserved and tested. Both ordinary and native StagedObjects drains use the same provider batch operation. Cancellation, partial publication failure and durable retry are covered by the staged suite.
Gaps: none.

Decisions

Align provider fixture futures with the target-specific StreamProvider contract, require the resolver's existing Send + Sync boundary explicitly, and use the portable empty-resolution representation. The oversized multi-root module is split mechanically into plain modules; publication behavior and public paths stay the same.
Retain every attempted member after a failed batch because put_many exposes no partial acknowledgement. Immutable retry verifies or deduplicates the durable prefix.
Independent test providers receive identical immutable hash keys, with separate tables, providers, ledgers and journals, so exact probe receipts are comparable. Shipping builds retain RandomState::new per index.
Keep the first body in the identity index to preserve mismatch detection without a second scan. Checked 7/8-load geometry guarantees an empty terminating slot; all backing storage is funded before allocation.
Preserve existing limits, denial checks and allocation charges.
Real journal timestamps can change protobuf varint width. Exact codec pins subtract their physical output contribution without clock coupling or slack.

Live steps

The earlier release macOS measurements use filesystem 6634a5e7dd plus merged #966 native-runtime fc3a577d7. The shared helper is acyclic_native_runtime::sync_file(file, Durability, meter). Full requires F_FULLFSYNC; ENOTSUP-only weaker fallbacks return Unsupported for Full. Its production syscall seam, fsync negative control and EINTR/admission test passed in #966; its review thread is resolved.
Earlier clippy runs stopped on baseline lint debt (41 dependency diagnostics in acyclic-auth; 270 filesystem library / 479 lib-test diagnostics). No lint suppressions or limits were added.