Prevent parked lock acquisitions in local and memory streams
Change
A stopped follow/children consumer can strand a read reservation handed off by Tokio's fair RwLock, deadlocking a later append. Route both LocalStream and MemoryStream through one finite-snapshot driver. A ready snapshot completes on the stack. A pending acquisition is dropped in that same poll, and the owned snapshot state moves to an independent worker; the consumer retains only its completion receiver. Neither LocalStream nor MemoryStream stores a pending lock acquisition in its returned streams. Worker ownership is recoverable: executor shutdown drops the lock future, then returns the cursor state and wakes the consumer. Its next poll retries on whichever executor is driving it. There is no channel-close panic.
The Loom shim now separates pending FIFO waiters from granted ownership reservations. Compatible readers can enter alongside an unpolled granted reader; queued writers retain priority. Cancellation releases either kind of reservation and wakes successors. Shared admission/cancellation cases run against real Tokio and the shim, with an old-shim negative control.
Apply the supplied CLI readiness patch: wait for process exit instead of a six-second deadline.
Decisions
- Use one shared
snapshot::readfor local and memory records/children. It accepts owned snapshot state rather than a boxed pending future, polls once on the stack, cancels before returning Pending, and allocates only when independent execution is necessary. Cursors change only after all locks have been acquired; every guard is released before completion is sent. - Keep cursor ownership in a job whose destructor returns unfinished state through an executor-independent completion slot. The completion mutex is a documented leaf lock; work, wakeups and state destruction happen outside it. A dropped consumer releases the returned state, while a live consumer retries without advancing its cursor twice. This covers workers dropped before their first poll as well as queued/granted acquisitions. The ready path never constructs this job or allocates.
- Keep read observation outside the cancellable probe: observation may pause after cursor advancement. Preserve existing local read accounting and worker context. Use the active Tokio executor, a thread for executor-independent MemoryStream use, and
spawn_localon WASM. - Separate local registration from stream construction and return observation as a future directly, reducing nested future stack use without increasing Loom's stack limit.
- Model the real local and memory provider paths through the crate-local synchronization shim. The dedicated parked-acquisition models are unbounded and include old-code deadlock negative controls. Bound the broader registration/failure-order races to one preemption, as requested; their missing-wakeup/lock-order negative controls still fail. Compaction and cancellation coverage remains unbounded.
- Compare independently initialized old/candidate providers with full record bytes and metadata, exact errors, and complete polling traces through the shared differential harness. The old local path boxes each record; the old memory path allocates zero. Assert candidate allocation count == 0 and bytes == 0 for 100 ready records or children. Reintroducing per-record boxing must fail these same assertions.
Validation
Final head: 988805843a8313ea34275ecd5b55e2f790fdcea4. All 33 changed file hashes match on bb1 and ivar.
- Loom: 31/31, including all negative controls, 0.773 s total; registration 0.008 s. Each model also has an external 10-second timeout.
- The public-API review probe is a regression test, extended to follow, checked follow and children; each resumes on a separate executor after shutting down its polling runtime, including runs that yield to the worker. Loom captures and drops the actual production worker future, racing shutdown, writer release and consumer cancellation. A mutant that discards the cursor on worker drop must deadlock.
- Linux: all 24 focused Stream native tests passed (1.01 s total), including the shutdown regression (0.016 s wall), ready allocations, memory parking, cancellation, compaction and wakeups.
- CLI: all four other modes passed.
dumb_terminal: 100/100, zero hangs; total 66.142 s, min 0.565 s, mean 0.661 s, median 0.666 s, p95 0.767 s, max 0.937 s, serial runs of the same built binary on bb1. - Windows GNU compile proof passed for Stream and GraphCoder (
--lib --tests, 17.02 s). - WASM compile check passed (2.26 s).
- Windows x64 runtime: all five focused tests passed, including the public-API executor-shutdown regression (0.00 s). Allocation/differential test 0.06 s; other tests <0.01 s each.
- macOS arm64 runtime: all five focused tests passed, including the public-API executor-shutdown regression (0.00 s). Allocation/differential test 0.14 s; other tests <0.01 s each.
Ready-path timing: median microseconds for 100 records/children, 31 paired samples, debug test profile. Each candidate path allocates 0 times / 0 bytes on every OS. Local baseline allocates 100 times; memory baseline allocates zero. These measure ready polling, not disk throughput.
| OS | Memory follow old → new | Local follow old → new | Memory children old → new | Local children old → new |
|---|---|---|---|---|
| Linux bb1 | 19.029 → 15.163 | 49.926 → 34.232 | 42.635 → 41.162 | 63.756 → 52.730 |
| Windows x64 | 11.100 → 9.600 | 46.900 → 26.600 | 32.700 → 31.100 | 61.400 → 42.400 |
| macOS arm64 (ivar) | 18.291 → 14.000 | 20.333 → 14.375 | 30.875 → 28.125 | 24.083 → 19.458 |
Linux commands through Q:\lanes\bs.ps1 (one build per target, then exact built-binary runs):
RUSTFLAGS="--cfg acyclic_loom" cargo test --locked -p acyclic-stream --features local,http,test-support,serde --test local_follow_loom --no-run
cargo test --locked -p acyclic-stream -p graphcoder --features acyclic-stream/local,acyclic-stream/http,acyclic-stream/test-support,acyclic-stream/serde --lib --test cli --no-run
cargo check --locked -p acyclic-stream -p graphcoder --lib --tests --features acyclic-stream/local,acyclic-stream/http,acyclic-stream/test-support,acyclic-stream/serde --target x86_64-pc-windows-gnu
cargo check --locked -p acyclic-stream --no-default-features --features wasm --target wasm32-unknown-unknown
Windows runtime build used Q:\lanes\lb.ps1 cargo test --locked -p acyclic-stream --features local,http,test-support,serde --lib --no-run. macOS used the same focused target over ssh ivar with CARGO_TARGET_DIR=~/mac-target/co-stream-lock. Paired measurement command on each OS:
"$STREAM_TEST_BIN" local::snapshot_ready::ready_snapshots_allocate_zero_and_match_independent_prefixes --exact --nocapture
"$STREAM_TEST_BIN" local::snapshot_ready::negative_control_reintroduces_per_record_boxing --exact --nocapture
"$STREAM_TEST_BIN" query_sync::parity::admission_and_cancellation_match_tokio --exact --nocapture
"$STREAM_TEST_BIN" memory::tests::parking::unpolled_streams_do_not_block_appends --exact --nocapture
"$STREAM_TEST_BIN" memory::tests::runtime_shutdown::pending_stream_survives_its_polling_runtime --exact --nocapture
CLI repetitions used the same built binary, serially:
for i in $(seq 1 100); do
"$CLI_BIN" dumb_terminal_uses_the_shared_headless_runner_without_tty --exact --nocapture || exit 1
done
Live steps
Await coordinator review and merge. No platform validation remains pending once the final results above are recorded.
Disclosure
- No new third-party imports or provenance changes.
- Generated descriptors and README unchanged; no private artifacts added.
- All commits are signed and include DCO sign-offs.