Reduce durable work in local command sessions

Merged
VA
2d ago
+822-28

Standalone command sessions repeatedly checkpointed private core-state records and published identical receive input. Exact stream retries also serialized and flushed another WAL frame. The production paths now retain the existing owned WAL, share the immutable receive input, coalesce JSON writes, and reuse the original retry frame while preserving its durability requirement.

Profiling on bb1 found 1,493 fsync and 3,504 fdatasync calls in the baseline inheritance suite. A separate profile of the real branch/fork/reopen command regression attributed 2.27 s to 458 data syncs, compared with 0.045 s waiting for shell output.

Measurements

Paired debug runs on the same bb1 slot, alternating baseline/candidate order, three pairs, fixed property seed. Baseline is 38f8731ba with only the compaction-test allocation-counter compile repair.

SuiteBaseline medianCandidate median
Client commands (98 tests)8.79 s7.42 s
Original inheritance/bounds cases (8 on both)3.69 s3.26 s

Client CPU median: 21.46 -> 16.61 s. Inheritance CPU median: 4.51 -> 4.21 s.

The matching eight-case syscall profile reduced writes from 32,901 to 20,887, and fsync/fdatasync calls from 4,997 to 4,302. Work-bound properties separately check exact savings rather than wall time.

All ten tests in the expanded inheritance suite pass; its median including the two added regressions is 5.32 s. The matched benchmark excludes only those two new tests, on both binaries, after running the full suite. All original cases and assertions are retained. The client target remains above 5 s.

Raw paired wall times (baseline / candidate), seconds:

  • Client: 8.787/6.806, 10.621/8.022, 8.220/7.424.
  • Original inheritance cases: 3.633/3.262, 3.695/3.073, 8.563/5.895.

Validation

  • All client-command and inheritance tests pass in every measured run.
  • Shrinking work properties cover one WAL barrier per standalone update, bounded resident cache, no publication for repeated constant input, exact JSON bytes with coalesced writes, and no duplicate replay frame. Each uses the same assertion against its old-behavior control.
  • Exhaustive models exercise the production owner-registry synchronization (3 cases, 0.008 s) and replay durability handoff against the actual journal flush/watch publication (2 cases, 0.032 s), including negative controls, without preemption caps.
  • Focused checkpoint failure, ownership, saved-view reopen, reconciliation interruption, cancellation/admission and compaction-allocation regressions pass. Stream conformance, deadlines, concurrent commits/reopen, clock-exhausted retries, deferred flush and caller-cancellation cases pass.
  • Windows GNU cross-check passed at 749a7d8f2: cargo check --locked -p acyclic-harness -p acyclic-fs -p acyclic-stream --lib --tests --target x86_64-pc-windows-gnu. Native closures: Harness tui,terminal,filesystem-local,native-execution,test-support; FS local,memory,native-mount without defaults; Stream grpc,local,serde,test-support. Callers checked.

Decisions

  • Keep embedded bind() on the existing shared-store path; standalone sessions own their private companion namespace. Same-process reopening joins a weakly held owner, while the OS lock excludes other processes. Bound the joinable resident cache using the existing checkpoint threshold.
  • Preserve legacy receive references and fingerprint checks on retries. Only the immutable input is shared; each receive command retains its distinct operation identity.
  • A normal retry of a deferred original requires and flushes that frame before acknowledgement. Replay fingerprint and deadline checks still run.

Summary by cubic

Reduces durable work in local command sessions: standalone harnesses now reuse the existing owned core-state WAL through an in-process owner registry, identical receive inputs are shared instead of republished per turn, and stream retries reuse the original durable frame instead of appending a duplicate. Legacy retry references, fingerprint checks, and deferred-frame durability requirements are preserved.

  • Reopening a standalone namespace joins a weakly held in-process owner (the OS lock still excludes other processes) and bounds its resident cache, so each transaction no longer reopens and checkpoints the WAL.
  • Normal retries of a deferred original require and flush that frame before acknowledgement, while ordinary replays elide a new frame.
  • JSON record writes are buffered into one backend write with canonical bytes unchanged; standalone sessions complete exactly one WAL barrier per update.
  • Paired runs on bb1: client-command suite median 8.79 s → 7.42 s, the eight original inheritance cases 3.69 s → 3.26 s, writes 32,901 → 20,887, and fsync/fdatasync syscalls 4,997 → 4,302.