Push live child discovery and browser follows
Current signed head: d7f9f90ac1e0bf0ce7f71910437aae9843fc8e3a; #579 inherits this same head. #647's compile fix is included. Review/land order remains #647 -> #557 -> #579 -> #650 -> #644.
Latest review fixes:
- A frame-validation error closes/fails only its originating node's link and routes. Shared dispatch keeps independent nodes and later opens live. The real two-node regression moves discovery to a second node, injects a non-direct child after its channel has received credit, and checks healthy existing/later follows, one connection/session per node and zero actor channels.
- Resumed child discovery validates anchor existence with accepted
watermarks(), rather than durability-gatedtail(). The held-device-flush regression now opens a fresh Session at the accepted child cursor and requires Opened before releasing the flush. client_pool_loomcompiles the production Pool/receive/close path under the crate-localacyclic_stream_loomMutex/Atomic/Signal/Outbound shim. It exhausts both peer-event arrival orders, checks independent delivery/credit/later opens, and includes a deliberately broken global-failure variant. The opaque transport handle is inert; dependency Tokio synchronization is not used as model evidence. Existing queue/wakeup/cancellation models remain unchanged.
Validation of this delta is pending bb1. The obsolete queued probe was cancelled before admission; one replacement batch is queued, building the native target and the necessarily separate instrumented pool-model target with the same local,http features/debug profile, then reusing their binaries. Every runtime invocation has a <10 s watchdog and the batch stops on its first failure. No current-delta pass is claimed. #647's two requested Windows checks passed at c075800b4; they do not validate this later delta.
The following earlier results and measurements are historical baselines, before the current main merge and latest two fixes.
Main was propagated through #647 with a signed normal merge at 54588582839997502494605edc24ff4cb9bbfb07. Resolution retains credential and latency test modules, pinned TLS/session initialization, and adapts main credential-provider fields to session constructors; changing credentials starts a separate session pool. Earlier results below predate this merge; merged-head validation is deferred until bb1 returns, per user instruction. No additional build was queued.
Live child discovery and record follows now use the multiplexed Streams session in Rust HTTP/gRPC and the TS HTTP/gRPC/default adapters. Local and WASM providers retain in-process push. The HTTP 250 ms loop, browser 100 ms loop and finite TS discovery path are deleted.
Review and land order is #647 -> #557 -> #579 -> #650 -> #644. This branch contains the native session foundation from #579 together with the live providers that foundation requires. #579 will inherit this head so switching discovery to the session adapter never lands ahead of live native providers. No additional main merge was needed for these review fixes.
Review fixes
- Node request streams drain independently from bounded byte/frame queues. Shared dispatch never awaits outbound capacity: subscription admission rejects the affected channel with Capacity; a control queue that cannot accept cancellation or returned credit fails only that node. Link epochs fence the cancelled response reader. Cancelling an opening follow fences its queued Open even when removal is dispatched first.
- The append/follow latency fixture registers StreamService and StreamSessionService against the same provider.
- Local child subscriptions and delivery use accepted visibility, independently of device flush. A held-flush regression delivers the child while its durability acknowledgement remains pending.
- Parked discovery also subscribes to provider failure. A real persistence failure on an unrelated stream wakes it with Unavailable.
- Browser publication refresh runs once per notification, then dispatches only changed record paths and newly created child parents. Followers read the refreshed cache without opening their own IndexedDB transactions. A 128-follower/eight-append test bounds refresh transactions by publications and record-read work by affected followers. Rejected writes remain invisible.
- Rust and TS child channels share record-follow connections, credit, cancellation and moved/reconnect handling. Opaque resume state advances atomically with the yielded child. Tests cover queued replay across moves and disconnects, discarded unread credit, deadlines during blocked subscription/ACK writes, and cancellation during default-provider selection.
- Child framing reserves both the maximum path and its resume anchor; a maximum-size runtime regression and shrinking encoded-size property cover the byte budget without raising session limits.
Verification
Current review-delta head: 4c47fc275. The real bounded-queue/cancellation model suite passes all 29 tests in 0.10 s, including negative controls and shrinking exact-byte/FIFO accounting. The first runtime run passed 26 session checks, including public gRPC creation after snapshot exhaustion, but the new hostile-peer fixture reached its eight-second watchdog without a phase diagnostic. Code inspection found that its remote-cleanup wait required EOF from a peer that never drained its request. The fixture now keeps the peer paused through healthy delivery and cancellation, then explicitly resumes request draining before requiring remote EOF: Hyper cannot finish a buffered request while that peer withholds HTTP/2 credit. The corrected focused runtime run is queued behind bb1 resize maintenance. Remaining gRPC/HTTP groups will reuse its built binary; current delta validation is pending bb1. No failed test is claimed as passing.
The following results are the earlier live-discovery baseline, before this bounded-queue review delta. All commands ran through Q:/lanes/bs.ps1; each executed suite is below 10 s. Built Rust binaries were reused.
| Focused suite | Result | Runtime |
|---|---|---|
| Native child discovery, including held flush and unrelated failure | 16 passed | 0.01 s |
| Native session/codec, including moved/reconnect and maximum child | 24 passed | 4.01 s |
| HTTP session/TLS and retained JSON unary regressions | 7 passed | 0.05 s |
| N-API cancellation and actual TLS record/child cursor lifecycle | 9 passed | 0.05 s |
| gRPC regressions, including independent retry span captures | 12 passed | 0.26 s |
| Browser IndexedDB/publication regressions | 8 passed | 0.28 s |
| TS child/session TLS/browser transports | 8 passed | 0.642 s |
| TS declaration, memory cursor and default selection checks | 6 passed | 0.276 s |
| Actual discovery/publication/cursor/shared-initialization models and properties | 32 passed | 0.03 s |
| Actual native cursor/credit/gate/signal models and properties | 19 passed | 0.04 s |
| Actual TS session/wrapper models and properties | 40 passed | 0.92 s |
The discovery models execute MemoryStream/LocalStream through the crate-local acyclic_loom lock/watch shim; the publication models instrument the actual Change, keyed interest registry and refresh handoff through acyclic_stream_loom. They cover parent creation, hierarchy-to-parent watch switching, wait registration, cancellation, accepted versus durable visibility and failure notification. Deliberately broken variants must fail. Lock orders are documented; reentry/waking-under-lock/source-destruction probes detect violations. No model relies on dependency Tokio locks being instrumented. Enable both cfgs only in the dedicated publication-model command.
TS models enumerate bounded external callback orders and run the actual Bun modules, including the raw Connect deadline wrapper and default provider selection. New negative controls drop the yielded opaque cursor, omit selection/deadline forwarding and make blocked writes uncancellable. Shrinking properties verify path/parent validation and codec round trips. Both existing cargo-fuzz session targets now exercise child frames/resume inputs; checked-in seeds include valid and non-direct children. Debug ASan/coverage runs completed 128 inputs each in <1 s. Fuzz runs characterize the parser boundary; they are not exhaustive parser proofs.
Strict stream-library clippy with --no-deps -- -D warnings, TypeScript project checking, and the maintained debug WASM producer pass. The new native ABI method is exercised directly over TLS. LF/source-size and diff checks pass.
Latency
All measurements are on shared bb1. Browser samples measure append invocation through follower yield, including IndexedDB acceptance: eight warmups and 32 measured pairs; p99 is the largest measured sample. Before used the original timed follow; after uses the new keyed publication path.
| Path | Before p50 / p99 | After p50 / p99 |
|---|---|---|
| Browser IndexedDB | 101.110 / 103.080 ms | 0.460 / 0.920 ms |
| MemoryStream (earlier discovery comparison) | 1.672 / 20.421 us | 1.492 / 18.869 us |
| Loopback gRPC (earlier discovery comparison) | 64.777 / 3154.107 us | 56.986 / 77.407 us |
| Rust HTTP through native Session, current head | unmeasured | 23.815 / 34.201 us |
Native samples use 16 warmups and 128 measured records with cursor assertions. Earlier native results characterize the discovery change and already-push-based record provider; they are not paired performance claims for the current session implementation. TCP_NODELAY is enabled. Deployed service measurements remain outstanding.
Decisions
- Harness/UI state remains volume files consumed by live queries. This is platform transport, not a harness server, gateway, daemon, store or application message. ClientViews and the interest set remain gc-client's reducer boundary; these providers supply canonical child/record deliveries to that fold rather than introducing a second reducer in Streams.
- The generic SDK Session service implements the server side for platform hosts. Thousands of logical follows share one connection per client/node; node providers use the same client for upstream fan-in. Browser HTTP uses
acyclic.session.v1; native Rust HTTP uses the existing Session RPC. Existing JSON unaries retain their response bounds. Session frames retain canonical byte/frame credit and bounded queues. - HTTP's additional CA extends platform trust for its Session connection and moved-node dials. Native gRPC's separately documented exclusive pinned-CA behavior is retained.
- Children replay follows creation order; finite
children_pageretains lexical ordering. Limit bounds replay work, not subscription lifetime. The per-parent creation index links live in charged PathState storage and rebuild from snapshots; limit=1 replay visits each creation once. - Explicit finite listing operations remain available in WASM/N-API via children_page. New live cursor operations reuse the existing cursor ABI. Third-party TS providers that omit optional live children get a typed Unsupported result, never a polling/page fallback. Finite fixture callers use the explicit page operation or stop their live iterator deliberately.
- Handshake deadlines and failed-transport retry backoff remain; none is a timer for discovering new data. SDK follow and discovery contain no idle reread loop.
- S3 finite listing callers use children_page. Its oversized source was split into plain methods below the file limits; extracted operation bodies were preserved.
Live steps
- Main-merge integration follow-up: native Session reconnect currently reuses the credential captured at bootstrap; renewal through main's new CredentialProvider must be wired into physical Session reopens. Changing credentials explicitly already creates an isolated session pool.
Cloud c41/c40 must deploy the canonical Session endpoint and node fan-in and remove their timed applied_after(seen, FOLLOW_POLL) reread. No new gateway is required. The gc-client/gc-query lanes own ClientViews interest-set integration, materialized head files and snapshot-at-cursor tier transitions. Native hosted checked-history follow for filesystem publication queries still needs its platform binding; the local task-volume snapshot/cursor follow is already pushed. Deployed before/after latency and the separately requested 10k-agent qualification remain to land.
Disclosure
- Uses existing pinned dependencies, with loom/proptest in dedicated model crates.
- Existing Session field numbers are retained; the additive checked-follow fields are reviewed in #647.
- No credentials or private operational data included.
- Signed commits with DCO sign-off. No README changed.