feat(host-service,mobile): host sessions sync v1 — canonical sessions over tRPC snapshots + /sessions/sync

SA
3mo ago
+23393-12595

Supersedes #5630. This is @bra1nDump's host sessions sync v1 — the canonical session plane (tRPC snapshots + /sessions/sync WS) with mobile fully rewired onto it — picked up, brought up to date with main, put through a four-track deep review, and hardened. All of Kirill's commits are preserved in the branch history; see #5630 for the full protocol spec (three review layers) and rollout doc, which still apply verbatim.

What this adds on top of #5630

Merge with main (139 commits of drift)

  • Kept the deletion of the mobile chat/acp/ route against main's re-registration.
  • Both sides had added host-service migration 0010; kept main's chain and regenerated session_meta as 0012 via drizzle-kit (identical DDL).
  • Ported main's new FinishReviewSheet (#5838) off the deleted ACP APIs onto useHostSessions + submitTurn.
  • Fixed the stale getSessionThreadPresentation test (still asserted the old ACP field names; mobile typecheck doesn't cover test files, so it shipped red on the original branch).

Host runtime hardening (packages/host-service)

  • The spec promises a "RAM ring (~5k events)" but the per-session log was unbounded — a long session grew host RSS until OOM. It is now an actual 5k ring with a serial offset: eviction resets late subscribers through the existing cold path (foreign_cursor → reset(CURSOR_INVALID)) and surfaces truncatedBefore in scrollback.
  • The session-meta orphan sweep deleted all titles/archive flags if registry hydration had failed at boot (empty "known" set = everything looks orphaned). The sweep now refuses to run after a failed hydration.
  • Evicted dead runtimes are parked back as offline registry records instead of becoming permanently un-gettable while their log stayed resident.
  • submitTurn on a closed session no longer spawns an adapter child before rejecting.
  • Failed settings/permission mutations disarm their causation attribution, so a later unrelated event can't echo a stale requestId; settings batches validate fully before the first port mutation (no more half-applied batches).
  • Translation and projection fold failures are logged instead of recorded silently; sync frames are measured in UTF-8 bytes, not UTF-16 code units; a dispose guard stops in-flight resurrections from re-registering after shutdown; turn-id caps got their own constant (were borrowing the tool-call/subagent caps).
  • Shared reducer: a finished turn now reaps its own still-active tool calls, so a window missing the host's terminalize updates can't leak "running" entries forever.
  • Timeline fold: the message→turn index is now rebuilt on incremental folds. Previously foldTimeline(empty, all) and the incremental streaming path marked different user messages as failed — a violation of the shared-fold contract the package rests on.

Sync client resilience (packages/host-service-sync/client)

  • A transient tRPC failure during the cold path (host or session snapshot) parked the client/stream in error forever — nothing retried. Both now retry with epoch-guarded exponential backoff.
  • Retain intent survives host-snapshot prunes; sessions created concurrently with a cold path seed when their sessionUpsert lands (previously the pane the user just opened stayed blank until remount).
  • Ping/pong heartbeat + hello deadline: half-open sockets (laptop sleep, dead tunnel) now close into the reconnect path instead of reporting a live connection that never delivers again.
  • The reset circuit breaker is cleared on socket close (the documented "cleared on reconnect" semantic — previously a parked stream re-parked on its first post-reconnect reset), and parks on the Nth reset as documented (was N+1).
  • A session snapshot resolving after socket loss downgrades the stream to idle instead of a permanent subscribing spinner.

Mobile chat fixes (apps/mobile)

  • Permission cards for already-settled asks stay dismissed (the blanket restore-on-error made a stale card undismissable — every tap NOT_FOUND → restore, with zero feedback); real failures restore the card and surface in the thread error banner.
  • A failed send restores the typed message into the composer (the prompt input clears synchronously on submit, so a relay hiccup destroyed long prompts).
  • The scroll-to-bottom button now shows whenever the reader is not pinned at the true end — the one-sided at-bottom test hid it for readers resting inside the banked inset, leaving no affordance to resume following (invariant documented in ai-elements/AGENTS.md).
  • scrollToBottom cancels in-flight anchor legs; lineDiffStats (megabyte string splits) is memoized.

Review coverage

Four parallel deep reviews (protocol/projection/timeline, host runtime + router + hub, client store + React bindings, mobile rewiring), cross-checked against the 66 existing review threads on #5630 (Kirill had fixed or explicitly deferred nearly all of them — the remainder are in this PR or the follow-ups below). Confirmed-clean areas worth recording: WS auth matches the other socket surfaces, first-write-wins permission resolution is genuinely atomic, no adapter double-spawn, reducer purity holds, no dangling references to the deleted ACP/Mastra planes anywhere in the repo, and Reanimated worklet usage in the new indicators is correct.

Known follow-ups (deliberately not in this PR)

  1. plan is projection state fed only by planUpdated events, and snapshots don't carry it — a plan renders empty after reload/re-tracking until the agent rewrites it. Needs plan in the session snapshot (protocol + host + client fold).
  2. The relay transport flattens tRPC errors to message, discarding data.code — the mobile permission-stack fix has to classify NOT_FOUND by message text. Preserve the code in packages/host-client transport and type it.
  3. Client store hot-path perf: per-event full-map copies, full-array rebuilds in useSessionTimeline, and a quadratic total-byte eviction loop — fine at current sizes, degrades on 10k-event sessions.
  4. Replay-overlap safety in the shared reducer holds only for contiguous windows (a re-folded turnStarted after its turnCompleted re-pins running); today's single fold-at-mint caller is safe, but the contract should be tightened or start events guarded.
  5. Hub outbound frames have no size guard, so a schema-valid >1MiB snapshot would force-close the receiving client (frame_size_limit) — snapshot pagination is the real fix (Kirill's deferral, still open).
  6. Hub packet processing is serialized per connection, so a multi-second warmSession (adapter spawn + replay) stalls pings behind it.
  7. withReceipt fingerprints with JSON.stringify(input) — key-order differences give spurious CONFLICTs.
  8. Mobile: handleSend resolves silently when mainThreadId is null (text lost with no feedback); duplicate permissionRequested ids append a second unanswerable card view in the timeline fold (needs a re-emitted live elicitation to trigger).
  9. Kirill's rollout items 3 and 4 remain the roadmap: direct Claude SDK adapter behind this same surface (prototype archived in #5582), then the Maestro phone replay as the final gate.

Verification

  • bun run typecheck — 37/37 tasks.
  • bun run lint — clean; sherif — clean (fixed a @types/react version split the new package introduced).
  • bun test: host-service 972 (0 fail, incl. the 11-test realistic-scenario suite and reboot/durability suites), host-service-sync 53, session-protocol 34, host-client 8, host-service-react 2, mobile 8.
  • The reviewer repro cases for the timeline fold-divergence and reducer tool-call leak now fail-as-bug-fixed.
  • Not run: the real-model gym E2E (needs a live Opus session) and the Maestro conversation-lab flow (needs a signed-in simulator) — both should gate merge per the spec's own checklist.

https://claude.ai/code/session_01XFhc7RWVyTo1VYgQKzq6GJ


Summary by cubic

Introduces the canonical sessions plane (tRPC snapshots + /sessions/sync) and migrates mobile chat to it for more reliable live sessions and simpler APIs. Removes the ACP path, adds React bindings, and ships a scroll regression lab.

  • New Features

    • Added canonical sessions.* API and sync client (@superset/host-service-sync) with a sessions-capable @superset/host-client.
    • Added @superset/host-service-react hooks (SessionsSyncProvider, useSession*, useSessionTimeline) and rewired mobile to them; removed ACP chat screens and types.
    • Added conversation scroll regression lab (Maestro flow) and AGENTS.md; ported FinishReviewSheet and session list to the new surface; unified chat model picker (CHAT_MODELS).
  • Bug Fixes

    • Host runtime hardening: bounded per-session event log (ring), safe orphan sweep, park evicted runtimes, reject turns on closed sessions, full batch validation, tool-call cleanup on turn finish, consistent timeline folds, and clearer logging/guards.
    • Sync client resilience: backoff retries for cold snapshots, ping/pong heartbeat and hello deadline, correct reset breaker semantics, and idle downgrade on socket loss.
    • Mobile chat: settled permission cards stay dismissed, failed sends restore the input, correct scroll-to-bottom affordance, anchor scrolls cancel cleanly, and diff stats are memoized.

Summary by CodeRabbit

  • New Features

    • Introduced a unified live chat experience with real-time streaming, session history, permissions, tool activity, plans, and typing indicators.
    • Added session creation, renaming, archiving, cancellation, and settings updates.
    • Added chat model selection with validated defaults and support for image attachments in new conversations.
    • Added reliable session recovery and history loading after reconnects or host restarts.
  • Bug Fixes

    • Improved conversation scrolling, keyboard handling, overlays, and message anchoring to prevent unexpected movement.
  • Tests

    • Added comprehensive regression coverage for chat scrolling, session synchronization, permissions, recovery, and tool activity.