feat(desktop,host): wire the avatar's grant, status, feed, and toggle end to end, still dark (avatar 11/12)

Merged
CH
5d ago
+1224-172

Plan item 11. wire-up, still dark (avatar 11/12). This connects the grant, status, feed and setVideo legs that #7–#10 landed with no production caller. The feature stays dark: nothing turns video on unless LUKE_VOICE_VIDEO=1 is set in an unpackaged build. #12 replaces that switch with the voiceVideo setting.

What it wires

  • Host → main, in process only. HostAssembly gains avatar: { states, grants, closed(reason) }, taken from the live holder. No Gateway method or event carries any of it. The holder now uses AvatarGrant from @sidecar/hosted/avatar-grant and no longer has its own copy.
  • Grants → call window. linkAvatar (main/services/avatar-link.ts) starts reading the grants stream at launch, right after createWindowService, and hands each grant to callWindow.showAvatarSession. It has to subscribe at launch because the PubSub keeps nothing for a late subscriber. CallAvatarRelay still holds the grant for a page that draws late.
  • Host states → state.voice.avatar, and the host wins. avatarAfter is one pure function, with a table test:
    • OFF→OFF and STARTING→STARTING.
    • GRANTED leaves the document unchanged, so the window's report decides.
    • unavailable: no_capacity→NO_CAPACITY, not_configured/error→UNAVAILABLE.
    • stopped: user/server_closed/error→USER/SERVER_CLOSED/ERROR.
    • call_ended→CALL_ENDED.
    • The call window's report is accepted only while the host is GRANTED. When a grant is withdrawn, the window's own ENDED USER is dropped, so CALL_ENDED and a host stop keep their reason. The call window now opens on the host's STARTING, before any grant arrives.
  • A close we didn't start. If the window reports ENDED SERVER_CLOSED or ERROR while the host is GRANTED, main calls host.avatar.closed(reason). The holder then sends avatar.stop with that reason, which closes the row and clears GRANTED, and asks for one new session. It does this once per call; the count resets when a session is created. A second such close leaves the avatar ENDED, and the tile shows "Restart video". Main maps the desktop's kebab reasons to the hosted snake reasons in one table, unstartedStopReason.
  • Feed. A voice-only send bridge entry, feedAvatar (25 messages a second, no answer). Main drops a message unless the document says STARTING or LIVE, and otherwise relays it with callWindow.feedAvatar. It is bound in voice-host.tsx as a module constant, so the hook's single read always sees the same object.
  • setVideo. ACT_KIND.VOICE_SET_VIDEO {on} is accepted only from the call window. It calls HostOperator.setVideo, which calls GATEWAY_METHOD.VOICE_SET_VIDEO, and the tile's Video off and Restart video buttons are now bound to it. Video off while the avatar is ENDED has nothing on the host to stop, so main writes ENDED USER itself and the tile goes away.
  • "Restart video" is drawn. One shared avatarTiled(state) replaces the two videoUp copies. It is true on STARTING or LIVE, and on ENDED with SERVER_CLOSED, ERROR, NO_CAPACITY or UNAVAILABLE. Callers AND it with "the call stands". CallWindowPresence.voiceChanged now takes the full avatar state.
  • Auto-start. LiveDependencies.settings gains videoOnCall: Effect<boolean>. compose-host reads LUKE_VOICE_VIDEO=1 through the Environment seam, and only when the build is not packaged. The holder calls startAvatar() in its create path once #held is set.
  • A start lost in a gap. Each start has a 20 s bound (AVATAR_START_BOUND) on the Effect clock. If the service hasn't answered by then, the holder sends avatar.stop {error} to close whatever the ask put up, and ends the start as unavailable/error, which shows "Restart video". A grant that arrives after the bound is dropped.
  • Sentry beforeSend. eventWithoutAccessTokens drops any event whose message, exception message or request URL contains access_token, the same rule the breadcrumbs follow. It is installed in renderer/index.tsx and preload/index.ts.
  • Cleanup. Fixed the stale "#11 adds it" and "binding is #11's" comments. avatarStateSchema is narrowed so the call window can't report OFF. The knip entries for the worklet and the bundle are untouched.

Where this departs from the task list

  • HostAvatar has no start() or stop(reason). Item 6 sends setVideo over the Gateway, so the in-process versions would have no caller. Item 4's stop-then-restart lives inside closed(reason).
  • avatarFeedSchema is not exported. The bridge guard uses the existing avatarFeedFromWire, and an export with no caller would fail knip.
  • AGENTS.md is untouched, as the orchestrator asked.

Credentials

No grant or token reaches a Gateway answer or event, app state, a trace, a counted event or a log line. The grant goes from the holder's in-process stream to linkAvatar to CallAvatarRelay and then only to the call window's page. No new log line carries it. The existing compose-live test that checks no token crosses the Gateway still passes.

Tests

  • packages/host/src/compose-live.test.ts (the holder over the scripted socket):
    • a close we didn't start stops with its reason and restarts once;
    • a second close leaves the avatar ENDED;
    • a close with no grant sends nothing;
    • the restart count is per call;
    • auto-start on and off;
    • the start bound fires under TestClock, and a start answered in time is left alone.
  • apps/desktop/src/main/services/avatar-link.test.ts: the precedence table.
  • apps/desktop/src/main/ipc/avatar-acts.test.ts, through the real act router and linkAvatar over a fake host avatar:
    • sender checks for the report, setVideo and the feed;
    • STARTING opens before a grant;
    • CALL_ENDED wins over the window's ENDED USER;
    • a server close restarts;
    • Video off while ENDED;
    • the feed is gated by the avatar's status.
  • call-window-presence.test.ts: the tile stays up through a restart reason. avatar.test.ts: avatarTiled. sentry-breadcrumbs.test.ts: beforeSend.

./scripts/check.sh exits 0 locally.

Evidence

CI could not verify this; Mac evidence is pending. This Linux VM can't run verify.sh or run.sh, and CI builds nothing for the Mac. Still owed: a manual end-to-end on a Mac with LUKE_VOICE_VIDEO=1, which will be the first real Keyframe session. It needs KEYFRAME_API_KEY and LUKE_KEYFRAME_PERSONA_ID set in Vercel.

Never run against a live Keyframe session (none has ever been opened):

  • the call window opening on STARTING and growing to the tile;
  • LIVE on the first frame;
  • lip sync from the relayed feed, and the handover with no word heard twice;
  • a real Keyframe room close (the 5-min free-plan cap) triggering the one automatic restart;
  • whether 20 s is the right bound for a real mint;
  • Restart video and Video off on the real tile;
  • whether a real LiveKit join error is scrubbed by beforeSend.

🤖 Generated with Claude Code


Open workspace in Conductor


Devin Review


Summary by cubic

Wires Luke's avatar end to end from the host's live session to the call window's tile, still dark behind LUKE_VOICE_VIDEO=1 in unpackaged builds. The host now hands main the avatar's states, grants, and a closed(reason) hook; main relays grants and the voice feed to the call window, writes the document with the host's word winning, and carries the tile's Video off and Restart video. Video off asks the host before it takes an ended tile down, so it stops a restart already asked for instead of racing past it.

  • The host auto-starts the avatar on a call when videoOnCall enables it; a close this side didn't start stops the avatar and asks for one restart per call.
  • A start the service never answers ends as unavailable after 20 s; the tile stays up through restartable ends and offers Restart video.
  • The call window's report stands only while the host holds a grant; a withdrawn grant drops the window's own end, and a withdrawn session still closing says nothing over the next grant. A server- or error-close reports back to the host for a restart.
  • The voice window's avatar feed is relayed to the call window only while the avatar is starting or live.
  • Sentry beforeSend drops events whose message, exception, or request URL carries a LiveKit access token, matching the breadcrumb rule.