feat(voice,host): the avatar's frames on the voice socket, its grant and status in process, and voice.setVideo

Merged
CH
5d ago
+720-18

Plan item 8. host: avatar frames and setVideo. Stacks on #2472 (contracts).

What

  • live-session-source.ts
    • The hosted source's LiveSessionOpened gains startAvatar() and stopAvatar(reason). They send avatar.start and avatar.stop {reason} on the same socket, held through a reattach gap like any send.
    • It also gains onAvatarStarted and onAvatarUnavailable. withoutServiceFrames takes avatar.started and avatar.unavailable off the arrivals the same way it takes the plan.* frames.
    • An avatar.started that its schema refuses (for example a ws: room) is still taken off the socket, since it may carry a token. It is told as avatar.unavailable {error}, so a start is never left waiting.
  • LiveSessionHolder
    • startAvatar(): boolean asks only while a call stands and no avatar is already asked for or up, because each start mints a Keyframe session the plan's concurrency pays for. After an end, it may ask again.
    • stopAvatar(reason): boolean withdraws a handed-on grant.
    • The call's end moves the avatar to ENDED {call_ended} and withdraws the grant. It sends no stop, because the service closes the avatar with the call.
    • Two in-process observables:
      • avatarStates: Stream<LiveAvatarState> replays the current state, then each change: OFF | STARTING | GRANTED | ENDED {by: stopped|unavailable|call_ended, reason}.
      • avatarGrants: Stream<AvatarGrant | undefined> replays nothing, so no grant is kept for reuse. A grant reaches only the subscribers standing when it arrives. undefined means the avatar it joined is down.
    • A grant that lands after a stop, a refusal, or the call's end is dropped.
  • voice.setVideo {on} (@sidecar/gateway and compose-live.ts): on calls startAvatar(), and off calls stopAvatar(user). The answer is {requested: boolean}, and no Gateway answer or event carries the grant.
  • LiveDependencies narrows settings and account to the fields compose-live reads, so the test can stand it up. Compose-host's call site is unchanged. HostedLiveSessionSource is exported from @sidecar/voice for that test.

Consumer

#11 (wire-up) consumes avatarGrants (to callWindow.showAvatarSession) and avatarStates (to state.voice.avatar), and adds the VOICE_SET_VIDEO act and the auto-start after session.created.

The host status is its own vocabulary, because packages cannot import #shared/messages/avatar. #11 maps it like this:

  • STARTING and GRANTED → STARTING. LIVE is the call window's own report.
  • stopped/user → USER
  • unavailable/no_capacity → NO_CAPACITY
  • unavailable/not_configured → UNAVAILABLE
  • unavailable/error → ERROR
  • call_ended → CALL_ENDED

When the call window's session closes or fails, #11 calls holder.stopAvatar(server_closed | error) in process. That ends the avatar here and tells the service, so "Restart video" (setVideo on) can draw a fresh grant.

Web side (#7): the sessions route still closes a desktop socket that sends any other frame. That's harmless now, because nothing calls voice.setVideo until #11.

Tests

packages/host/src/compose-live.test.ts runs the real hosted source over the existing scripted socket (@sidecar/voice/testing), the real holder, and compose-live's method table:

  • on → one avatar.start, and a second press while starting sends nothing;
  • the grant reaches avatarGrants; off → avatar.stop {user}, the grant is withdrawn, and the states run in order;
  • no Gateway answer or event carries the token, checked against the serialized bytes;
  • with no call, nothing is asked; a non-boolean is an invalid-params refusal;
  • after an avatar.unavailable, a late grant is dropped, and the next start draws its own grant;
  • a ws: grant is read as unavailable error;
  • the call's end withdraws the grant and sends no stop.

I mutation-checked them: removing the call-end withdrawal fails a test.

./scripts/check.sh exits 0. This PR doesn't touch the Mac or the UI (packages only), so it needs no verify.sh evidence. CI builds nothing for the Mac.

🤖 Generated with Claude Code

Open workspace in Conductor

Devin Review

Summary by cubic

Adds the avatar lifecycle to the hosted voice session and exposes it through a new voice.setVideo Gateway method, keeping every grant in process only.

  • startAvatar() and stopAvatar(reason) send avatar.start and avatar.stop on the voice socket; each start carries its own request id, and only the latest start's answer is heard.
  • avatar.started and avatar.unavailable are taken off arrivals like the plan.* frames; an answer its schema refuses still ends the start it names as unavailable (error).
  • voice.setVideo {on} answers only {requested}; no Gateway answer or event carries the grant token.
  • avatarStates replays the current state (OFF | STARTING | GRANTED | ENDED); avatarGrants replays nothing, so a single-use credential is never kept for a later start.
  • A grant landing after a stop, refusal, or call end is dropped; the call's end withdraws the grant without sending its own stop.
  • A new call starts with the avatar off, not at the previous call's end state.