feat(web): the voice service answers avatar.start and avatar.stop on Luke's Keyframe key and keeps each session's row (avatar 7/12)

Merged
CH
5d ago
+1016-18

Plan item 7. web: avatar frames and records, built on #2472 (contracts), #2469 (Keyframe client) and #2467 (DB).

What changes

  • function.ts reads KEYFRAME_API_KEY (as Redacted) and LUKE_KEYFRAME_PERSONA_ID, and hands the service Keyframe.layer({apiKey, personaId}) over FetchHttpClient.layer. If either is missing there is no avatar, and every avatar.start gets avatar.unavailable {reason: not_configured}.
  • frames.ts / relay.ts: avatar.start and avatar.stop get a new AVATAR decision. The relay checks them against the contract's reader and hands them to the service (one that fails the reader closes the socket, like a malformed report). They are never forwarded to OpenAI. The relay also tells the service when the device hangs up.
  • avatar.ts (new) is one call's avatar. Asks are handled in order by the repo's serialQueue, so a stop sent right after its start closes that start's row.
    • start: closes whatever avatar rows the call still has open (reason user). Then it mints a fresh grant (never cached) and checks it with the contract's avatarStartedFrame reader, so a non-wss: room becomes an error answer instead of a frame the desktop would silently drop. Then it opens a row and sends avatar.started with the requestId echoed. If openAvatarSession returns none, the answer is error. A start for a device that has already gone mints nothing.
    • Keyframe failures: NO_CAPACITY → no_capacity; PLAN_REFUSED / UPSTREAM → error. Keyframe documents 503 for busy and 403 for plan refusal; #2469 also reads 429 as no capacity, which Keyframe does not document.
    • stop: closes the row its request opened. Hosted user / server_closed / error map explicitly to DB user / server_closed / error. A stop for an older request than the one this connection holds is ignored. If this connection holds no avatar (it re-attached), the stop closes what the call still has open.
    • hang-up: stops new starts and closes open rows as call_ended. session.closed writes the voice row first and then runs closeOpenAvatarSessionsOf (call_ended), the order #2467 requires.
    • Logging: the log keeps only new URL(serverUrl).host (avatar-started) and the unavailable reason with Keyframe's status. The grant and token are never logged, traced or counted.
  • orphan-sweep.ts: whenever the sweep closes an orphan's voice row (closed, lost or expired), it then closes that call's open avatar rows as orphaned.
  • Every orphan sweep also runs closeAvatarSessionsOfEndedCalls(): any avatar row still open on a voice row that is already closed is closed as call_ended, as of the call's own close. This repairs an avatar close that failed after the voice row's close landed (the closes aren't one transaction).
  • A stop that a connection holds no row for closes only rows started before that connection began, so a late stop from an older connection can't close a newer connection's avatar. An instance with no Keyframe key still closes, at the hang-up and the call's close, rows another instance opened. A start whose mint finishes after the hang-up sends no grant and opens no row.
  • voice-session-record.ts (small model change): owned() and detached() now also return the voice row's id, so a re-attached connection and the sweep can name the call's avatar rows.

Tests

apps/web/tests/voice-avatar.test.ts runs the real service over PGlite (openHostedStoreTestDatabase) with a fake OpenAI and a test Layer on Keyframe:

  • start → avatar.started with the requestId echoed, and a row opened; nothing reaches OpenAI; no log line carries the token
  • no key → not_configured; NO_CAPACITY → no_capacity; PLAN_REFUSED / UPSTREAM → error; no row is opened in any of these cases
  • a start while one is held closes the old row (user) and mints again; a late stop closes nothing; the current request's stop closes its row (server_closed)
  • a hang-up closes the rows as call_ended
  • the orphan sweep closes an orphan's avatar rows as orphaned

voice-frames.test.ts checks that avatar asks are read and never forwarded, and that the service's own answers are refused when the device sends them. ./scripts/check.sh exits 0 locally.

Not verified

  • CI could not verify this; Mac evidence pending. A run.sh --preview on a Mac, confirming voice calls are unchanged, is still owed. This Linux VM can't produce it.
  • No live Keyframe session was ever opened. Untested against the real service: the real POST /v1/sessions answer shape and statuses (503 / 403 / 429), whether a real grant's server_url is wss: (if it isn't, every start answers error), and the seconds a row records compared with what Keyframe bills.
  • Preview and Production need KEYFRAME_API_KEY and LUKE_KEYFRAME_PERSONA_ID set in Vercel. Until then, every start answers not_configured.

Consumers

No desktop sends avatar.start yet. The host's frames are #8, and the wiring is #11. The README voice section is left to #12.

🤖 Generated with Claude Code


Open workspace in Conductor


Devin Review


Summary by cubic

The voice service now answers avatar.start and avatar.stop asks itself on Luke's Keyframe key instead of forwarding them to OpenAI, and keeps one row per avatar session.

  • A start closes whatever avatar rows the call still holds, mints a fresh grant, opens a voice_avatar_sessions row, and echoes the request id in avatar.started; the grant and token never reach the logs, and a grant minted after the device hung up is never sent.
  • Deployments missing KEYFRAME_API_KEY or LUKE_KEYFRAME_PERSONA_ID answer not_configured; Keyframe's NO_CAPACITY becomes no_capacity, and PLAN_REFUSED/UPSTREAM become error.
  • Rows close as user on a newer start, with the stop's own reason on a stop, as call_ended on hang-up or voice-row close (voice row written first), and as orphaned via the orphan sweep.
  • A stop holding no row closes rows started at or before this connection's own instant, so an instance with no Keyframe key still closes, at the call's close, the avatar another instance put up.
  • The orphan sweep also repairs avatar rows left open on calls already closed, closing them as of that call's own close.
  • The relay refuses malformed avatar asks, and the device can never send the service's own answers.