PR #2474 - ReviewStage/luke

Plan item 4. desktop: PCM tee and turn gate (video calls with Luke). Independent PR, base main.

What it does

Worklet (avatar-feed/pcm-tee-worklet.ts): its own esbuild entry, built as renderer/pcm-tee-worklet.js (430 gzipped bytes, with its own bundle-budget.json line) and served from beside voice.html. Worklet modules fall under script-src 'self', so the CSP does not change. It turns Float32 into PCM16LE, clamped (a NaN reads as silence), and posts 960-sample (40 ms) chunks with the buffer transferred.
Resampling goes through a 24 kHz AudioContext, with no filter in the worklet. Spike 0 Q7 found that Chromium accepts the 48 kHz remote track in a 24 kHz context and resamples it cleanly: Pearson 0.9916 against a decimated 48 kHz copy, lag 0. A context that will not open at 24 kHz leaves the feed dark and logs a warning.
The <audio> caveat (from the spike) is written into the code: the remote stream must stay on a playing <audio> element (muted is fine), or every context reading the track gets zeros. #10 has to keep the element playing while it mutes it.
Turn gate (avatar-feed/turn-gate.ts): a pure function over chunks. It counts samples and arms no timers.
- Any 10 ms frame (240 samples) whose RMS is ≥ the threshold opens a turn at its chunk, and up to 200 ms (4,800 samples) of the quiet chunks just before it go out first as pre-roll.
- Once 19,200 quiet samples have run in a row, the chunk where the quiet completes goes out, followed by turn_end.
- If the scope closes while a turn is open, a final turn_end is sent.
Public API: startAvatarFeed({stream, onFeed}): Effect<void, never, Scope> (avatar-feed/start-avatar-feed.ts). The context, the nodes and an open turn are released with the scope.

Gate constants (Spike 0)

Measured on TTS through a Chrome Opus loopback, not on GPT Live. They must be re-checked on a real GPT Live capture on a Mac.

	Value	Basis
RMS threshold	0.001 (-60 dBFS) per 10 ms frame	Noise floor after Opus: -109 dBFS (DTX off) or -99 dBFS (DTX on). Opus tail is under 1e-4 within 40 ms. Speech frame RMS p10 is 2.1e-3.
Quiet N	0.8 s = 19,200 samples	The avatar renders in 1 s blocks, so N up to 1 s adds no lag the block wouldn't add anyway. A pause longer than N splits a reply into two avatar turns, which the SDK handles.
Pre-roll	200 ms = 4,800 samples	Plan

The feed type

avatar-feed/feed.ts is a local copy of #1's pinned AvatarFeed shape (audio with base64 PCM16LE 24 kHz mono in 40 ms chunks, turn_end, interrupt). #10 swaps it for #1's schema.

No caller yet

startAvatarFeed and the worklet have no caller until #10 (the voice window's feed path), which runs it from use-voice-session.ts and stacks on this branch. Until then both are listed as knip entries, as #2443 did for its tile, and #10 removes those lines.

Tests

turn-gate.test.ts drives the gate with synthetic chunks:
- nothing goes out over the Opus noise floor;
- p10 speech opens a turn, and so does one loud 10 ms frame;
- the pre-roll includes the five chunks just before the turn, oldest first;
- samples come back intact as PCM16LE, decoded independently of the encoder;
- turn_end comes after the quiet, but not after a pause one chunk shorter;
- closing the gate mid-turn sends turn_end.
pcm.test.ts:
- 40 ms chunks are cut across 128-sample render quanta;
- the Int16 clamp holds past full scale;
- NaN reads as silence.

What CI could not verify

CI could not verify this; Mac evidence is pending. CI builds nothing for the Mac.

The worklet loading under voice.html's policy, the zero-output node being pulled, and the live chunk cadence are untested on a real Mac.
The gate constants were measured on TTS through Opus, not on GPT Live.
Untested against a live Keyframe session: the user did not allow a live session, so Spike 0 ran with no key. Nothing here has fed sendAudio or endAudioTurn, and how the avatar handles the 40 ms chunk cadence and a reply split into two turns comes from reading the SDK, not from a session.

./scripts/check.sh exits 0.

🤖 Generated with Claude Code

Open workspace in Conductor

Summary by cubic

Adds the avatar feed's PCM tee and turn gate, routing Luke's remote track into 40 ms PCM16 chunks gated into turns for the avatar.

startAvatarFeed({stream, onFeed}) holds the context, worklet, and nodes in the caller's scope.
The turn gate opens on a 10 ms frame at -60 dBFS, sends 200 ms of pre-roll, and ends with turn_end after 0.8 s of quiet, counted in samples. When a chunk's early frames complete the quiet but its last frames speak again, the gate ends the old turn before that chunk, which then opens the next.
The worklet ships as its own bundle from beside voice.html; a 24 kHz context resamples the 48 kHz track.
The remote stream must stay on a playing <audio> element or contexts read zeros.

Caveats

Gate constants were measured on TTS through Opus, not GPT Live, and need re-checking on a Mac with a live capture.
CI could not verify worklet loading or chunk cadence, and nothing has fed a live Keyframe session.
No caller yet; #10 wires the feed into use-voice-session.ts and swaps the local feed type for #1's schema.