Voice: a dropped device socket detaches the call instead of ending it
Why
In prod, voice calls end at about 300s (Vercel cuts the silent socket) and at 800s (the function's maximum duration). On the sessions route, the relay's onDeviceGone treated any end of the device's socket as a hang-up and sent session.close to OpenAI. That ended the WebRTC call even though the Mac re-attaches (reattachingSocket). In the logs the re-attach shows up as session-attached followed straight away by session-ended, because it lands on a session that is already closing.
The Mac's real hang-up already sends an explicit session.close over the socket (graceful-close.ts), and the relay forwards it. So a hang-up and a dropped socket can be told apart on the server.
What
relay.ts- The relay sets
deviceAskedClosewhen the device forwardsLIVE_CLIENT_EVENT.CLOSE, and setsrefusedByPolicyinrefuse. - When the device socket ends on
VOICE_ROUTE.SESSIONSwith neither flag set, the relay sends nothing upstream and settles at once with the newFINALIZATION.DETACHED(log.ts). The existing tail then closes only the sideband. - Everything else ends as it did before: the audio and introduction routes, refusals, real hang-ups, and a device that asked to close and then dropped.
- The relay sets
LiveSessionService.release()doesstop()'s bookkeeping (stops run events, clears the queue, tears down the standing session, writes pending rows, releases the transport) and sends nosession.close. Astop()after it does nothing.live-exchange.tsaddsEXCHANGE_ENDING(CLOSE/DETACH) andHostedLiveExchange.endAs(ending). The scope finalizer callsrelease()onDETACHandstop()otherwise.service.ts- The service passes the ending in before it closes the exchange scope:
DETACHwhen the relay settledDETACHED. - On the still-starting path (the device left while the exchange was standing), it passes
DETACHonly for a sessions-route re-attach (opened.started). A fresh creation the device never heard an answer for is still closed.
- The service passes the ending in before it closes the exchange scope:
- A detached relay calls no
record.close, so usage stays unconfirmed. The connection that later seessession.closedrecords the seconds, and the ledger dedupes them. - Drain:
#drainnow detaches sessions-route calls. I updated its comment, the relay and service header comments, and the README paragraphs on the service scope, the exchange finalizers, "One connection is one invocation", "How a session ends", and the writer's re-attach note.
The bound on a detached session
Security review flagged that, without a bound, a signed-in client could create a session, drop the socket, and keep its WebRTC up. Nothing would ever send session.close, so the session would run on Luke's key with no host watchdog. This PR adds the bound:
- Migration 0052 adds a nullable
detached_attovoice_sessions, plus a partial index over open, detached rows.voice-schema.tsdeclares both. - Stamp and clear. When the relay settles
DETACHED, the service stampsdetached_aton the row, but only whileclosed_atis null. The same stamp happens on the still-starting detach path. A successful re-attach (openAttached) clears it. The writes are newVoiceSessionRecordmethods:detach,attached,detached, andcloseLost. - The sweep lives in
server/voice/orphan-sweep.tsand runs on/api/observation/tickright after the speech push.- It takes open rows stamped more than
VOICE_DETACH_GRACE_MS(60s) ago, oldest first, at most 20 a tick, ten at a time. - For each row it attaches a fresh sideband through the existing
LiveUpstream, sendssession.closethrough the existingcloseGracefully(now exported from@sidecar/voice/live-session), and onsession.closedwritesrecord.closeandrecordVoiceSecondsthe way the service does. The seconds are recorded once through the ledger. - The attach and the wait for
session.closedare each bounded at 5s. Worst case, the sweep takes two rounds of 10s out of the tick's 50s budget. - If the attach is refused (the session is already gone at OpenAI), or the session never answers, the row is closed as
connection_lostand its last unconfirmed snapshot stands, so it is never swept again. - A deployment without
OPENAI_API_KEYskips the sweep, the same kill switch as elsewhere. The tick's answer gainsvoice: { closed, lost, failed }.
- It takes open rows stamped more than
- Function externals: the tick now imports the voice upstream, so
ws(bufferutil,utf-8-validate) and the Node built-ins it reaches are new externals ofobservation-tick.js, and also ofdefault.js(the grouped dispatcher that mounts the observation routes). I regeneratedserver/function-externals.jsonwithpnpm --filter @luke/web functions:externals. These are the same externals the voice functions already load. - README: "One connection is one invocation", "How a session ends", and "Scheduled Conductor observation" now describe the sweep.
- For your approval: the CLAUDE.md scheduled-pass paragraph may want a sentence about this sweep, since it is the tick's one write that reaches OpenAI. I left CLAUDE.md untouched.
Tests
- The hang-up helpers now send
session.closebefore closing, through the new sharedhangUpDeviceintests/support/voice-fakes.ts:hangUp,hangUpConnection, the two-connection loop, and the two hang-up tests invoice-service.test.ts. One assertion moved fromframesToUpstream: 0to1, because the hang-up's own close is now that one frame. - Changed to the new behavior:
- The service close/drain test now expects a detach: no CLOSE upstream,
finalization: "detached", no record close. - The re-attach test's
terminate()no longer expects a CLOSE. - The "seconds recorded once across a re-attach" test now sends the hang-up before
terminate(), so it still checks the ledger dedupe.
- The service close/drain test now expects a detach: no CLOSE upstream,
- New:
voice-service.test.ts: a terminated sessions-route socket sends no CLOSE, logsdetached, and records nothing. Asession.attachon a fresh connection resumes the session, and a later real hang-up records the seconds once.voice-service-exchange.test.ts: a planning call dropped mid-call detaches, re-attaches, and its next ask (dl_2) lands in the plan's conversation.live-session-service.test.ts:release()sends nothing upstream, writes the words said so far, and astop()after it closes nothing.voice-orphan-sweep.test.ts(new store test on PGlite, with the fake OpenAI): a detach stamps the row and a re-attach clears it. The sweep ends only the row detached past the grace and never re-attached, throughsession.close. It records the close and 42 seconds once, and a later sweep attaches nothing and records nothing more. A second test refuses the attach (404): the row is closed asconnection_lost, its unconfirmed snapshot stands, no seconds are recorded, and it is not swept again.- The detach tests in
voice-service.test.ts(fake record) andvoice-service-exchange.test.ts(real record) now also check that the row is stamped on the drop and cleared on the re-attach. hosted-observation-tick.test.tschecks that the tick runs the sweep at its own instant and carriesvoicein its answer.
- With the
relay.tschange reverted, the three new or changed relay tests fail. - The refusal, introduction, and audio-route tests are unchanged and pass.
./scripts/check.shexits 0.test:storewas not run here because this sandbox has no Postgres. CI's postgres job covers it, including the newvoice-orphan-sweep.test.ts.
This is a server-only change. CI runs no Mac build and verify.sh was not run, but this PR changes no Mac code.
Known limits
- Events in the 1ā10s reconnect gap are lost on each drop.
- A delegation in flight when the socket drops is lost, because the new connection gets a fresh exchange. Fixing this means resuming in-flight asks from the store on re-attach, or a make-before-break handoff.
- The Mac-side ping that stops the 300s cut is a separate PR.
š¤ Generated with Claude Code