feat(spf): render MoQ audio through the @caprid/moq-media worklet core
Closes #66
Summary
The relay SDKs are now on npm as @caprid/*, so this makes the AudioWorklet player core from @caprid/moq-media the only MoQ audio renderer and deletes the AudioBufferSourceNode path. The latency controller's rate nudges now reach the core, which was the last reason the core was opt-in.
Changes
@caprid/moq-media^0.1.0is now a dependency of@videojs/spf.@caprid/*is added tominimumReleaseAgeExclude, because 0.1.0 was published the day before.- The worklet renderer replaces
audio-renderer.tsand keeps thecreateAudioRendererActorname. The source-node renderer and its tests are removed. playoutRatereaches the core as the input'splaybackRate, clamped to the 0.5–2 range the core accepts. A new input starts at the current rate. On each change the clock first saves its position at the old speed, so it always moves at the rate in effect. The output-latency term is converted to media time at the current rate.- With no factory given, the renderer builds the core on the render context's
baseContext. Without one, the renderer goes to anerrorstatus. The engine'screateAudioPlayeroption stays, but only as an override. AudioContextLikedropscreateBuffer/createBufferSource, and so do the adapter's render context and the test fakes.- The renderer's local Opus duration helper is replaced by the package's
opusPacketDurationUs. Comments that referred to@mux/moq-mediaor "the relay SDKs" now name the@capridpackages.
Testing
pnpm -F @videojs/spf test: 2414 passed.pnpm typecheck, format andpnpm check:workspacepass. Thesimple-moq-videohtml tests pass.- New unit tests:
- the rate reaches the input, and the clock speeds up only after the change (100 ms, then 1.05× → 205 ms);
- rates outside 0.5–2 are clamped, and a new input starts at the current rate;
- the default factory loads the worklet onto
baseContext, and goes to an error status without one.
- A one-off check in headless Chromium (not committed) used the real core, a real
AudioContext, and 6 s of Opus encoded with WebCodecs:- first audio after 41 ms;
- clock speed 1.003× the context clock at rate 1, and 1.056× after a 1.05 nudge;
- 0 underruns, and frames were metered from the source buffer rather than pushed all at once.
- Bundle (
moqentry, esbuild, minified): 102.9 → 111.6 KB, 33.1 → 36.5 KB gzipped.pnpm -F @videojs/spf sizeis broken onmmcc/moq(it callspnpm build, which the Vite+ migration removed), so I measured with esbuild instead.
Follow-up, not in this PR: the dry and hold detection could read the core's dryAtUs and playStarts stats instead of working them out from timestamps.
🤖 Generated with Claude Code
https://claude.ai/code/session_01DaH4ihf1LLPqmXShYkvBat
Summary by cubic
Makes @caprid/moq-media's AudioWorklet player core the only MoQ audio renderer, deleting the AudioBufferSourceNode path, and sends the latency controller's playout rate into the core, which was the last reason the worklet renderer was opt-in.
- Adds
@caprid/moq-media^0.1.0 as a dependency and@caprid/*tominimumReleaseAgeExclude. - The worklet renderer replaces
audio-renderer.ts; the source-node renderer and its tests are removed. playoutRatereaches the core asplaybackRate, clamped to the 0.5–2 range, and the clock rebases on each change so it advances at the rate in effect.- With no override given, the renderer builds the core on the render context's
baseContext; without one it goes to an error status. The engine'screateAudioPlayeroption stays as an override. - Packets resuming past a re-join gap now start a fresh fill instead of arriving one per stall rescue, which froze the clock for up to the core's jitter ceiling.
AudioContextLikedropscreateBuffer/createBufferSource, and the local Opus duration helper is replaced by the package'sopusPacketDurationUs.