Add Meta Muse as a transcription and intelligence provider
Summary
Problem: A Meta Model API key only worked for Intelligence through the generic Custom entry. Transcription's Custom entry assumes a Deepgram-compatible /listen WebSocket, so Muse Voice Transcribe could not be used at all: Meta's ASR API is its own protocol (in-band auth handshake, raw PCM frames, typed turn events), not Deepgram- or OpenAI-shaped.
Fix: Adds a named Meta Muse provider on both tabs. Transcription gets a new owhisper-client adapter for Meta's realtime WebSocket and batch endpoint, wired through Provider, AdapterKind, BatchProvider, the listener and proxy dispatch sites, and the desktop catalog. Intelligence gets an OpenAI-compatible catalog entry that reuses the generic listing, key verification, and client paths, with muse-spark-1.3 ordered first.
What changed
- Live transcription (
crates/owhisper-client/src/adapter/meta/live.rs): connects towss://api.meta.ai/v1/asr/realtimeinDIARIZATIONmode, sends the key in the handshake frame (the endpoint ignores HTTP headers), streams 16 kHz s16 PCM, and maps events to stream responses: cumulativetranscriptevents become partials scoped to the open turn,speechCompletebecomes the final, andspeakerlabels attach to the open turn. Selected languages are sent aslanguageBias(English names, 25 documented languages); keywords are forwarded. - Batch transcription (
meta/batch.rs, behind thelocalfeature): multipartPOST /v1/asr/transcribe. Meta only accepts mono s16 WAV at 16/24 kHz, so segments are decoded and re-encoded withhound(new optional dependency in thelocalfeature). Upload limit is 590 s per request under Meta's 10-minute cap. - Intelligence (
apps/desktop/src/settings/ai/llm/shared.tsx): catalog entry with the existing Meta logo and fixed base URLhttps://api.meta.ai/v1. No lister case is needed: the generic keyword filter already dropsmuse-voice-transcribe-1.0andmuse-image-1.0, and onemodelPriorityPatternsentry putsmuse-spark-1.3first so it becomes the default. - Desktop transcription settings: "Meta Muse" entry with the single model, shown as "Muse Voice Transcribe"; batch mapping and default model; regenerated
plugins/transcription/js/bindings.gen.ts. - Provider suggestion registries: the live registry moves from the Tauri command into
listener2-corebeside the batch registry so both are plain functions with tests; both now include Meta. - Docs: one line each for the Transcription and Intelligence lists in
docs/models-and-providers.mdx.
How the live event mapping works
Observed against the real API, the order at a turn boundary is speaker → speechEnd → speechComplete → next speechStart, and partials reset at each speechStart. The adapter keeps per-turn state (start_ms, end_ms, speaker index) keyed by turnId so speechComplete can arrive after the next turn has started. Meta returns turn-level timing only, so words are spread evenly across the turn, following the existing Mistral adapter.
Known limits
- Partial transcripts carry the previous turn's speaker label until the turn completes; Meta emits the label at turn end.
- Meta's 60-minute session cap closes the socket; the existing listener supervisor reconnect handles it, no adapter-level retry.
- Direct BYOK only; no Anarlog Cloud proxy routing for Meta.
Verification
cargo test --locked -p owhisper-client --features localand without the featurecargo test --locked -p listener-core,-p listener2-core --lib,-p transcribe-proxy --libcargo check --locked -p api,-p mobile-bridge,-p tauri-plugin-transcriptioncargo clippy --locked -p owhisper-client --features local -p listener-core -p listener2-core -p transcribe-proxy -p tauri-plugin-transcription --all-targets --no-deps -- -D warnings(no new findings)cargo test -p tauri-plugin-transcription export_types(bindings regenerated, diff limited to theBatchProviderunion)pnpm -F @anlg/ui build,pnpm -F @anlg/desktop typecheck,pnpm -F @anlg/desktop test,pnpm exec oxlint --quiet --format=github apps/desktop/src/,pnpm -F @anlg/provider-validation testrustfmt --edition 2024 --checkon changed Rust files;dprint checkon changed TS/MD files; license-boundary scripts andnode --testfromci.yaml- Live API with a Meta key:
cargo test -p owhisper-client --features local meta -- --include-ignored(single and dual sockets, batch file), plusGET /v1/modelsand amuse-spark-1.3chat completion - Manual on macOS with a local stable-config build: Meta Muse selected on both tabs, live transcription with speaker labels, re-transcribe after recording, Intelligence model list shows the five Spark models with 1.3 preselected, summary generation
[!NOTE]
Medium Risk
New third-party STT/LLM integration touches live capture, batch re-transcription, and API key handling; batch audio re-encoding and turn-level timing mapping could affect transcript quality or edge-case failures.
Overview
Adds Meta Muse as a first-class bring-your-own-key provider on Intelligence and Transcription, instead of relying on Custom endpoints that cannot speak Meta’s ASR protocol.
On the desktop, Intelligence gets a Meta Muse catalog entry (https://api.meta.ai/v1, API key) with branding via ProviderBrandImage, plus listing tests and a priority pattern so muse-spark-1.3 sorts first while voice/image Muse models stay filtered out. Transcription adds the same provider with muse-voice-transcribe-1.0 (“Muse Voice Transcribe”), default model wiring, batch provider mapping, and updated provider-order tests.
The Rust stack introduces MetaAdapter in owhisper-client: live WebSocket at /v1/asr/realtime (in-handshake auth, diarization mode, turn/speaker event mapping, interpolated word timing) and batch POST /v1/asr/transcribe (mono WAV re-encode via optional hound on the local feature, ~590s segment cap). AdapterKind / Provider / BatchProvider, listener adapters, direct batch routing, language suggestion (new suggest_providers_for_languages_live in listener2-core, Tauri command delegates there), and transcribe-proxy BYOK exclusions are all extended for meta.
Docs note Muse Spark for Intelligence and Muse Voice Transcribe for STT; plugin BatchProvider bindings include meta.
Reviewed by Cursor Bugbot for commit 1823255fdcde917e55372b3278c8b289fdbf9886. Bugbot is set up for automated code reviews on this repo. Configure here.