Add browser-side audio response timing diagnostics

CH
3m ago
+1570-11

Summary

Add opt-in browser-side audio response timing diagnostics to help developers investigate the delay between the interviewer's speaking state and the first detectable remote audio activity.

The diagnostic is disabled by default and does not change the candidate-facing interface or avatar playback behavior.

Changes

  • Add web/audio-response-timing.js to measure the browser-observed delay using Web Audio analysis.
  • Enable the diagnostic only when ?audioTiming=1 is present in the URL.
  • Start timing when LiveKit reports lk.agent.state=speaking and report the elapsed time when audio activity crosses the configured detection threshold.
  • Cancel incomplete measurements when audio monitoring is interrupted or the connection becomes unavailable.
  • Handle audio track changes, reconnections, and browser audio context lifecycle events.
  • Log timing results to the developer console without modifying the interview UI.

Validation

  • 65 relevant browser tests passed.
  • ESLint and formatting checks passed.
  • Git pre-commit and pre-push checks passed.
  • Local interactive review completed.

Limitations

  • Timing is browser-observed and does not represent end-to-end model or server latency.
  • Actual LiveKit audio timing has not yet been verified in a live interview.
  • A review identified a remaining P2 test gap: reconnect coverage should exercise the registered room callbacks directly.

Close #712


Summary by cubic

Adds an opt-in, browser-side diagnostic that measures the delay between the interviewer's speaking state and the first detectable remote audio activity, helping developers investigate response timing without changing the candidate-facing interface or avatar playback. It is disabled by default and only activates when the URL contains ?audioTiming=1.

  • Starts timing when LiveKit reports lk.agent.state=speaking and only reports after a quiet frame is followed by audio above a configured threshold, so pre-existing audio never inflates the gap.
  • Cancels incomplete measurements on interruption, page hide, connection drops, and track changes, and ignores stale room events that could release a newer monitor.
  • Reuses the shared startMediaMeter loop, which now supports configurable sampling, a deferred first frame, and explicit cleanup.
  • Logs results to the developer console; timing is browser-observed and not end-to-end model or server latency.

Recovery coverage

  • Added integration tests confirming that monitor setup failures and interviewer unavailability clear stale timing state and allow a later speaking turn to be measured.