feat(backend): persist bounded per-speaker voice-match scores (prod on)

Merged
GI
10d ago
+1792-16

Persist bounded per-speaker voice-match evidence on existing conversation writes for future offline relevance evaluation.

Capture snapshots the existing arbitrated owner/person cosine comparisons in session memory; sync carries its existing pooled comparisons through internal segment dumps; resolution captures its existing 0.50 pooled-voice comparisons under resolved speaker IDs. Existing conversation writes aggregate a speaker_match_scores blob; transient per-segment score payloads are removed before storage. The in-memory Conversation holds it before relevance runs. Jev inputs, verdicts, thresholds and speaker assignments are unchanged.

Rows record stage-local speaker ID/scope, owner distance, nearest non-owner person distance/ID, nearest-to-runner-up margin, embedded evidence seconds, decision/status, accepted ID, schema version and actual operating thresholds. Floats round to 3 decimals and unavailable/nonfinite values become null. Retain 16 rows per stage (48 total); scopes are capped at 128 characters. The stored field is additionally capped at 8,192 bytes including compression/encryption; worst-case added Firestore storage is 8,214 bytes, Document protobuf field growth is 8,224 bytes, and the complete CommitRequest protobuf body grows by at most 8,249 bytes including the update mask and length-prefix boundaries. Tail snapshots are trimmed as needed. Existing write snapshots suppress new scores and delete an already-stored score blob above the 900 KiB document budget, reclaiming the optional space with no extra reads. Separate stages preserve capture/sync evidence when resolution renumbers voices. Missing fields on old conversations remain absent. No backfill.

Optional score collection, generation, transfer, decode/merge/serialization/compression/encryption and size-estimation errors emit shared fallback telemetry while speaker decisions and content writes continue. Partial retention reports scores_trimmed; scores_omitted means nothing new is stored. Every Conversation constructor, model_validate and factory entry drops raw or malformed optional score values before typed validation. Manual assignment returns its scores through the same validated UID-aware decode path, preventing post-commit assignment/rejection 500s; undecodable evidence is omitted. Model serialization and embedding-cache metadata also isolate optional faults. Manual capture decisions retain the existing comparison with the actual manual outcome. Processing transactions decode and merge the current stored scores ahead of stale capture/sync snapshot rows, replacing resolution rows with the processing result and re-encoding at the current transaction protection level. Failed optional merges leave the current stored blob intact. Sync bridges, manual conversation merges and Jev smart absorbs aggregate decoded donor rows before the existing codec/write. Unscoped donor keys receive conversation provenance so equal resolution IDs remain distinct. Malformed decoded shapes are dropped before typed Conversation construction. Resolution rows sum constituent embedded seconds and retain minimum per-candidate distances after identity-cluster merges; merged/scored counts and distance/margin aggregation tags distinguish this evidence from a fresh pooled comparison. The decision and margin still describe the original representative comparison. Manual rejection records no accepted identity and the manual_rejected outcome. Protection migrations delete old optional scores when re-encryption fails, preventing plaintext retention at enhanced protection. Partial text edits, manual assignments and translation materialization run the capacity guard against their already-read stored snapshot. Successful resolution with no comparisons clears stale resolution rows, including in the processing transaction: explicit empty score snapshots survive encoding, and explicit resolution metadata authorizes stage replacement. Score-only size growth is suppressed before commit; ordinary content/Firestore failures keep their existing semantics.

No new collections, Firestore operations, model calls or audio replay. Resolution adds actual embedded PCM seconds to its existing encrypted embedding-cache header without changing the cache's transcript-duration validation; older cache entries report null evidence seconds. Tests verify reuse makes no additional embedding calls.

Privacy: although speaker_resolution is coarse plaintext metadata, distances linked to enrolled people are biometric-adjacent. Use the existing compressed JSON conversation codec with per-user encryption at enhanced protection, compressed bytes at standard protection, and re-encode with existing protection-level migration writes. This follows manual-speaker receipt protection rather than weakening it to plaintext metadata. Fields are SkipJsonSchema/excluded and available only in internal Python-mode dumps; JSON responses, live client projections, dict views and integration/webhook payloads omit them. The conversation-field trust pin explicitly classifies this as server-derived numeric evidence, not an untrusted display projection.

Rollout: prod ON at merge+deploy on GKE backend-listen/pusher and Cloud Run backend, backend-sync, backend-sync-backfill and backend-integration. All six explicitly pin SPEAKER_MATCH_SCORES_ENABLED=true in the production overlay; both production charts mirror it. The manifest and registry are regenerated. Backend-integration shares the main image and its developer processing routes; desktop-backend and jobs do not compute or persist this evidence. Rollback = flag off on all six hosts. Code defaults remain dev/local/offline on and unset prod off. No deployment is performed by this lane.

Product invariants affected

  • INV-MEM-4: runtime manifests add only internal speaker evidence persistence; canonical Long-term promotion authority and memory rollout controls remain unchanged.

Failure-Class: none
Reason: Optional encoded evidence was reaching typed response validation after a manual mutation committed. This change adds a shared fail-open model boundary and existing-owner decode/persistence guards; reviewed failure classes cover index maintenance, closed job keys and explicit-null persistence, rather than this optional conversation-evidence codec/model boundary.

Validation

Round 4 is 73c79f1dcb (raw-model and partial-write guards) and 789a2c547f (empty-resolution persistence), then merged with origin/main after #20737 (feature-flags conflict kept both flags, registry regenerated). The score file passed 115 cases; it includes real FastAPI HTTP requests through the assignment and rejection handlers with valid and corrupt persisted score blobs at both protection levels, constructor/model_validate/factory fail-open cases, all three partial-write paths, and successful zero-comparison resolution persisted with/without other-stage rows. Initial fixture setup/body errors were corrected before this passing run.

Final-source affected validation uses 95 file-isolated files, four workers, not integration and not slow, and CI's 1.0-second CPU guard. Final affected run: 2,519 passed, 4 skipped, 5 deselected, exit 0, with no CPU-guard failures. The single final preflight passed all 45 checks in 223.75 seconds, exit 0. No preflight gates are unsatisfied. No entire backend unit suite is run; CI owns that validation. The enforced final-source typecheck passed: 0 errors, 13,004 warnings, exit 0. Google credentials are stripped and the test PATH omits kubectl/gcloud. The audit inventory contains 51 matching production call/definition lines, including MCP/chat projections that do not declare scores; the field-level before validator covers their factory and direct model entry points without uid-dependent decryption in models.

Generated clients

The stale TypeScript/Swift clients on main were regenerated separately in #20737 (merged). This PR's numeric fields remain SkipJsonSchema/excluded and change no generated client.

Existing owner integration

Line-Count-Exception: backend/database/conversations.py | 3512 -> 3657 | Small integration hooks at existing persistence/session owners; numeric summary policy is extracted into a pure bounded helper rather than duplicating it.

Affected test command (Google credentials stripped; test PATH excludes kubectl/gcloud):

TASK_TEST_PATH=$(cat /private/tmp/speaker-match-scores-test-path.txt)
env -u GOOGLE_APPLICATION_CREDENTIALS -u GOOGLE_CLOUD_PROJECT -u GCLOUD_PROJECT -u SERVICE_ACCOUNT_JSON -u FIREBASE_AUTH_CREDENTIALS_PATH -u GCP_SERVICE_ACCOUNT_JSON -u CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE -u GOOGLE_IMPERSONATE_SERVICE_ACCOUNT PATH="$TASK_TEST_PATH" BACKEND_UNIT_TEST_FILE_LIST=/private/tmp/speaker-match-scores-round3-affected-files.txt BACKEND_PYTEST_WORKERS=4 BACKEND_FAST_UNIT_FAIL_SECONDS=1.0 BACKEND_PYTEST_FILE_ISOLATION=1 BACKEND_PYTEST_MARK_EXPR='not integration and not slow' bash backend/test.sh

Validation log: /private/tmp/speaker-match-scores-round3-affected.log. Preflight log: /private/tmp/speaker-match-scores-round3-preflight.log.

Raw-model audit: /private/tmp/speaker-match-scores-round3-raw-model-audit.txt.

🤖 Generated with Claude Code