#6901 feat: NER-based speaker identification with spaCy (#3039)

What this does

Replaces the regex-only speaker name detection with NER (Named Entity Recognition) using spaCy, while keeping regex as a graceful fallback.

Fixes #3039

Why

The current regex approach only catches explicit patterns like "My name is John" or "I'm Sarah." It completely misses natural mentions:

• "Tell Mike I'll be late" → missed by regex ✗, caught by NER ✓
• "Sarah mentioned the meeting" → missed by regex ✗, caught by NER ✓
• "Hey David, how's it going?" → missed by regex ✗, caught by NER ✓

How it works

1. Primary: spaCy NER (en_core_web_sm) extracts PERSON entities from transcript segments
2. Scoring: Each candidate gets a confidence score based on context clues, position, and frequency
3. Filtering: False positives ("Omi", "Google", day names, etc.) are filtered out
4. Fallback: If spaCy isn't installed, original regex patterns still work

Technical details

• Model: en_core_web_sm — MIT license, ~15MB, < 5ms per segment on CPU
• Lazy loading: Model loads on first call, not at import time (no startup penalty)
• Zero breaking changes: detect_speaker_from_text() signature unchanged
• Performance: < 10ms per segment (tested, see unit tests)

What's included

File	Purpose
utils/ner_speaker_identification.py	New NER module with confidence scoring
utils/speaker_identification.py	Updated to use NER as primary, regex as fallback
tests/unit/test_ner_speaker_identification.py	9 unit tests covering detection, filtering, perf
requirements.txt	Added spacy>=3.7.0

Setup

pip install spacy
python -m spacy download en_core_web_sm

For Dockerfile, add to the build step:

RUN python -m spacy download en_core_web_sm

Testing

cd backend && python -m pytest tests/unit/test_ner_speaker_identification.py -v

AI Tools Disclosure

Used AI pair-programming (Claude) to design the confidence scoring system. All detection logic and test cases manually reviewed.