feat(sync): sync health watchdog and stall alerts (#15501)
What does this PR do?
Re-implementation of community PR #17825 by @TheRWX (approved, but conflicted and unmoved since 2026-10-05) on today's sync code and the rulings that landed since. Fixes #15501: Android users silently lost days of pendant recordings with no "device not syncing" warning; others only sync when they open the app.
A new SyncHealthWatchdog (app/lib/services/sync_health_watchdog.dart) watches the paired pendant's pipeline:
- Activity signals. BLE audio bytes arriving, a WAL accepted by the server (
onWalSynced— every upload path, manual and drain), a batch recording finalized, and the pendant (re)connecting. Any of them proves recordings are moving — audio is then safe on the phone or on the server, and the #23679/#23822 drains (oldest-first, Retry-After, quiet refusals) own recovery. Only silence across every signal is a stall the user must hear about. Refusal-only stretches stay quiet by design: a refusal with live ingestion is not data loss, and #23822 deliberately keeps refusals out of user attention. - Stall threshold: 24 hours. Longer than the longest normal daily gap — a pendant taken off and charged overnight (~8–12h silent) — and far inside the reported 4-day silent losses, with #23679's 30-day server backfill window still far away. (The original 4h threshold would alert every morning when quiet hours end.)
- Alert cadence. At most one reminder per day (24h cooldown), never 22:00–08:00 local (quiet hours do not spend the cooldown), notification id 15501, tap lands on Device settings (
/settings/device, the capture-recovery push precedent). The silence clock seeds on first run, so the app update itself cannot alarm. - Not a disconnect push (#3328). No per-event notification; onboard storage keeps recording across BLE drops. This alert fires only after a full day of zero flow — the state where users actually lost data. Acute stalls stay the wedge monitor's job in-app (connected-no-bytes within minutes, upload silence at 2h with a backlog); this is the out-of-app last resort.
- Scope. Paired pendant types only (
omi,friendPendant); watches and glasses sync on their own cadence. Ships on by default with a kill switch ("Sync Health Alerts", Device Settings).
Surfacing, per #22119 (capture/sync problems speak on the live capture page and the Activity bell, never Home):
- Activity → Needs attention: "Omi isn't syncing · No recordings synced for over N hours" (cloud-off row, opens Offline Sync; ordered after the upload row as the broadest, slowest condition). Fed through
CaptureWedgeMonitor.observeSyncSilence, so the bell's unseen-dot machinery is reused unchanged. - Device Settings: the "Sync Health Alerts" toggle sits under the device group; its subtitle switches to the live stall duration while stalled.
- The original PR's Home indicator is deliberately not re-implemented.
The watchdog's periodic loop starts in app bootstrap (main.dart, after prefs init); DeviceProvider wires each evaluation into the wedge monitor and exposes isSyncStalled/syncHealth. Timer ownership stays out of the provider so provider tests never hold a pending timer.
Renders (headless visual-audit captures, synthetic state)
Activity — the new sync stall row:

Device Settings — the alert toggle with the stall subtitle:

Activity — the existing capture rows, unchanged by this PR:

Product invariants affected
- INV-DATA-1
- INV-CAP-1
INV-DATA-1 applies because app/lib/main.dart is a protected data-plane routing authority; this PR only starts the watchdog's periodic timer there after prefs init — no routing, endpoint, build define, or bundled env value changes.
INV-CAP-1: touches app/lib/services/capture/capture_wedge_monitor.dart only to add a slow-burn CaptureSyncSilence row (declared and ended solely by the watchdog's evaluation) to Needs attention and the bell's unseen logic. Ingress, retention and admission behavior are unchanged; the new row makes no saved-audio claim.
How it was verified
flutter test(full hermeticbash test.shCI suite): 7921 passed / 9 skipped; the single failure during the run was this PR's own new visual-audit scenario before a fixture fix, green on re-run (flutter test test/visual_audit/visual_audit_smoke_test.dart: 158 passed).- New suites:
test/unit/sync_health_watchdog_test.dart(device gating, 24h threshold arithmetic, daily cooldown, quiet hours, toggle, seeding, throttle incl. no clock rewind, timer lifecycle) and new groups intest/services/capture/capture_wedge_monitor_test.dart(sync-silence row + bell) andtest/widgets/needs_attention_capture_test.dart(row render, absence, Offline Sync tap, ordering behind the upload row). make mobile-verify ARGS="fast --all": 31/31 journeys passed (full-suite fallback selected — cross-cuttingapp/libchange).flutter analyze: no new findings (pre-existing environment-only errors in flavor-generated files aside, whichtest.shbootstrap generates).python3 app/scripts/l10n.py check: ok, 49 locales; 6 new keys via the tool.check_mobile_ux_contract.pyon the two touched UI files: clean.tool/perf_shell.py --check: PASS (shell regenerated for the main.dart import).- Renders above:
app/scripts/visual_audit.sh --head WORKTREE --only activity-sync-stall,device-sync-stall-alerts,activity-capture-attention.
Not verified / limitations
- The watchdog evaluates only while the app process runs (the capture foreground service keeps it alive on Android). If the OS kills the process, no alert can fire until it runs again — the same boundary the original PR had.
- The Activity row's hour count refreshes on watchdog ticks (30 min), not per minute.
- No on-device run: all verification is hermetic (widget tests, journeys, headless renders).
Line-Count-Exception: app/lib/backend/preferences.dart | 1612 -> 1630 | three persisted sync-health keys: alert toggle, last activity timestamp, last alert timestamp
Line-Count-Exception: app/lib/providers/device_provider.dart | 1624 -> 1663 | watchdog wiring, stall getters, connect/disconnect hooks
Line-Count-Exception: app/lib/services/capture/capture_controller.dart | 4602 -> 4609 | record sync-activity on BLE audio and batch finalization
Failure-Class: none
Closes #15501
Re-implements #17825 (original by @TheRWX, co-authored there)