PR #23822 - BasedHardware/omi

Problem and behavior

Deterministic upload refusals created repeated user Retry prompts even though the same bytes and date were resubmitted. Unreconciled historical uploaded handles also stretched Catching up across weeks.

Retain refused audio, persist closed reason/status and daily retry timestamps, retry for seven days, then count as reported. Keep refusals outside user attention and the Failed filter, preserve explicit storage clearing, and remove per-row Retry. Expose count, seconds, reported count and user-action capability for the UI lane.
Accept empty/torn-header framed clips as a 200 no-op; preserve tiny encoded speech for decoding and VAD. Return bounded multipart/filename refusal codes.
Reconcile completed/missing jobs through the existing custody-safe poller, track authoritative pending confirmations, and bound the displayed range to the latest contiguous pending run on one day.
Register WAL Upload Refused telemetry with reason and HTTP status, plus an executable consumer.
UI (David's 10-27 choices). Catching up, option C: Device Settings drops the Right now catching-up card. The third stat tile reads "Catching up", the pending duration and "on its way ›" ("when online ›" offline), and opens On the Way; it is On Device again when nothing is pending. No stage words, date ranges or titles. On the Way, option B: " catching up", a thin bar for audio already on the server, one line, then one card with a row per day (count · duration, and Uploading / Next / Queued / Transcribing following the oldest-first upload order). Failed and refused recordings are never rows and there is no per-row Try Again or Delete; a footnote says they recover in the background, and "Details for support" opens a plain list of every recording with its reason tokens. A single action line appears only when refusalSummary.userActionCanHelp (currently always false).

Root cause and evidence

Code tracing and local HTTP tests identify 400 paths in multipart parsing (missing boundary/header, malformed boundaries, part count/size), filename validation, and capture-time admission. Empty audio is currently accepted past staging but fails decode in a server job; this is distinct from admission HTTP 400. Short valid frames are not evidence of silence. Authentication/BYOK uses 401/403, throttling uses 429, the recovery window uses 422, backfill resource limits use 413, and duplicate/language-routing admission uses 409 or a legacy fallback. Those paths do not explain an untyped admission 400.

David-scoped metadata read with ro-prod showed invalid-capture-time admission decisions at 15:50:15Z and 15:50:18Z on October 27, followed by valid legacy backfill decisions (including 15:57:30Z). These iOS requests identified themselves in headers as 1.0.578+1490, not the reported TestFlight 1.0.579, and have no trace IDs. The 16 reported Cloud Run 400 requests at 15:57–15:58Z therefore remain unattributed, and their exact refusal reason is not proven. No other users' logs, transcript or audio content were read, and no signed-in API was called.

Product invariants affected

INV-CAP-1: captured audio remains intact across refusal, backoff and missing-job reconciliation. Only a framed file with no audio payload is acknowledged as a no-op; even one tiny encoded frame remains retained for decoding/VAD.

Validation

bin/omi-heavy run -- flutter test --no-pub <77 sync/WAL test files>: 874 tests passed.
BACKEND_UNIT_TEST_FILE_LIST=... bin/omi-heavy run -- bash backend/test.sh: 949 tests passed across 22 files (760 sync tests plus 189 Cloud Tasks admission tests), including real local FastAPI multipart/empty/mixed-manifest/refusal requests with synthetic auth and storage boundaries.
Focused typed refusal decoder/recovery rerun after extraction: 46 Flutter tests passed.
On the Way widget policy: 7 tests passed, covering quiet retained refusals, transient recovery actions, grouped recordings and uploading-member deletion protection. The full CI suite exposed these old refusal-action expectations; all six were migrated to the requested policy.
bash scripts/analyze_ratchet.sh from app: passed without changing the analyzer baseline; the full 874-test sync suite passed again after the analyzer and filter follow-up.
PYRIGHT_PYTHON=.venv/bin/python bin/omi-heavy run -- bash scripts/typecheck.sh: zero errors (existing repository warnings remain).
Multipart regression verifies that a malformed second part closes the spool file already opened for the first part. Task-admission fixtures now contain one complete framed payload so they exercise admission rather than the new empty-clip no-op.
python3 scripts/check_event_registry.py: 200 registered events and executable consumers valid.
Screenshot harness: 3 widget tests passed; both tracked PNGs visually inspected. The harness loads fixture fonts and uses an iPhone viewport; these are synthetic renders, not a physical-device acceptance receipt.
scripts/pr-preflight --all --pr-body-file .agent-briefs/pr-body.md: all 48 gates passed, zero failures and zero checks skipped for time (438.8 seconds).
OMI_PR_BODY_FILE=.agent-briefs/pr-body.md bin/omi-heavy run -- make preflight: 47 checks passed (183.4 seconds). The bounded pre-push gate also passed, including backend type checking, selected tests and Flutter generated-file validation.

UI lane:

Widget tests touched by the UI: device_catching_up_test, episodes_ui_test and sync_indicator_isolation_test together 56 passed (the Device page still rebuilds 0 times on progress ticks and reads the inventory once per sync update), on_the_way_days_test 11, on_the_way_attention_test 6, home_pending_row_test 41, sync_provider_background_admission_test 6, sync_attention_evidence_test 6. They cover the tile at 200% text with a ≥44pt target and button semantics, rows and the support link at 3.1× text, and no Try Again or Delete on either view.
Full app suite flutter test --no-pub test/: 7,836 passed, 9 skipped, 1 failed: a visual-audit scenario whose duration finder also matched the device header's chip. It is now scoped to the tile, and the device-catching-up and on-the-way audit scenarios pass.
scripts/pr-preflight --pr-body-file .agent-briefs/pr-body.md: 48 checks passed (536.9 s), including ui-evidence, mobile-ux-contract, brand-ui, mobile-ui-isolate and the ARB contract.
New strings: English, then omi-l10n-fill for 48 locales (5 batches, 0 retried cells, review applied 13 fixes); python3 scripts/l10n.py check passes for 49 locales. The four On the Way summary keys with no callers left were removed.

No merge or deploy. No physical iPhone/TestFlight acceptance or production fix verification. PostHog delivery and ingestion keep-list remain operationally unverified.

Copy note: uploads have no Wi-Fi-only gate in the app, so the tile caption and header line name the real condition (a connection) instead of "on Wi-Fi".

Home fold ("N short clips hidden · Show") is not built. Noise conversations are served as discarded, the Home list is fetched with include_discarded=false, and the list response carries no hidden count, so the client cannot know how many were hidden on a day. It needs either a per-day hidden count from the server or an always-include fetch filtered on the phone (which changes paging); both are outside this UI lane.

Integration points

SyncProvider.refusalSummary provides count, seconds, reportedCount and userActionCanHelp. Refused files remain in the full inventory. deviceCatchUpLocal and DeviceCatchUpSummary supply the bounded range; reconciliation renews jobPendingConfirmedAt only on an authoritative pending reply. The UI reads these contracts and does not change them: the tile adds pendingSeconds to the memoized deviceCatchUpLocal snapshot and a seconds total to DeviceCatchUpSummary; the bounded span and target fields remain but are no longer displayed.

The multipart helper remains an HTTP body parser with no Firestore access. Malformed-body spool cleanup belongs to the parser subclass, leaving the existing reviewed inventory checksum unchanged. Refusal parsing, retry reservation and admission helpers were split out to keep all three large files at or below their original line counts. No line-count exception or gate bypass is used.

UI evidence

Synthetic Flutter widget renders of the real pages with injected fakes, 390×844 iOS viewport, fixture fonts. The blank band above the Device Settings tiles is the device image, which the test engine does not paint. The previous two renders showed the removed card and rows.

Failure-Class: new

Review follow-up (GLM)

GLM's review of this PR blocked on one item, now fixed in fix(sync): cap refusal recovery per drain. A server-wide 400 could refuse thousands of recordings, and their daily recovery would then fire every single-file resend in one drain: a 400 engages neither the 429 limiter nor the 5xx pause. One drain now resends at most 25 (walMaxRefusalRetriesPerDrain), and the rest wait for a later drain. too_large and too_many_parts refusals are never resent automatically. A unit test covers both.

GLM's second finding said exhausted transient failures get no prompt on On the Way. That is intentionally not changed. David picked option B ("problems out of sight"), and those recordings are re-armed automatically when connectivity returns.

Line-Count-Exception: app/lib/services/wals/local_wal_sync.dart | 2653 -> 2654 | the per-drain refusal cap must sit at the drain's candidate filter; the file already shrank in this PR

🤖 Generated with Claude Code