fix(backend): prevent large list GETs from exhausting the 30s request budget

Merged
GI
2mo ago
+2297-227

Summary

GET /v1/action-items, GET /v1/conversations, and GET /v3/memories could each drive aggregate Firestore work past HTTP_GET_TIMEOUT (30s) and come back as a bare middleware 504 on large accounts (#11831). This PR makes those three reads budget-bound: they either land, or return an honest partial page with an explicit truncation surface.

Closes #11831

Design

  • Request-scoped budget — utils/other/list_budget.py owns a monotonic internal deadline (24s default: the 30s middleware cutoff minus serialization headroom), a 25k-document allowance, aggregate counters, and a typed ListReadBudgetExhausted. Created once per request at the route from the TimeoutMiddleware-stamped request start (request.state.omi_request_started_monotonic), so pre-handler time (auth, middleware) counts. TimeoutMiddleware still owns the hard 30s cutoff — unchanged.
  • Wall clock interrupts the blocking RPC — every Firestore stream() / get_all() / document.get() on these routes receives timeout=budget.rpc_timeout() derived from the remaining internal deadline. A blocked RPC raises DeadlineExceeded at the internal deadline, leaving ≥6s headroom for serialization; only that exception (and allowance exhaustion) maps to the typed outcome. Loop-side monotonic() checks alone cannot stop a blocked next() — the RPC timeout is what does.
  • Charge actual aggregate work — conversations charge the server-side offset() prefix before the query (Firestore bills and streams every skipped row); action-items share one budget across the active, legacy-harvest, and completed queries; memories charge both dual-window index scans plus emitted and suppressed rows, the canonical stream, status batches, and hydration.
  • Fallback shares the budget — the /v3/memories first-page scan-budget 503 (Memory scan budget exceeded) still falls back to the legacy offset read(), now on the same request budget; it can never start a fresh unbudgeted 5000-row window.
  • Non-breaking truncation surface — conversations and memories keep their bare JSON arrays (Flutter/macOS/Windows decode arrays; no envelope migration) and instead set the documented X-Omi-List-Truncated: true response header. Action-items gains an optional truncated: bool field on ActionItemsResponse and the same header; on exhaustion has_more is forced true so a missing lookahead can never report "complete". A truncated memory page carries no X-Omi-Memory-Next-Cursor, because its cursor state would not cover every consumed scan position.
  • Error semantics preserved — non-budget Firestore/auth/validation errors keep their current statuses (503/4xx); nothing becomes a catch-all HTTP 200 partial.
  • Telemetry — list_read_requests_total{route,outcome}, list_read_documents_total{route}, list_read_seconds{route}. Labels carry only route/outcome — no uid, query, payload, or raw URL.

HTTP defaults are untouched (action-items 50/500, conversations 100/1000 with unbounded offset, memories 100/500 with the 5000 compatibility window intact).

Tests

New backend/tests/unit/test_list_read_budget_contract.py (34 tests):

  • Fake-clock deadline/allowance unit tests incl. off-by-one exhaustion and first-reason-wins; blocked-RPC fake proving the internal deadline fires with ≥6s headroom under the 30s cutoff; middleware-stamp propagation.
  • Action items: one shared budget across active+legacy+completed; deadline cut mid-bucket returns a partial page + truncated; non-budget NotFound keeps raising.
  • Conversations: 1k/5k/20k-row large-offset fixtures served and charged (docs_scanned == offset + limit); offset beyond the allowance truncates with no stream issued; mid-page deadline cut keeps an honest newest-first prefix; budgeted/unbudgeted parity.
  • Memories: parent-budget exhaustion mid-merge returns a partial page with no cursor; complete budgeted page keeps the cursor header; scan-budget 503 → offset fallback provably reuses the same budget object; dual-window charging.
  • Route-level header/field assertions for all three routes.

Existing suites updated for the new call kwargs (budget=...) and the optional action-items truncated field: pagination order, router pagination, bounded list reads, memories pagination clamp, universal cursor/service, offset read cost, archive/read contracts, egress contract, atom keyword index, desktop migration hermetic stubs.

Product invariants affected

  • INV-INT-1: Integrations must not claim a complete export from a truncated response.
  • INV-MEM-3: Universal memory authority fails closed. The budget never
    bypasses canonical authority — typed list-read exhaustion returns a partial
    page of already-verified rows or an explicitly truncated empty page, and
    non-budget canonical/readiness failures keep their fail-closed 503s. The
    historical adapter stays a read-only input; the offset fallback runs under
    the same request budget and the same canonical suppression checks.
  • INV-MEM-4: Memory promotion authority unchanged. This PR bounds read
    work only; it does not alter intake, consolidation, promotion, or
    projection authority.
  • INV-MEM-1: Exactly three product memory tiers. The budget only bounds
    read work; it does not add tiers, collections, or archive exposure, and
    include_archive gating is threaded through unchanged.

Failure-Class: FC-bounded-read-exceeds-request-budget

Line-Count-Exception: backend/database/conversations.py | 1735 -> 1759 | Budget parameter, pre-query offset charging, and budgeted partial-page accumulation threaded into the existing no-photo list helper; no new responsibilities.
Line-Count-Exception: backend/routers/conversations.py | 1659 -> 1676 | Route creates the request-scoped budget and sets the documented X-Omi-List-Truncated header on the unchanged bare-array response.
Line-Count-Exception: backend/utils/memory/canonical_memory_adapter.py | 2593 -> 2606 | Budget threading through the canonical scan page and lineage point-reads so their RPCs run under the derived per-RPC timeout.
Line-Count-Exception: backend/utils/memory/memory_service.py | 2774 -> 2891 | Scan sub-budget derives from the shared request budget, typed-exhaustion re-raises before each broad 503 wrapper, and read/read_page return explicitly truncated partial pages without cursors.

Line-Count-Exception: desktop/macos/Desktop/Sources/MainWindow/Pages/MemoriesPage.swift | 3576 -> 3606 | Truncation-aware pagination and sync/reconcile guards added to the memory list view model.
Line-Count-Exception: desktop/macos/Desktop/Sources/MemoryExportService.swift | 1604 -> 1610 | Reject truncated memory pages during full export so exports cannot be mistaken for complete.