feat(batch): bound serving sweep page size and tick

Merged
GI
5h ago
+209-5

Serving discovery is fixed at 100 rows per page and the supervisor sleeps 20 seconds, so operators cannot tune discovery supply after claim contention falls. The recorded lead2 D2 gate is in /Users/dazheng/workspace/omi/data/run-2026-11-05-omi-pipeline/posttrain-gate-lever-train.txt; runtime performance is not remeasured by this code-only PR.

Adds call-time bounded config accessors using the existing numeric helper:

  • BATCH_PIPELINE_SERVING_SWEEP_PAGE_ROWS: default 100, clamp 50–200, malformed → 100. The assignment and finish discovery pages share this cap; the database query helper ceiling increases to 200 while its default stays 100; explicit smaller sweep limits remain supported. The shared Projector.sweep() accessor also applies when the dedicated pump calls it; pump-specific settings are unchanged.
  • BATCH_PIPELINE_SERVING_TICK_SECONDS: default 20 seconds, clamp 10–60, malformed → 20. The combined page rows/tick budget is at most 20 rows/s per serving instance. Over-budget requested pairs raise the effective tick to at least page_rows/20 without reducing pages, and log one WARNING with the effective values. Only the serving supervisor sleep changes; dedicated pump cadence remains independent.

Registers both settings in the pipeline flag budget and source registry, regenerating the registry documentation. Defaults preserve existing behavior. Claims, striping, owner-drain capacity, pump settings, follow-up paging, record-drain and recovery-dispatch loops are unchanged.

Validation for the B1 correction:

  • Affected backend tests through BACKEND_UNIT_TEST_FILE_LIST + bash test.sh, including stage-env ratchet, flag budget and Firestore query guards: 1,298 passed, 1 deselected across 18 files; 59 serving-config/runtime tests passed. make preflight: all 42 checks passed.
  • No additional review round; the single-round prod-safety blocker is addressed inline.

Risk and ramp plan (owner: lead2): backend-integration cohort only, via env in a separately authorized rollout. Step 1 sets page rows to 200 and tick to 20 seconds (2x today's 100/20 discovery rate). Step 2 keeps page 200 and sets tick to 10 seconds (4x today), only if Firestore reads remain within +50% of the pre-ramp rate and claim_busy <40% plus owner_drain rows/s guards hold for 30 minutes. Measure document reads using the same Cloud Monitoring Firestore document-read-rate series before and during each step; watch request p50 for regression. Hold or roll back on guard failure. Rollback: unset both env values to restore 100 rows and 20 seconds. The page setting also affects dedicated pump sweeps through shared Projector.sweep() (correctness follow-up); the serving supervisor budget does not govern the pump's separate cadence. No production access or live performance validation was performed. This PR does not merge or deploy anything.

Product invariants affected

none

Failure-Class: none

Line-Count-Exception: backend/utils/batch_pipeline/projection.py | 1702 -> 1704 | Two-line net growth wires the bounded config accessor into the existing sweep without unrelated projection refactoring.