perf: drain delivery backlog and copy backups without compaction

Merged
JC
6d ago
+1029-102

[!TIP]
PR walkthrough: Open the visual walkthrough

Large delivery backlogs were throttled by per-batch and per-pass sleeps, while upgrade backups rebuilt the entire SQLite database. Cleanup now runs continuously while it makes progress, and backup-only operations use incremental native page copying while retaining existing verification gates.

  • Small transactions, cancellation, maintenance admission, idle waiting and error backoff remain in place. Eligibility and recovery proofs are unchanged.
  • Explicit physical compaction still uses VACUUM INTO; page-copy backups preserve free pages.

Validation

  • Race tests passed for db, persistence, backups, database, maintenance, inventoryrepair and deliveryretention.
  • Historical sweep/protected-gap regressions and persistence store conformance passed.
  • Targeted golangci-lint: 0 issues; SQL guard passed.
  • Documentation, specification, harness and PR coverage checks passed.
  • New regressions reproduced rebuilding and premature waiting before the fix. Fixtures are small; no million-row test.
  • CI remediation: worker integration fixtures passed under the race detector; recovery browser failures reproduced, then all 4 affected desktop tests and the mobile recovery test passed with retries disabled.

Checklist

  • If I do not have repository write access and this is a large architectural change, I discussed the direction in a linked issue before opening this PR.
  • This PR contains one logical change; unrelated work is split into separate PRs.
  • I have performed a self-review of my code.
  • I have manually tested my changes and they work as expected.
  • My changes have tests that cover the new functionality and edge cases.
  • If my change touches UI files (apps/web/), I have added or updated Playwright e2e tests in apps/web/e2e/ and verified them with make test-e2e.
  • I checked whether this affects public docs in docs/public/** and updated them or noted why no docs change is needed.