#9419 perf(db): cut hot-path database work and close copilot cleanup gaps

Summary

Cuts Sim's heaviest database work and closes cleanup gaps. Each change was checked against production query statistics and plans first; reads return the same results as before.

Billing

• New usage_log_billing_entity_period_cost_idx (migration 0441) leads with both period stamps and carries the summed columns, so the billing-period sums and weekly refresh become index-only; it replaces two indexes (the old covering index lost to the slim one on column order and a misestimate)
• Member removal revoked sessions (an organization row write) before locking the org's workspaces, the reverse of the storage writers' workspace-then-organization order, which deadlocked; revocation now runs after the handoff holds the workspace rows

Hot reads

• The deployed-state read withholds the state blob when this process already caches that version (one round trip either way)
• Audit-log lists read each scope branch through its keyset index and merge, instead of an OR the planner could only seq-scan; the cursor is a row comparison
• Table-row mutations fetch only their workspace's table triggers instead of every workspace's

Files and knowledge

• Workspace list invalidation broadcasts coalesce per tab while a refetch is in flight, so an upload burst no longer refetches the full list once per upload per viewer
• The file-search backfill locks only the files it seeds, not every row it walks (its owner trigger was locking each one)
• Completed-listing reconciliation counts window by window under the lease, so a large connector's count can't hit the statement timeout

Cleanup

• Hard-deleting a copilot chat records its copilot backend and stored-file purge in the same transaction as the delete (outbox event, retried until it succeeds)
• Account removal records the same purge for the chats it cascades away inside its transaction; admin user removal queues it, delayed, before the delete and refuses the removal if it can't
• Outbox retention prunes completed checkpoint-expiry, file, memory and chat cleanup events; events read back by id are kept
• Idempotency cleanup drops four full-table stat scans and resumes each batch where the last ended instead of rewalking from the oldest key

Schema (migration 0440)

• Drops 32 indexes that a valid, unpredicated b-tree leads with the same columns (never unique or constraint-backing); checked against index usage on the primary and both replicas
• Vacuums embedding, workspace_files and copilot_async_tool_calls at 2% dead/inserted instead of 20%

Type of Change

• Bug fix
• Other: performance

Testing

• Unit: 20 apps/sim files (333 tests) plus packages/audit rollup tests
• Integration: 43 files (661 pass, 2 skipped) against a freshly migrated database (0440 and 0441 applied) and a private Redis, including new suites for the lock order, deployed-state reads, audit-log scope paging, table triggers, chat backend cleanup, outbox retention, idempotency cleanup, file-search backfill and windowed counts; each new regression test fails on the pre-fix code
• 0441 replay verified: a valid index is kept, an invalid one is moved aside and rebuilt, an unexpected valid recovery index refuses
• bun run check:migrations origin/staging, drizzle-kit generate (no changes), schema mock, Phase A generators (no changes), bun run lint, block registry, bun run check:audits (61), bun run type-check
• The full bun run test runs in CI

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing (new tests pass the test-audit authoring gate)
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)