fix(server): compact superseded history in the background

Merged
JU
5d ago
+376-16

The server never deleted superseded orchestration history, so databases grow without bound. One maintainer database reached 114 GB, most of it repeated whole-payload updates of the same thread, message, node or PR link.

The server now runs compactEventStore one minute after startup and every six hours after that. Each pass deletes events whose state a newer retained event already carries:

  • Whole-thread events (rename, archive, settle, pin, model changes…): only the newest per thread is kept. Visit and mark-unread events supersede only each other, so a later visit never removes the rename that carries the thread's activity time.
  • Messages, nodes and PR links: the oldest and newest event per entity are kept. Clients add an entity where they first see it, so keeping the first event preserves message order for a client that reconnects from an older cursor.
  • thread.created, thread.deleted, runs and turn items are never compacted.

No replay boundary or snapshot fallback is needed: every deleted row has a newer row for the same key, so any replay from an older cursor still ends in the same state and order.

New databases are created with auto_vacuum = INCREMENTAL, and the server returns freed pages to disk online in small yielding batches. Existing databases get that with t3 doctor (#19436). Compaction pages are 50 rows, because a 500-row page with large payloads blocked the shared connection for ~90 ms.

A retried MCP metadata update whose events were compacted now answers from the current thread instead of failing.

Takes over #19249.

šŸ¤– Generated with Claude Code

Fixes #18559


Devin Review