fix(host-service): isolate usage scans in a capped worker and read agent logs incrementally

KI
1h ago
+2769-409

Summary

  • Usage scans run in a dedicated worker with a 1.5 GB heap cap and no main-thread fallback, so a scan that runs out of memory fails its request instead of host-service.
  • Agent usage is read incrementally: SQLite collectors read only token fields and only new or changed rows; transcript logs resume where the last scan stopped.
  • Scan state is cached on disk, so a restarted host does not rescan everything (11.5 s cold → 0.3 s on a heavy machine).

Why / Context

A user report on 2.0.5: host-service ran out of memory (V8 heap ~3.7 GB) about 6 s after every start, so the host kept disconnecting until the app stopped restarting it. The Devin collector loaded the full text of every assistant message from a 7.3 GB sessions.db with .all(). The startup leaderboard publish triggers that scan.

Two things turned one large file into a crash loop:

  1. Host workers had no heap limit, so an OOM in a worker was fatal for the whole process.
  2. A crashed worker task was retried inline on the main thread with the same input.

How It Works

Isolation (workers/):

  • HostWorkerPool gains resourceLimits and noInlineFallback: { pauseAfterCrashMs }.
  • usage-worker-pool.ts creates one usage worker with a 1.5 GB heap cap. It never runs a task inline; after a crash it pauses for 10 minutes.
  • usage.history and usage.leaderboardPayload use this pool. The shared host-worker pool is unchanged.

Incremental reads (usage/history/):

  • SQLite collectors project only token fields in SQL and stream rows.
    • Devin and Copilot read rows past a rowid watermark (sqlite-tail.ts).
    • OpenCode re-reads rows by time_updated.
    • Hermes is reused until its database or WAL changes.
  • Line-based logs (Claude, Codex, Pi, Muse, agy, fx, Grok) use resumable parsers whose state is plain JSON (line-cache.ts). A file that grew is read from its saved byte offset. A file that shrank or was replaced is read again.
  • usage-index.ts keeps a 92-day snapshot in the usage worker.
    • A refresh runs in the background and outlives the request that started it.
    • A request waits up to 60 s, then gets the last snapshot. On a cold start with no snapshot yet, it gets a retryable error.
    • Cursor is fetched per request, because it attributes entries with the caller's workspace paths.
  • usage-store.ts saves parser state and watermarks to $SUPERSET_HOME_DIR/usage/usage-cache.db.
    • It writes at most every 5 minutes, and right after the first scan.
    • Rows are written only at the end of a scan, so any saved row is a consistent point to resume from.
    • The cache is keyed by the host-service version, so each release starts a new one. An unreadable file is deleted and rebuilt.

Manual QA

All runs used the dev desktop app driven over CDP with real clicks, while a probe sampled renderer long tasks, host event-loop delay (health.check every 100 ms) and Electron-main round trips.

  • Cold scan of a real 90-day history (~30B tokens; Claude, Codex, a 2.3 GB OpenCode DB). The page loaded in 12.3 s; host loop p99 46 ms; renderer longest task 58 ms; Electron main max 1 ms.
  • Warm 30d / 90d / 7d switches loaded in 0.5 / 3.1 / 0.5 s, with host loop max 3–12 ms and no renderer long tasks.
  • A 2 GB synthetic Devin DB like the report's: the cold scan took 14 s with no renderer long tasks. Devin's total was exact (230,000 tokens). host-service RSS was about 477 MB, including the worker.
  • Host-service restarted mid-scan: the in-flight request failed with a network error, not a hang. The next host computed correct totals, and no partial cache was left behind.
  • Under Electron with the production bundle, the pre-fix scan code on the 2 GB DB hit ERR_WORKER_OUT_OF_MEMORY. The request failed cleanly, the pool paused, and the host process stayed alive.
  • On real data, the uncached scan, a cold scan writing the cache, and a restart from the cache give identical per-agent counts, tokens and session labels.

Testing

  • bun test src/trpc/router/usage src/workers src/no-main-loop-blocking.test.ts: 191 pass.
  • node --import tsx --test …/usage-store.node-test.ts …/usage-sqlite.node-test.ts: 9 pass. Both run in CI through test:integration:terminal.
  • New tests cover:
    • resuming an appended file, and resuming after a restart from JSON state;
    • a replaced, shrunk or in-place rewritten file; a saved state the parser rejects; an unterminated last line;
    • single-flight refresh and the bounded wait;
    • the no-inline-fallback pool;
    • the store's version reset, corrupt-file rebuild, leaving a locked file alone, eviction and write throttle;
    • Hermes scan reuse, a mid-read failure not double-counting, dedupe-key pruning, and a Muse resume matching a fresh parse.
  • I confirmed the key tests fail when the behaviour they cover is removed.
  • bun run typecheck: no errors in host-service.
  • bun run lint is clean.
  • The real-data comparison was re-run after the review fixes: the uncached scan, a cold cache and a restart give identical per-agent counts, tokens and session labels.

Design Decisions

  • A worker cap, not just smaller reads: reads are now small, but the cap and the no-inline rule mean the next unexpected input fails one request instead of the host.
  • Cache on disk, not only in memory: with a memory-only cache, every host restart (the moment the crash happened) is a full rescan, and an idle-reaped worker loses it.
  • Parser state as plain JSON: a parser can be rebuilt from saved state and resume mid-file.

Known Limitations

  • The first scan after a release, or on a new machine, is still a full read (in the background, with flat memory).
  • On plain Node (not Electron), a worker heap cap does not contain every allocation pattern. Electron, which production uses, did.
  • The old OpenCode JSON-file format is still read in full each refresh (each file is capped at 32 MB).
  • The store's Node test sits in test:integration:terminal, the only Node suite CI runs. A dedicated task would need turbo.jsonc and ci.yml changes.

Follow-ups

  • Move session-share off the main thread. Its first share of a large profile does synchronous renames at boot and on account switch. I'm doing this in a separate PR.
  • .claude.json is parsed whole on the main thread for quota and logins (about 100 ms for a 100 MB file).

Risks / Rollout

  • Risk: the usage-history numbers come from new parsing paths. These were checked against the old code on real data and matched exactly.
  • Risk: the new cache file under $SUPERSET_HOME_DIR/usage/, about 50 MB for a heavy user.
  • Rollback: revert this PR. The cache file is ignored by older builds and can be deleted safely.

Summary by cubic

Fixes the host-service crash loop where a usage scan could run out of memory and take down the whole process (~3.7 GB heap, ~6 s after boot, triggered by the Devin collector loading every message text from a 7.3 GB sessions.db with .all()). Usage scans now run in their own worker capped at 1.5 GB with no inline fallback, so a scan that runs out of memory fails its request and pauses the pool for 10 minutes instead of killing the host. A worker that crashes while idle pauses the pool the same way, queued work is rejected until a replacement runs, and the crash is reported to Sentry.

Incremental reads

  • SQLite collectors project only token fields and read only new or changed rows (rowid watermarks for Devin and Copilot, time_updated for OpenCode, file signature for Hermes). Devin and Copilot rows are read in 2,000-row batches with yields; batches end on existing rowids so a sparse table loops no empty ranges, and malformed rows are skipped via a json_valid guard. OpenCode projects time and tokens with -> so a string value skips one message instead of aborting the whole read.
  • Line-based log parsers are resumable; a file that grew is read from its saved byte offset, and a line a parser throws on is skipped, not the rest of the file.
  • Scan state is cached in $SUPERSET_HOME_DIR/usage/usage-cache.db, so a restarted host resumes instead of rescanning (11.5 s cold → 0.3 s warm on a heavy machine). The cache dir is owner-only because titles come from prompts.
  • The usage worker keeps a 92-day index refreshed in the background; requests wait up to 60 s, then get the last snapshot, and requests for the same window share a scan. Parser state is pruned past the window — every 50,000 lines while reading — so unbound logs like Grok's unified.jsonl stay small, and a file is read again when the cutoff moves earlier than its saved prune.
  • Cursor has one in-flight collection per window and workspace set, shared and reused across requests and capped at 20 s so a hung keychain lookup cannot retire the warm index. A failed collection is dropped, not reused, so the next request tries again.
  • The cache is deleted only when SQLite reports corruption; a locked file falls back to memory, failed reads count as misses, and failed saves are logged. A file replaced mid-read is never saved under its new inode.
  • Hermes dropped its fingerprint cache, whose signature could miss same-size writes; its resume tests run in WAL mode to match how Hermes writes state.db.

Rollout

  • First scan after a release or on a new machine is still a full background read, with flat memory.
  • A cold index or a paused pool surfaces as a retryable TIMEOUT, which the Usage page retries every 30 s; SERVICE_UNAVAILABLE is reserved for an offline host and stays with the connection retry policy. A missing worker bundle is an internal error reported to Sentry.
  • The cache file is ~50 MB for heavy users, keyed by host-service version, ignored by older builds, and safe to delete.
  • Session titles are read only for sessions that have entries; session cwd is read whole, re-applied each scan, and a cleared cwd replaces the cached one.
  • On plain Node a worker heap cap does not contain every allocation pattern; on Electron (what production runs) it does.

Summary by CodeRabbit

  • New Features
    • Usage history and leaderboard data now use cached, incremental scans, which can reduce repeated processing and improve responsiveness.
  • Bug Fixes
    • Usage collection handles changing, incomplete, or unreadable files and databases more reliably.
    • Host scans that time out or are temporarily unavailable are retried automatically.
    • Worker failures return errors instead of falling back to main-thread processing.
  • Changes
    • Cursor network usage is no longer included in usage history.