perf(host-service,desktop): move hot git reads off the event loops, add ratchet enforcement

Merged
KI
4mo ago
+1381-390

Problem

Both single-threaded event loops — host-service (per org) and Electron main — serve all tRPC traffic, and recent changes kept adding in-process git reads to them. One slow spawn (or an inline network fetch) head-of-line blocks every response on that loop, which is why the app feels unresponsive under load. Worker pools exist on both sides; coverage was the gap — only 3 host-service tasks and 2 desktop tasks were worker-routed, everything else ran on the loop.

Profiling the live app confirmed the mechanism: ~830ms of raw spawn self-time per minute on host-service at light watcher churn, scaling linearly with git-call volume.

Changes

Host-service

  • #5913 regression: workspace-create's inline git fetch (any upstream base) now runs via the existing gitFetchBaseRefTask in the worker pool, coalesced per repo+ref. The await stays — forking from a fresh tip was the point — but the spawn/drain leaves the loop.
  • New git/readWorkspaceRefs worker task: the PR-sync loop's 5–8 spawns × N workspaces (every git-watcher event + 5-min sweep) collapse into one off-loop task per workspace. Helpers moved verbatim to runtime/pull-requests/utils/workspace-refs.ts so the worker bundle doesn't pull in the runtime's DB/Octokit graph.
  • ctx.git() env resolution: the git remote get-url origin spawn is TTL-cached per repo (~30 call sites paid it per call, one inside project.remove's loop). Credentials stay uncached — the provider remains authoritative.
  • base-ref freshness (#5776): the rev-parse --git-common-dir spawn that ran before the fetch-TTL check on every status poll is now TTL-cached per worktree path. A stale entry can only mis-key the dedupe (one extra/suppressed fetch, TTL-bounded) — the fetch itself always runs in the worktree path.
  • Recursive rmSync rollbacks in resolve-repo.ts / project/handlers.ts → async rm (deleting an entire failed clone held the loop for the whole walk).

Desktop

  • changes.getBranches (~6 serialized spawns, polled every 10s per inactive pane + per hovered sidebar row), workspaces.getAheadBehind (rev-list per sidebar row), and both file-contents procedures (cat-file -s + git show pairs) now run in the changes git worker. Response shapes are byte-identical; the localDb reads stay on main and cross as payload data, mirroring the existing getStatus pattern.

Enforcement — how we keep it this way
Two ratchet tests fail CI on regressions:

  • packages/host-service/src/no-main-loop-blocking.test.ts
  • apps/desktop/src/no-main-process-blocking.test.ts

Each bans sync subprocesses, recursive sync fs, and in-process git-client construction outside the worker dirs. Current legacy offenders are allowlisted; any new matching file fails with advice pointing at the worker-task pattern, and an allowlisted file that gets fixed also fails until its entry is deleted — the lists only shrink. Remaining migration backlog with priorities: plans/off-loop-git-reads.md.

Verification

  • Unit/integration: host-service 920 pass (1 pre-existing pty-daemon flake, passes in isolation); desktop worker-handler tests 12/12 over real temp repos, mutation-checked (5 deliberate impl mutations each failed exactly their corresponding test); both ratchet tests verified to fail in both directions. Typecheck + bun run lint clean.
  • CDP end-to-end on the dev app built from this branch: getBranches / getAheadBehind / getGitFileContents driven through the real renderer→IPC→worker path against a registered worktree, cross-checked against direct git (branch name, 1488 local refs, 2642-byte base blob — all exact matches). Real-click navigation into a v2 workspace populated the Changes panel via live host-service git.getStatus through the worker pool, reporting exactly this branch's own diff (23 files +1308 −389). Worker bundles confirmed loaded, zero inline-fallback warnings.

Follow-up idea (not in this PR): add event-loop-delay p99 to the resource_snapshot telemetry so future loop-blocking regressions surface in PostHog instead of user reports.


Summary by cubic

Moves hot git reads and network fetches off the host-service and Electron main event loops into worker pools, and adds ratchet tests to prevent new blocking call sites. This reduces UI stalls and keeps tRPC responsive under load.

  • Refactors

    • Host-service: workspace-create base fetch now runs via gitFetchBaseRefTask (coalesced per repo+ref); PR sync reads branch/HEAD/upstream via git/readWorkspaceRefs; ctx.git() env resolution TTL-caches and coalesces git remote get-url origin; base-ref freshness TTL-caches --git-common-dir; recursive rmSync rollbacks replaced with guarded async rm.
    • Desktop: moved changes.getBranches, workspaces.getAheadBehind, and changes.getFileContents to the changes git worker via runGitTask; hardened dedupe keys (JSON-encoded paths for file-contents; per-worktree generation bump after base/branch mutations); worker failures now log and degrade to safe defaults. Response shapes unchanged.
  • Enforcement

    • Added CI ratchet tests (packages/host-service/src/no-main-loop-blocking.test.ts, apps/desktop/src/no-main-process-blocking.test.ts) to block sync subprocesses, recursive sync fs, and on-loop git-client construction outside worker dirs; allowlists only shrink. Backlog tracked in plans/off-loop-git-reads.md.

Summary by CodeRabbit

  • Performance

    • Improved responsiveness by moving Git operations off the main application loop.
    • Reduced repeated repository lookups through short-term caching.
  • Bug Fixes

    • Improved branch, worktree, ahead/behind, pull-request, and file comparison handling.
    • Added safer handling for missing remotes, unborn branches, missing files, and large file contents.
    • Improved repository cleanup reliability during failed operations.
  • Reliability

    • Added timeouts and request coalescing to prevent hangs and duplicate work.
    • Improved workspace creation and synchronization when resolving branches and remote references.
  • Tests

    • Expanded coverage for Git operations, caching, cleanup, and non-blocking behavior.