perf(desktop): evict parked terminal runtimes past a configurable cap

Merged
KI
4mo ago
+1970-5

Links

  • Part of SUPER-1545 (terminal half; parked-webview half deferred)
  • Findings doc: apps/desktop/docs/MEMORY_AND_WORKERS_REVIEW.md (added in this PR)

Summary

  • Parked (hidden) terminals were never disposed — every one kept its 5,000-line buffer, WebGL context, and WebSocket alive with no cap, which is the largest renderer memory leak-by-retention we have (CDP-measured: 18 terminals → 1.59 GB renderer RSS).
  • The registry now LRU-evicts parked runtimes past a cap: buffer serialized to localStorage first, PTY untouched, rebuilt from the persisted buffer on next mount.
  • The cap is user-configurable (Settings → Terminal → "Background terminal memory", 2–64, default 12) and applies live — lowering it sweeps immediately.

Why / Context

Full investigation + measurements in apps/desktop/docs/MEMORY_AND_WORKERS_REVIEW.md. Controlled A/Bs (identical scripted CDP protocol, fresh launch per build, measured at cap 5):

18 terminals, 5k lines eachBeforeAfter
Live runtimes / WebGL contexts176
JS heap658 MB381 MB
Renderer RSS1,655 MB1,139 MB
Tab-switch p5059 ms76 ms
24 terminals with live streaming processesBeforeAfter
Live runtimes236
JS heap (steady state)836→846 MB climbing373→376 MB flat
Renderer RSS1,741 MB1,165 MB
Renderer CPU (30 s cputime delta)30 %21 %

The CPU gap is structural: eviction closes released terminals' WebSockets, so their streams are neither delivered nor parsed. Savings scale with how far usage exceeds the cap; at the shipped default of 12, users under a dozen background terminals never see eviction.

How It Works

  • RegistryEntry.lastUsedAt (monotonic counter) is bumped on mount/detach; a deferred (one-tick) sweep after each detach releases the least-recently-used parked runtimes past the cap. The deferral matters: a workspace switch detaches outgoing panes before mounting incoming ones, so an immediate sweep could evict a terminal about to be re-adopted.
  • Eviction is disposeEntry({clearPersistedState:false}) — same as release(): serialize 1,000 lines to localStorage, close the socket, dispose xterm/WebGL. The PTY and host-side replay buffer are untouched; re-opening the tab rebuilds from localStorage and reconnects (the existing cold-restore path, incl. the "Session Contents Restored" divider).
  • Attached (visible) terminals are never candidates; runtime-less entries (transport-only) are skipped.
  • Cap plumbing: terminal_parked_runtime_cap on the local-db settings row (migration 0042, generated with drizzle-kit), zod-clamped get/set procedures, seeded into the registry by the authenticated layout at boot, and applied optimistically (with rollback) by the settings UI.

Manual QA Checklist

All exercised live over CDP against the dev app (see doc for full protocol):

  • Cap enforced: cycling 18 tabs leaves cap+1 live runtimes; zero WebGL context loss
  • Evicted terminal re-open: runtime rebuilt, 1,000-line scrollback restored (verified against content written in a previous app session), WS reconnects to the live PTY
  • Streaming processes in evicted terminals keep running host-side and resume display on re-open
  • Default 12 seeds from settings at boot; changing the setting applies immediately (12→6 swept live) and persists across restart
  • Input validation: out-of-range values (999) rejected server-side and revert on blur client-side; unchanged values don't mutate
  • Settings UI renders in Settings → Terminal; discoverable via settings search ("memory", "ram", "parked")
  • Migration 0042 applied cleanly on the existing dev DB

Testing

  • bun run lint, bun run typecheck — clean
  • Two controlled CDP A/B stress runs (18-terminal fill; 24-terminal live-stream) — tables above

Design Decisions

  • Parked-count cap, not total-live cap: visible terminals can never be evicted, so capping parked instances is the only enforceable invariant (a 6-way split stays fully live).
  • Number input over dropdown: matches FontSettingSection blur-commit pattern and exposes the full validated range; backend validation (2–64) is intentionally wider than any UI presets so the control can change without schema churn.
  • Deferred sweep over immediate: see How It Works — avoids evict/rebuild churn during workspace switches.

Known Limitations

  • Evicted terminals lose scrollback beyond 1,000 lines (serialize window). Deliberate trade-off; the cap default (12) keeps most users out of eviction entirely.
  • In-place TUIs (vim, htop, agent TUIs) restore as a static snapshot — same behavior as today's app-restart cold restore, but eviction can trigger it mid-session for users past the cap.
  • Restore replays localStorage + the host's 64 KB ring; the overlapping tail matches existing cold-restore behavior.
  • terminal_warm_attached/terminal_cold_restored telemetry will shift slightly (evictions produce cold-restore-shaped attaches).

Follow-ups

  • Parked-<webview> eviction (second half of SUPER-1545)
  • Alt-screen exemption (skip evicting terminals with an active TUI; host ModeTracker knows)
  • Persist-failure guard: skip eviction if the localStorage write fails (quota)

Open in Stage

Summary by cubic

Evicts least‑recently‑used parked terminal runtimes past a configurable cap to cut renderer memory and GPU context usage, delivering ~30–40% lower renderer RSS in stress tests (terminal half of SUPER‑1545). Also adds a live-applied setting (default 12) and fixes formatting in ai-workspace-names.ts to satisfy Biome.

  • New Features

    • LRU-evict parked terminal runtimes past a cap; on eviction we persist 1,000 lines to localStorage, close the socket, and dispose xterm/WebGL; the PTY keeps running and the runtime rebuilds on reopen.
    • New setting: Settings → Terminal → “Background terminal memory” (2–64, default 12); applies instantly; attached terminals are never evicted; sweeps are deferred one tick to avoid workspace-switch churn.
    • Settings search entry added; findings doc at apps/desktop/docs/MEMORY_AND_WORKERS_REVIEW.md. Extracted a pure eviction-selection policy for isolated unit tests.
  • Migration

    • Adds terminal_parked_runtime_cap to the local DB (migration 0042); clamped to 2–64 and seeded into the registry at boot.

Summary by CodeRabbit

  • New Features
    • Added a Terminal Settings option to cap retained inactive (“parked”) background terminals.
    • The setting is persisted, loaded into the terminal runtime registry, and applied to evict excess parked terminals using an LRU-style policy while preserving saved scrollback.
  • Documentation
    • Added “Memory Profile & Worker Offload Review,” including measured behavior and recommendations for terminal and worker/memory usage.
  • Tests
    • Added unit tests covering parked-terminal eviction selection and parked-cap normalization/validation.