feat(scheduler): add cache_ttl "never" sentinel for always-warm lanes

Merged
IC
2mo ago
+313-56

Problem

cache_ttl assumes idle > TTL means the provider evicted the prompt cache, so the next prefix rebuild is free. Two consumers act on that assumption:

  1. the scheduler converts a defer pass into an execute pass (scheduler.ts), and
  2. mustMaterialize fires the ttl_idle HARD fold (via hardCacheExpired in transform.ts).

On lanes kept warm by an external keepwarm mechanism (prewarm proxies, dedicated cache-keep tools that re-warm the provider cache out-of-band), the assumption is wrong: the cache never goes cold, MC's lastResponseTime goes stale anyway, and both consumers false-positive. MC then initiates a rebuild it believes is free that is actually a full paid cache-write — measured 450-560K tokens per fold on large sessions. There was no way to express "this lane never goes cold": the config requires a duration.

Fix

cache_ttl: "never" (case-insensitive, works as the string form or any per-model value):

  • parseCacheTtl("never") returns Infinity; both consumers go inert through the existing comparisons (elapsed > Infinity / elapsed >= Infinity are never true) — no new branches in the hot path.
  • The Rust scheduler mirrors the sentinel with u64::MAX (its predicates already use saturating_sub, so no overflow path).
  • Status surfaces render honestly instead of a bogus countdown: /ctx-status shows never expires (always-warm lane); the TUI sidebar gets a JSON-safe cacheNeverExpires flag on StatusDetail (Infinity does not survive JSON-RPC); Pi's status dialog now uses the shared parseCacheTtl (it previously used a private fallback parser that would have shown a 5m countdown).
  • hardCacheExpired is extracted into a pure computeHardCacheExpired helper so the "never" -> Infinity -> false chain has direct unit coverage instead of only flag-consumption coverage.

Docs: CONFIGURATION.md describes the sentinel, what it disables, and the tradeoff — after a genuinely cold start (e.g. the keepwarm process died), the free-fold window is not detected on such lanes; mutations then apply at the execute threshold.

Verification

  • plugin: 2995 pass / 0 fail; pi-plugin: 666 pass / 0 fail; tsc --noEmit clean on both
  • new tests red-verified: reverting the sentinel fails the scheduler tests and the computeHardCacheExpired test
  • Rust: sentinel + predicate + scheduler-decision tests added in scheduler.rs's test module (note: I could not run cargo test locally — the crates workspace has path deps outside this repo; the change is a 3-line early return mirroring the TS logic)
  • schema + generated docs regenerated (build-schema / build-config-docs drift tests green); check:tui-compiled green

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Adds cache_ttl: "never" to disable the idle-TTL heuristic on always-warm lanes, preventing false executes and paid cache rebuilds. Status surfaces now show “never expires” and keep RPC fields JSON-safe; invalid-TTL diagnostics are restored.

  • New Features

    • parseCacheTtl("never") returns Infinity (TS) / u64::MAX (Rust) so idle-execute and ttl_idle never fire; computeHardCacheExpired exported for consistent checks with onInvalid.
    • Status UIs (/ctx-status, TUI, Pi) render “never expires” via a JSON-safe cacheNeverExpires flag; Pi uses the shared parser; docs and schema updated.
  • Bug Fixes

    • Status RPC avoids Infinity by keeping cacheRemainingMs numeric (0) when TTL is “never” and keying on cacheNeverExpires; auto-execute messaging switches to threshold-only.
    • Transform restores invalid-TTL pass outcomes and session logs through the onInvalid callback.

Greptile Summary

Adds an always-warm cache sentinel across scheduler and status paths.

  • Parses cache_ttl: "never" as a non-expiring TTL in TypeScript and Rust.
  • Prevents idle-TTL scheduling and hard-fold decisions for always-warm lanes.
  • Propagates JSON-safe non-expiring status semantics through RPC and TUI surfaces.
  • Restores invalid-TTL diagnostics after extracting the expiration helper.
  • Updates tests, schema, and configuration documentation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain in the fixes associated with the previous review threads.

Important Files Changed

FilenameOverview
packages/plugin/src/plugin/rpc-handlers.tsRepresents non-expiring cache status with an explicit JSON-safe flag while keeping numeric RPC fields finite.
packages/plugin/src/hooks/magic-context/transform.tsCentralizes hard-cache expiration calculation and preserves the existing invalid-TTL diagnostics.
packages/plugin/src/features/magic-context/scheduler.tsAdds case-insensitive parsing of the never sentinel as positive infinity.
crates/mc-module/src/scheduler.rsMirrors the non-expiring sentinel in Rust using u64::MAX.
packages/plugin/src/tui/index.tsxDisplays non-expiring cache state and threshold-only auto-execution using the RPC flag.
packages/pi-plugin/src/dialogs/status-dialog.tsUses the shared TTL parser and renders the always-warm state without a countdown.

Reviews (5): Last reviewed commit: "fix(scheduler): keep cacheRemainingMs JS..." | Re-trigger Greptile