feat(scheduler): add cache_ttl "never" sentinel for always-warm lanes
Problem
cache_ttl assumes idle > TTL means the provider evicted the prompt cache, so the next prefix rebuild is free. Two consumers act on that assumption:
- the scheduler converts a defer pass into an execute pass (
scheduler.ts), and mustMaterializefires thettl_idleHARD fold (viahardCacheExpiredintransform.ts).
On lanes kept warm by an external keepwarm mechanism (prewarm proxies, dedicated cache-keep tools that re-warm the provider cache out-of-band), the assumption is wrong: the cache never goes cold, MC's lastResponseTime goes stale anyway, and both consumers false-positive. MC then initiates a rebuild it believes is free that is actually a full paid cache-write — measured 450-560K tokens per fold on large sessions. There was no way to express "this lane never goes cold": the config requires a duration.
Fix
cache_ttl: "never" (case-insensitive, works as the string form or any per-model value):
parseCacheTtl("never")returnsInfinity; both consumers go inert through the existing comparisons (elapsed > Infinity/elapsed >= Infinityare never true) — no new branches in the hot path.- The Rust scheduler mirrors the sentinel with
u64::MAX(its predicates already usesaturating_sub, so no overflow path). - Status surfaces render honestly instead of a bogus countdown:
/ctx-statusshowsnever expires (always-warm lane); the TUI sidebar gets a JSON-safecacheNeverExpiresflag onStatusDetail(Infinity does not survive JSON-RPC); Pi's status dialog now uses the sharedparseCacheTtl(it previously used a private fallback parser that would have shown a 5m countdown). hardCacheExpiredis extracted into a purecomputeHardCacheExpiredhelper so the"never" -> Infinity -> falsechain has direct unit coverage instead of only flag-consumption coverage.
Docs: CONFIGURATION.md describes the sentinel, what it disables, and the tradeoff — after a genuinely cold start (e.g. the keepwarm process died), the free-fold window is not detected on such lanes; mutations then apply at the execute threshold.
Verification
- plugin: 2995 pass / 0 fail; pi-plugin: 666 pass / 0 fail;
tsc --noEmitclean on both - new tests red-verified: reverting the sentinel fails the scheduler tests and the
computeHardCacheExpiredtest - Rust: sentinel + predicate + scheduler-decision tests added in
scheduler.rs's test module (note: I could not runcargo testlocally — the crates workspace has path deps outside this repo; the change is a 3-line early return mirroring the TS logic) - schema + generated docs regenerated (
build-schema/build-config-docsdrift tests green);check:tui-compiledgreen
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.
Summary by cubic
Adds cache_ttl: "never" to disable the idle-TTL heuristic on always-warm lanes, preventing false executes and paid cache rebuilds. Status surfaces now show “never expires” and keep RPC fields JSON-safe; invalid-TTL diagnostics are restored.
-
New Features
parseCacheTtl("never")returns Infinity (TS) /u64::MAX(Rust) so idle-execute andttl_idlenever fire;computeHardCacheExpiredexported for consistent checks withonInvalid.- Status UIs (
/ctx-status, TUI, Pi) render “never expires” via a JSON-safecacheNeverExpiresflag; Pi uses the shared parser; docs and schema updated.
-
Bug Fixes
- Status RPC avoids
Infinityby keepingcacheRemainingMsnumeric (0) when TTL is “never” and keying oncacheNeverExpires; auto-execute messaging switches to threshold-only. - Transform restores invalid-TTL pass outcomes and session logs through the
onInvalidcallback.
- Status RPC avoids
Greptile Summary
Adds an always-warm cache sentinel across scheduler and status paths.
- Parses
cache_ttl: "never"as a non-expiring TTL in TypeScript and Rust. - Prevents idle-TTL scheduling and hard-fold decisions for always-warm lanes.
- Propagates JSON-safe non-expiring status semantics through RPC and TUI surfaces.
- Restores invalid-TTL diagnostics after extracting the expiration helper.
- Updates tests, schema, and configuration documentation.
Confidence Score: 5/5
The PR appears safe to merge.
No blocking failures remain in the fixes associated with the previous review threads.
Important Files Changed
| Filename | Overview |
|---|---|
| packages/plugin/src/plugin/rpc-handlers.ts | Represents non-expiring cache status with an explicit JSON-safe flag while keeping numeric RPC fields finite. |
| packages/plugin/src/hooks/magic-context/transform.ts | Centralizes hard-cache expiration calculation and preserves the existing invalid-TTL diagnostics. |
| packages/plugin/src/features/magic-context/scheduler.ts | Adds case-insensitive parsing of the never sentinel as positive infinity. |
| crates/mc-module/src/scheduler.rs | Mirrors the non-expiring sentinel in Rust using u64::MAX. |
| packages/plugin/src/tui/index.tsx | Displays non-expiring cache state and threshold-only auto-execution using the RPC flag. |
| packages/pi-plugin/src/dialogs/status-dialog.ts | Uses the shared TTL parser and renders the always-warm state without a countdown. |
Reviews (5): Last reviewed commit: "fix(scheduler): keep cacheRemainingMs JS..." | Re-trigger Greptile