feat(qwen4exp): reuse cached prompt prefixes across requests (stacked on #808)
Stacked on #808 (MTP); PR #774 has merged. Merge order: #808, this. Review only the prefix-cache work: https://github.com/dusterbloom/lucebox-hub/compare/d54df4bc...f89be47d
What
Agent turns re-send the whole conversation, and qwen4exp re-prefilled all of it every turn. The server's existing prefix cache now works for qwen4exp: at the chat cut points it saves a prefix-sized device snapshot of the full sequence state (KV, recurrent GDN/conv state, PLE history, QSA indexer and pooled keys, MTP K/V and carry), restores it on a hit and prefills only the new tokens. A resident slot continues without a copy.
- Snapshots are sized to the prefix: 2.37 GB at 64K, 4.61 GB at 128K (UD-Q4_K_XL, F16 KV, MTP on).
- When a new snapshot does not fit, it replaces the restored prefix it extends.
- Cut points stay the same on hits and misses, including tool-result turns.
- No new flags or environment variables; it uses the server's
--prefix-cache-slots/ resident budget.
Exactness
A restored turn produces byte-identical output to the same request prefilled cold, with MTP adaptive and with MTP off.
test_qwen4exp_prefix_cache: save/restore, overwrite/reset, partial match, cancel, resident continuation, MTP rollback, allowance accounting, device bytes.- A rejected final MTP verify used to leave rollback copies in flight while the next request reset state on another stream; reset and snapshot now wait for them.
Speed (gfx1151, UD-Q4_K_XL, --max-ctx 131072)
64K agent context plus ~2K-token turns, time to first token (MTP adaptive / MTP off):
| cold | cached | |
|---|---|---|
| turn 2 (66,435 tokens, restores 64,224) | 64.5 / 65.3 s | 5.4 / 5.1 s |
| turn 3 (68,499 tokens, restores 66,429) | 68.0 / 64.8 s | 3.3 / 3.2 s |
Prefill keeps a 4096-row chunk in both cases. Memory is split speed-first: the chunk planner reserves the 4096-row prefill workspace and snapshots get the rest (6.3 GB at 131K with MTP). At 262K with MTP only ~0.6 GB remains, so long prefixes are not cached there; the guide recommends --max-ctx 131072 or less for agent workloads.
2026-10-07 stacked validation. The cumulative parent at a93d8bef passed the full gfx1151 HIP/model V7 gate (result sha256:93210dffb2442f36a841a066f10eb298898a84e389b94c55f2993bcde61e66f1). The only follow-up source change restores main's four-line backend-neutral CMake registration for test_ds41_mmid_width_invariance; the fixture source, runtime, kernels, and model code are unchanged. The exact follow-up tree 3bdbeee6 compiled and the restored fixture passed on gfx1151 (result sha256:bcb945b7c3d749ac239269f3f7f553bd707d92a977dbba1747089b16b7c1a906; binary sha256:75883d3798beb1a1f68f6b2a3669c855813a84468395c9548a245fc744dfd73f). The #810 candidate at 4a5a16f5 is byte/mode-identical to that tested tree. The throughput tables above retain their original measurement attribution.
For #809, this is cumulative stack evidence; its prefix-cache-only source boundary is the compare link above.