perf(qwen35moe): pool-sized KV reservation keeps experts hot at high max_ctx
Merged
DU
4mo ago+374+374+012345678901234567890123456789-19-19-01234567890123456789
What
At high --max-ctx, the qwen35moe (Qwen3.6-35B-A3B hybrid SSM-MoE) expert
placement reserved KV for the full max_ctx, forcing experts cold even when
KVFlash bounds resident KV to a small pool. This reserves only pool-sized KV
when KVFlash is active, so experts stay hot.
Why it matters
Real claude-code client, RTX 3090 24 GB, Q3_K_M, --max-ctx 131072:
| metric | cold cliff (before) | all-hot (after) |
|---|---|---|
| cold experts | 2203 | 0 |
| cold prefill (26.8K tok) | ~108 s | 16.1 s |
| decode | ~43 tok/s | 66 tok/s |
Shape
- New
server/src/common/kvflash_placement.h— architecture-agnostic placement
decision; the next MoE backend (DS4/gemma4) can reuse the same reservation rule. - ~280 LOC. Tests:
test_kvflash_placement.cpp(5-case CPU unit) +
test_kvflash_moe_placement.sh(GPU, 2203 → 0 cold experts).
Base of the KVFlash-MoE series: placement → pager-serde → prefill-snapshot.