perf(dflash): add opt-in exact prefill bands
Summary
Adds an opt-in exact-prefill band scheduler for DeepSeek-V4 while keeping tokenwise q1 exact prefill as the safe default.
- enables compressor-safe q2-q4 exact-prefill bands with
DFLASH_DS4_EXACT_PREFILL_BANDS=1; - integrates #666 by @dusterbloom so exact width-2-to-4 feed-forward graphs use the qualified narrow MMVQ path;
- preserves exact behavior across attention, MoE routing, cache updates, logits, snapshots, and repeated requests;
- respects an explicit
LUCE_MMVQ_MAX_NCOLSthreshold and retains fused MMQ whenever grouped MMVQ does not support the selected width; - keeps q1 available immediately by leaving the exact-band flag unset.
This is an opt-in performance PR, not a general DeepSeek bug fix.
Integrated history
Current head: 25625f95f9e364d3b953ad027fa563f57a2088aa
PR #666 is merged as commit 27b615dc4b8246b2c1a7ed085fecd0bdfe783efb, preserving dusterbloom's authorship and original commit. The integration also resolves its two pending dispatch-threshold review concerns in 2e568087.
Existing #590 qualification
The following qualification was completed before the #666 integration at #590 head c8f9cc07e4ba708f0b2e9c473cef9f4196867f05.
Matched Lucebox6 condition: R9700 gfx1201 + Strix Halo gfx1151, exact prefill, --chunk 4, identical model, topology, prompt, and runtime settings, caches disabled. Each arm used two warmups followed by five measured requests.
| Mode | Measured prefill samples (tok/s) | Median | MAD |
|---|---|---|---|
| q1 reference A | 16.82, 16.58, 16.68, 16.76, 16.77 | 16.76 | 0.06 |
| exact q2-q4 bands | 17.70, 17.62, 17.59, 17.58, 17.89 | 17.62 | 0.04 |
| q1 reference B | 16.62, 16.49, 16.58, 16.71, 16.71 | 16.62 | 0.09 |
The pooled 10-sample q1 median was 16.695 tok/s. Exact bands reached 17.62 tok/s, a 5.54% gain. The two q1 brackets differed by 0.84%.
All 21 requests were cache-cold and produced the same response SHA-256:
cd5cb9fb5ac3c4f4007e8b41d117da21622439cd05c1728f3e82f90e4f869dad
Evidence root: /home/cheese/pr590-pr633-ready-20260825T1815Z
Evidence manifest SHA-256: 3517837f76bb5978154ca4de46e47625fbf2fbe91d1fbfd4aa6c0d0f13b52140
Standalone #666 qualification
PR #666 was independently qualified at 27b615dc on gfx1151 with ROCm 7.2.2, exact all-hot prefill, no drafter, and caches disabled.
| Prompt | Main median | #666 median | Lower prefill time |
|---|---|---|---|
| 64 tokens | 3341.6 ms | 2776.9 ms | 16.90% |
| 128 tokens | 6710.7 ms | 5602.6 ms | 16.51% |
Its frozen five-run qualification measured 17.72% and 16.62% lower total prefill time, with byte-identical q1/q4 route IDs, route weights, recurrent state, all-position F32 logits, and committed tokens. These are #666's standalone all-hot M64/M128 results, not a newly measured combined-head claim.
Band selection
--chunk remains the operator's maximum exact-band cap. The scheduler downshifts at compressor boundaries and prompt tails. Leaving DFLASH_DS4_EXACT_PREFILL_BANDS unset preserves q1. The integrated narrow-MMVQ path applies only to active exact multi-token bands and does not change speculative verification or approximate prefill.