perf(ds4): enable sparse gate-up MMQ prefill
Summary
- default the existing qtype-106 ROCmFP2 MIX MMQ path only for the routed gate/up pair on gfx1151
- leave qtype-106 down projections and every qtype-105 projection on the existing generic policy
- preserve the existing RDNA4 opt-in and
DFLASH_DS4_MIX_MMQ_PREFILL=0opt-out - add no flag, kernel, graph operation, or model-specific scheduler
The role boundary is structural: ggml_cuda_try_fuse_mul_mat_glu() sees the matched routed gate/up pair. Down is a later independent MUL_MAT_ID and never enters this default.
Result
Ryzen AI Max+ 395 / gfx1151, ROCm 7.2.4, monolithic sparse prefill, chunk 512, all six routed experts, caches disabled:
| Prompt | Baseline | This branch | Gain | Lower time |
|---|---|---|---|---|
| 8K | 109.548 tok/s | 146.474 tok/s | +33.71% | 25.21% |
| 16K | 108.691 tok/s | 143.697 tok/s | +32.21% | 24.36% |
| 64K | 100.128 tok/s | 128.580 tok/s | +28.42% | 22.13% |
| 128K | 85.236 tok/s | 106.166 tok/s | +24.56% | 19.71% |
The 8K row is a same-binary default-versus-explicit-opt-out A/B. The longer baseline rows are retained from the frozen clean-environment campaign on the same hardware, model, chunk, and request family; every final role-scoped candidate was rerun after this fix.
Why this path
An 8K request-scoped profile attributed 45.28% of GPU time to MIX dequantization and 17.54% to its GEMMs. Flash attention was 12.87%; the Lightning Indexer was 0.67%.
The faster type-wide experiment was rejected. It admitted qtype-106 down projections too, exceeding the qualified gate/up boundary. Enabling both MIX types was also rejected after failing the retained long code-audit prompt.
Dispatch telemetry over the final 8K run recorded all 688 down projections on the native fallback: 448 qtype-105 and 240 qtype-106. Gate/up was consumed by the paired fusion. Explicit DFLASH_DS4_MIX_MMQ_PREFILL=0 restored the 109.548 tok/s baseline.
Verification
dflash_serverbuilds on gfx1151 with-j4test_rocmfpx_mmq: all cases pass on gfx1151test_rocmfp3_mix_registry: pass on gfx1151- long memory prompt: exact expected answer
- long code-audit prompt: correct consolidation diagnosis
- no graph-capture eligibility change: native wide down remains truthfully graph-ineligible
One file, 8 insertions, 1 deletion.