perf(ds4): unify gfx1151 sparse-prefill stack

Draft
DA
1mo ago
+3693-413

Status

Draft integration branch. Do not merge until a matched 8K/32K/128K qualification runs on the 128 GiB Strix Halo target.

Composition

  • current main, including merged #673 sparse gate/up MMQ prefill;
  • #684's 8K layer-major sparse scheduling and direct long-prefill top-k handoff;
  • #667's already-qualified adaptive/fused gfx1151 verification stack;
  • #683's mixed ROCmFP MMQ policy, streaming MLA attention, vectorized staging, ROCmFP2/3 loads, adaptive sparse-MoE tiles, persistent compact work queue, and fast softmax.

The commits are replayed rather than percentage-stacked. Existing measurements from divergent branches are not added together.

Validation completed

Physical gfx1151 / ROCm HIP build on Lucebox Cina:

  • dflash_server builds;
  • test_deepseek4_unit: pass;
  • test_deepseek4_mmid_grouped_cuda: 76/76 parity cases pass;
  • test_server_unit: 455/455 pass;
  • test_feature_gate: 1/1 pass.

The composed streaming-attention microbenchmark measured 2613.4 us for the grouped path and 2079.0 us for FP32-staged streaming MLA with fast exp: 1.26x kernel speedup, NMSE 1.97e-14 and max absolute error 4.1e-8 versus the precise path.

Existing full-model evidence, not a unified result

  • #684 reported 254.921 tok/s at 7680 tokens versus a cited Vulkan 254.07 tok/s. The prompt/model artifact was not proven identical, so this is directional short-context parity, not a strict cross-engine result.
  • #667's independent 122879-token result was 142.82 tok/s versus cited Vulkan 226.82 tok/s. This remains the last matched long-context baseline; no unified long-context number is claimed yet.

Qualification switches

The first matched run should preserve the deployed release settings and additionally test:

  • DFLASH_DS4_DIRECT_INDEXER_TOPK=1
  • GGML_CUDA_MLA_STREAM_TOPK=1
  • GGML_CUDA_MLA_STREAM_F32_STAGE=1
  • GGML_CUDA_MLA_STREAM_FAST_EXP=1
  • GGML_CUDA_MMQ_MOE_ADAPTIVE_X=1
  • GGML_CUDA_MMQ_MOE_PERSISTENT=1
  • DFLASH_DS4_LONG_CONTEXT_CHUNK=8192

Each switch remains independently killable until the full-model matrix is complete.