#684 perf(ds4): extend sparse prefill scheduling to 10K

Summary

• integrate current main 2f0eff051 including #658/#664/#667/#683, retaining fused-combine alignment admission, MLA scratch barriers, saved verifier rows, runtime RoPE positions, and model-local mixed-MMQ policy
• keep main's already-merged fused MoE combine and its existing disable/trace controls; this is not an additional gain over current main
• extend current main's DS4 prefill schedule from two to five independent
  2,048-token numerical bands, admitting one physical frontier up to 10,240
  tokens
• derive long ratio-4 causal visibility in the HIP attention path instead of
  retaining and uploading its quadratic score mask
• pass authoritative long-prefill top-k rows directly, keep selected
  raw/compressed KV transport in F16, and stream each selected latent row once
  across sixteen wave32 heads
• preserve native top-6 routing, selected-row order, per-row reduction order,
  independent 2K attention bands, and chronological state commit
• preserve mrciffa's authorship on the three extracted streaming-attention
  commits from #683
• preserve cheese-cakee's authorship on the fused-combine implementation and
  correctness commits from #658

Routing, expert matrix multiplies, sparse/exact prefill admission, and state
commit retain main's policies. Maskless ratio-4 prefill selects streaming
attention automatically when eligible, while other indexed shapes keep the
existing opt-in policy. Existing streaming, FP32 staging, fast-exp and fused
combine overrides are preserved; no new serving flag or private BWS policy
is added. GGML_CUDA_MLA_STREAM_TOPK=0 retains the compact maskless fallback.

September 10 integration fix

The old PR head predates important merged correctness fixes. This update
retains them, removes the duplicate fused-combine CMake target produced by
the overlap, and adds permanent HIP regression coverage:

• compact versus streamed attention at 10,240 tokens and a 129-token tail
  starting at position 122,880, with pinned inputs and three submissions;
• explicit-mask versus analytical-mask outputs and numerical tolerance
  against the compact reference, with no loosened tolerances;
• default streaming dispatch and the explicit disable switch;
• rejection of runtime-position or oversized (>512) top-k contracts on the
  fixed-position maskless path; masked verification keeps its wider support;
• fused-combine alignment and new maskless tests included in both HIP CI jobs.

Fresh integrated LB8 validation: Release HIP gfx1151, graphs ON, 504/504
selected CTests passed. At 10,240 tokens the compact/streamed maximum
absolute error is 1.2218952e-6; at the 129-token tail starting at 122,880 it
is 3.4272671e-7. Explicit-mask and analytical-mask streaming outputs are
byte-identical on all three submissions of each fixture. These are kernel
fixtures, not full-model long-context measurements.

Wide-batch memory qualification did not pass. On the same 9,133-token
retrieval task with the current main optimizations and installed target,
candidate AR and adaptive runs at requested chunk 10,240 both crossed the
unchanged 4 GiB host-headroom guard before producing output. Candidate AR at
chunk 8,192 also crossed the guard while handling the tail. The runner
terminated these attempts; no actual OOM was observed. Their partial timings
are not counted as gains. The safe-chunk regression check is reported
separately, not substituted for 10K qualification.

The older model timings and quality results below are historical receipts,
not measurements of this merged head. The recorded prose quality miss and
wider-batch memory qualification remain open. Keep this PR draft; no matched
Vulkan parity or full-model byte-identity claim is made.

Fresh matched 4K-chunk model regression

Same 9,133-token cold-prefix retrieval task, installed ROCmFPX MIX target,
six experts, same input token IDs, 32K context allocation, caches off, two
measured requests per arm. Both use effective chunks 4,096 + 4,096 + 941.
Rates use API-reported full stage time, not client decode windows.

All eight responses pass strict 5/5 retrieval, contain 59 tokens, and have
the same output hash. Mean request wall time improves from 43.541 to 40.054
seconds in AR and from 42.680 to 38.779 seconds in adaptive mode.

Adaptive decode is not preserved in this check. Adaptive width is enabled
with q cap 4. Main uses 15 rounds with 98% draft acceptance; the candidate
uses 16 with 90% in both repeats. Round wall times are similar, but the extra
round lowers decode throughput. The precise numerical cause of the acceptance
change is not isolated. This regression, wider-batch memory headroom, and the
historical prose-quality miss remain open before merge-readiness. No fixed
high-acceptance or unmatched Vulkan win is claimed.

Fix head 431721f1e; engine/test code 1b91332f6 (last commit is documentation
only), integrated main 2f0eff051. Full receipts on LB8:
/data/pr684-fix-20260910.HsTxSi/, including model-summary.json, the four
completed arms, three aborted candidate attempts, regression-tests.log,
and the exact scripts. The initial main/adaptive arm requested chunk 10,240;
main's 4,096 cap gives the same effective shape as the candidate's explicit
4,096 setting.

Historical fused-combine discriminator (already merged in #658)

A frozen F-C-F-C process bracket used the same full model, prompts, runtime
shape, and machine state. Four warm observations per arm produced:

The exact same-binary comparison measured +3.270% at 2K and +4.923% at 8K.
The final flag-free build independently reached 204.044 and 252.362 tok/s.
Its server SHA-256 is
cc21d9f4257492f5ae46758088baaa0b459f43c0dcd3cfe280c13bd7cbf9c0dd.

The focused HIP microbenchmark improved from 1.876 to 0.240 ms at 401 rows
(7.807x), and from 10.374 to 1.158 ms at 2,048 rows (8.962x). Decode remained
within noise, so no decode gain is claimed.

That historical A/B used #658's selector before the old branch removed it.
The current integration retains main's selector and trace support.

Historical production workflow (before current-main integration)

Strix Halo gfx1151, 98.3 GB ROCmFPX MIX artifact, target-only sparse
prefill, native top-6 routing, fused autoregressive decode, Q4_0 K/V cache,
caches off, 9,133-token SWE-bench-shaped prompt, and up to 128 generated
tokens:

The final two warm prefill samples were 252.09 and 252.26 tok/s. The matched
128-output-token warm sample saved 3.522 seconds. Decode code is unchanged;
its small movement is reported, not claimed as a decode optimization. Sparse
replay produced 47 tokens in one warm sample and 128 in the other, so whole
wall is compared only between matched 128-token samples.

The earlier clean-head 7,680-token receipt remains 254.921 tok/s, nominally
0.335% above the cited 254.07 tok/s Vulkan result at the same token count. The
prompts, artifacts, and quality protocols are not proven identical, so this PR
does not claim a cross-runtime record from that comparison.

Memory and execution shape

The ratio-4 analytic path removes retained masks from 21 attention layers:

• 6.562 GiB at 8,192 tokens
• 8.157 GiB at 9,133 tokens

That memory headroom permits the five-band frontier without a runtime memory
guard. The four full-vocabulary quality processes reported zero swaps; the
end-to-end runs had no OOM, managed-memory cliff, or system-manager failure.

Failure found by the real workflow

The first maskless implementation passed units but segfaulted on the real
9,133-token request. A GDB hardware watchpoint found the generic ring-mask
writer overwriting the layer-2 graph header: after omitting the layer-major
score mask, control incorrectly fell through to old-row logic that assumes a
saved ring unavailable in layer-major prefill.

The root-cause correction restricts that branch to
causal_batch && !layer_major_batch. It adds no fallback, allocation, flag,
or defensive guard. The corrected source crosses the former crash point and
passes the real workflow.

Historical qualification (before current-main integration)

Release HIP gfx1151, build concurrency capped at four. The full pinned-Halo
test_deepseek4_unit passes, including the ratio-4 boundary formula and a
focused explicit-mask versus analytic-mask GPU oracle with bit-identical
streaming-attention output. The fused-combine qualification also passes all
2x4 top-k/row/shared-output HIP cases, meta allocation/layout contracts, CPU/HIP
parity, legacy differential tolerance (1e-6), and a byte-exact signed-zero
case. test_ggml_meta_split_layout passes 17/17 checks.

The unchanged Q2048/Q8192/Q8192/Q2048 ABBA harness used four frozen real-domain
token tapes, 128 teacher-forced full-vocabulary rows, and 32 greedy rows per
arm:

Code, tool, and multilingual passed the preregistered limits. Prose passed its
top-1 and greedy rules but measured KLD 0.011536 against the frozen
max(0.006, 1.25 * noise) limit of 0.011216, a 0.000320 miss. This is
consistent with the inherited sparse replay noise, but the sample remains
reported as failed: no threshold was changed and it was not rerun until
passing. The focused attention-kernel oracle is bit-identical.

Known limitation

Sparse prefill remains approximate and nondeterministic across identical
same-arm replays. State and logit hashes differ even within equal-width
controls. These results do not claim bitwise full-model determinism.

The 9,133-token timing fixture carries an agent-shaped coding transcript but
no actual tool schema and only a 128-token output budget. It directly exercises
the changed path; it is not a complete agent-task quality proof.

Scope

This PR contains public scheduling, selected-row dataflow, F16 transport,
generic HIP attention-kernel work, tests, and the narrow ring-mask root-cause
fix. It contains no private Behavioural Working Set mask, calibration, expert
reuse, slab, or placement policy. Temporary attribution telemetry was removed
before the final commit.