perf(ds4): elide intermediate prefill logits
Status
Draft: fresh current-main qualification does not support merging this as a performance PR.
The old +4.1% dual-GPU and +5.7% Strix-only measurements were made against an older main. Since then, #640 and #647 merged and changed the relevant DeepSeek4 paths. The old gfx1151 MMQ tile edits also overlap newer code in current main and are no longer part of the reconciled candidate.
Remaining candidate scope
The current-main reconciliation keeps only the DeepSeek4 logits-elision work:
- avoid materializing vocabulary logits on intermediate prefill chunks that do not consume them;
- keep terminal and snapshot-boundary logits;
- distinguish logits and no-logits layer-major graph variants;
- allow repeated terminal-only prompts to populate the graph cache while protecting an established bulk no-logits graph.
A focused unit test was added for the graph-cache replacement policy. That local candidate has not been pushed because its fresh performance result is not acceptable.
Fresh current-main results
Base: current main at c994209a2c03c1bf3426924aba52b5edc05501ff
Local reconciled candidate: d38bb5dc5d883b37f3ba7822bf864d51c501070d
Matched Lucebox6 runs used the same ROCmFP2 model, placement, prompt, runtime settings, cache policy, two warmups, and five measured requests per arm. Main was bracketed around the candidate.
| Workload | Pooled current-main median | Candidate median | Delta |
|---|---|---|---|
| Dual GPU, 401 tokens, chunk 512 | 178.43 tok/s | 178.51 tok/s | +0.04%, neutral |
| Dual GPU, 2048 tokens, chunk 512 | 213.91 tok/s | 198.49 tok/s | -7.21% |
| Strix only, 401 tokens, chunk 512 | 178.365 tok/s | 178.27 tok/s | -0.05%, neutral |
Every request produced the same exact response SHA-256:
cd5cb9fb5ac3c4f4007e8b41d117da21622439cd05c1728f3e82f90e4f869dad
The 2048-token candidate arm had high variance, but it did not show a positive signal and therefore fails closed. No performance claim is made from it.
Validation completed on the local reconciliation
- clean Release HIP build for
gfx1151;gfx1201, HIP graphs enabled, ROCm 7.2.4; test_deepseek4_unitexitedOKon physical gfx1201 and gfx1151;- the two-device scratch subtest self-skipped in the deliberately single-device invocations and is not represented as passed;
- exact full-model response parity across all measured cells;
git diff --checkclean;- KFD clean after testing.
Evidence root: /home/cheese/pr590-pr633-ready-20260825T1815Z
Manifest SHA-256: 26e936d5ad47d26cc9b45ce9c01d87d9e6e83543c0435faeaa2780678a4b497f