perf(ds4): overlap heterogeneous owner prefill
[!IMPORTANT]
Stacking: #647 is the incremental scheduling optimization on top of #640 (51d2929). Its feature commits are cdcd4ac4, 42bd9ab2, 538b3041, and 2eab093e. The current GitHub head 7edb92b702efebc31116655910a81a44c2260d8a merges that unchanged feature history with current main at 850b5154.
Summary
Removes a host-side serialization point from heterogeneous DeepSeek-V4 expert prefill while preserving the existing device-side copy dependency. The cold owner begins first; hot-owner submission can proceed once the common-input peer-copy dependency has been enqueued.
The D2D copy is neither removed nor made smaller. This PR improves scheduling so independent work overlaps while the destination stream still waits correctly before consuming copied input.
Classification: primarily a performance PR. It includes exception-safety, deadlock prevention, and peer-copy cutoff fixes required to make the scheduling optimization safe, but it is not primarily a standalone bug-fix PR.
Fresh stack requalification
Freshly remeasured on Lucebox6 against recorded main snapshot f686c447f067a04ea100a996e4c826e8cc4decc1, with matched placement and chunk sizes, caches disabled, two warmups, and five measured requests per cell.
| Workload | Recorded main | #640 | #640 + #647 |
|---|---|---|---|
401-token prefill, chunk=512 | 94.59 tok/s | 178.98 tok/s | 191.47 tok/s |
2048-token prefill, chunk=2048 | 329.16 tok/s | 298.97 tok/s | 328.89 tok/s |
At 401 tokens, #647 adds 7.0% on top of #640 and takes the combined stack to +102.4% (2.02x) over the recorded main result.
At 2048 tokens, #647 adds 10.0% over #640 and recovers the stack to recorded main. This is a strong recovery of #640's wide-context regression, not a net wide-context speedup over main.
All retained responses produced SHA-256 cd5cb9fb5ac3c4f4007e8b41d117da21622439cd05c1728f3e82f90e4f869dad, and every retained request was cache-cold.
Evidence:
- 401-token root:
/home/cheese/pr640-pr647-fusion-live-main-short401-20260825T121700Z - 401 manifest SHA-256:
6a9d6b586ac04fc153b9af2b87a5175c01fe3178d9808c9817f7b89f87ac8d21 - 2048-token root:
/home/cheese/pr640-pr647-fusion-live-main-requal2-20260825T112300Z - 2048 manifest SHA-256:
b73afed6803e26ed111f521eb0690cf9d8056e30c5f890b0067a5604d181e66a
Earlier matched incremental qualification
The preserved earlier off/on qualification, where both arms already included #640, measured:
| Workload | #647 off | #647 on | Incremental gain |
|---|---|---|---|
| 2048-token prefill | 308.67 tok/s | 321.86 tok/s | +4.27% |
| 401-token prefill | 179.62 tok/s | 187.87 tok/s | +4.59% |
Those are historical matched-session results for the qualified source. They are retained separately from the fresh current-stack table and are not mixed with it.
Why it works
| Causal trace metric | Disabled | Enabled |
|---|---|---|
| Expert-owner layers with overlapping MMQ | 2 / 43 | 40 / 43 |
| Simultaneous gfx1201/gfx1151 busy time | 47.1 ms | 535.7 ms |
| Median cold-minus-hot owner start gap | 23.50 ms | 0.57 ms |
| D2D copies | 43 | 43 |
| Median D2D duration | 9.680 ms | 9.720 ms |
Safety and validation
- default-off escape hatch: leave
DFLASH_MOE_COLD_INPUT_FIRSTunset or set it to0; - original scheduling retained when GGML peer-copy batching is enabled;
- readiness is single-shot and exception-safe, so pre-readiness worker failures propagate instead of deadlocking;
- exact output oracle: PASS throughout the retained timing cells;
test_moe_input_ready: PASS, including bounded exception propagation and concurrent single-shot signaling;test_deepseek4_unit: PASS on physical gfx1201 and gfx1151;- current head retains the exact four reviewed feature commits; the merge commit itself has not been separately performance-qualified;
- GitHub currently reports the PR mergeable and all currently reported checks successful; no merge-ready claim is made pending maintainer review.