perf(ds4): overlap q5 verification and stabilize 16K
Merged
DA
2mo ago+2351+2351+0123456789012345678901234567890123456789-263-263-012345678901234567890123456789
Summary
- batch q=5 hyper-connection and drafter-feature work across verifier lanes
- balance expert placement against the two-device critical path
- use bounded heterogeneous-prefill scratch and stable long-context graph shapes
- add overlap tracing and exact-context qualification controls
This PR is stacked on #569.
Validation
Configuration: R9700 target plus Strix Halo expert device, q=5, four routed experts, two warmups and three measured 128-token replies per context.
| Context | Client decode median |
|---|---|
| 2K | 75.818 tok/s |
| 4K | 74.530 tok/s |
| 8K | 69.898 tok/s |
| 16K | 62.685 tok/s |
| 2K after 16K | 76.703 tok/s |
All 25 responses matched the expected deterministic hash. Peak target-GPU memory use was 31.089 GiB.
test_deepseek4_unit: passedtest_server_unit: 322 passed, 0 failed- shell and Python checks: passed
git diff --check: passed