fix(ds4): preserve reference-exact speculative verification

CH
3mo ago
+715-16

Summary

  • preserve DFLASH_DS4_SPEC_REFERENCE_EXACT=1 as the supported reference-exact speculative verification mode
  • keep DFLASH_DS4_FUSED_VERIFY=1 intentionally approximate for throughput experiments
  • add a fail-closed, model-backed target-only-vs-reference-exact token-ID parity gate
  • register parser, policy-sanitization, and evidence-lifecycle regressions with CTest
  • preserve the normal fused q=1 target path during DSpark feature capture so exact prompt replay remains numerically identical to target-only execution

This is the current-main rewrite of the superseded historical implementation. Reference-exact verification and approximate fused verification remain separate, explicit modes.

Exactness contract

The model-backed gate holds the target execution policy constant between greedy target-only generation and reference-exact speculation. It removes inherited DS4/MoE/MMVQ policy variables, then differs only in DSpark/spec/reference-exact activation.

A passing result requires:

  • identical complete generated token-ID sequences
  • the reference-exact activation banner
  • speculative work with exact integer matched / offered counters
  • at least one rejection (matched < offered)
  • full_snap=1, proving the rollback-capable path was active
  • SHA-256 identities for the server binary, target GGUF, DSpark GGUF, and prompt

Evidence is staged under unique attempt directories. Failed attempts retain diagnostics without occupying final evidence paths; complete logs and a manifest are promoted without overwriting existing evidence.

Demonstrated defect and fix

The first real-model gfx1151 run exposed a token-0 mismatch between target-only greedy generation and the DSpark-enabled reference-exact arm.

Root cause: q=1 target-feature capture did not enter the normal fused-decode graph unless approximate fused verification was enabled. Exact prompt replay therefore used a numerically different dynamic layer path before speculation began.

The first correction routes q=1 feature capture through the already-enabled fused-decode graph while leaving q>=2 approximate fused verification behind DFLASH_DS4_FUSED_VERIFY.

Cubic then identified that the capture execution scope still forced mmvq_max_ncols=3 while native gfx1151 AR inherited the process ceiling LUCE_MMVQ_MAX_NCOLS=4. Follow-up commit 125b869 removes that capture-only MMVQ override and executes q=1 capture eagerly, preserving the native MMVQ/MMQ selection while avoiding the known gfx1151 width-4 HIP graph-replay fault. q>=2 graph policy is unchanged. No exactness check, rollback requirement, or fail-closed condition was weakened.

Validated revisions

  • current PR head: 125b869a6b90589f6aa1840f957425a2d8a40f86
  • last hardware-validated predecessor: 64205712b7824fd479fecd4cbdf84da751f4be3d
  • validated upstream/main: 4fa8937c69aa68658a14ec3cab16e1da22886dd0
  • original pre-qualification PR head: 626e51384e3282e597ada97ed08e7db5cb5e531e
  • current-main synthetic merge analysis: no conflicts

Source and focused validation

  • parser/environment/evidence unit tests: 17/17 passed
  • Python byte compilation passed
  • repository Ruff gate passed
  • git diff --check passed
  • relevant HIP CTest: 4/4 passed
  • HIP server unit suite: 344/344 passed, 0 skipped
  • relevant CUDA CTest: 3/3 passed
  • CUDA server unit suite: 355/355 passed, 0 skipped

HIP / gfx1151 validation

Hardware and toolchain:

  • Framework Desktop, AMD Ryzen AI Max+ 395
  • Radeon 8060S, gfx1151
  • Ubuntu 24.04.4, kernel 6.17.0-35
  • ROCm/HIP 7.2.4; HIP compiler 7.2.53211 / Clang 22

Results:

  • full HIP/gfx1151 build: 465/465 targets built
  • corrected HIP server SHA-256: 37f7a6aacb7c6dee5567205698f7a4eca2a2c2bf8ca7e68bd3eb28c7d76bb9ba
  • bounded issue-#516 cached ggml_set_rows trust gate passed:
    • 3 requests
    • 34 positions per request
    • 8 rows
    • width 512
    • cached-graph reuse compared with rebuilt-graph and CPU oracles
    • return code 0, no HIP error

This bounded pass qualifies the cached-graph evidence used here; it does not claim to resolve issue #516 generally.

Real-model reference-exact proof

Pinned artifacts:

  • target: DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf
    • size: 86,720,111,488 bytes
    • SHA-256: efc7ed607ff27076e3e501fc3fefefa33c0ed8cf1eff483a2b7fdc0c2e616668
  • DSpark drafter: DeepSeek-V4-Flash-DSpark-draft-Q4RMFP4-denseF16.gguf
    • size: 11,304,737,056 bytes
    • SHA-256: 48883d35b8a67ecfd2858a90e12a47d04cb5ac581acef868ca0f58544816f746
  • prompt SHA-256: 5801a972ad5f8d5297c5764f7c4f2f677872a9b50e41cf582945277cd21af179
  • seed: 1234
  • generated tokens per arm: 96

Target-only greedy versus DFLASH_DS4_SPEC_REFERENCE_EXACT=1:

WidthToken parityStepsMatched / offeredRejection observed
q=296/96 identical4946/49yes
q=396/96 identical3560/70yes
q=496/96 identical2867/84yes

Additional q=4 lifecycle evidence:

  • two sequential requests on the same server produced identical 96-token traces
  • both requests produced identical 67/84 matched/offered counters
  • timing traces covered ordinary positions and ratio-4 compressor boundaries
  • rejected proposals occurred in boundary-touching batches, including verify batches starting at positions 29 and 32
  • decoding continued with exact token parity after rollback
  • full snapshots were active

Parity is exact token-ID equality; plausible text and approximate float comparisons were not used as the oracle.

CUDA negative-control validation

Hardware and toolchain:

  • NVIDIA GeForce RTX 3090, 24 GB
  • compute capability 8.6 / sm_86
  • CUDA toolkit 12.0.140
  • driver 535.309.01

Results:

  • full CUDA build: 472/472 targets built
  • relevant CTest: 3/3 passed
  • server unit suite: 355/355 passed
  • Compute Sanitizer memcheck on test_deepseek4_unit: ERROR SUMMARY: 0 errors

CUDA is reported separately and is not presented as gfx1151 evidence.

Review and merge status

  • all seven existing review threads are resolved
  • the user waived an additional independent final-diff review for this update
  • Cubic passed on exact head 125b869
  • repository CI on this exact head and maintainer approval are still required before merge

Head 6420571 is locally hardware-validated. The narrow MMVQ-policy follow-up at current head 125b869 passed local static/diff checks but has not been rerun on gfx1151 because lucebox3 is currently in active use; a focused q=2 parity rerun remains required when the box is free. No performance claim is made, and this PR does not include #554 or the separate August-10 exact-prefill optimization.