port(ggml): in-place GDN state write + qwen dense decode recovery (lucebox-ggml #24, #27)
Merged
DU
3mo ago+122+122+012345678901234567890123456789-69-69-01234567890123456789
Ports two open ggml feature PRs from Luce-Org/lucebox-ggml into the in-tree vendored ggml (server/deps/llama.cpp/ggml/), on top of the vendorization from #486. Both were intentionally excluded from the initial snapshot (see VENDOR.md) and are brought in here.
Included
- lucebox-ggml #24 —
perf(cuda): add in-place GDN state write(2576c152)
Adds public APIggml_gated_delta_net_inplace, which writes the final recurrent state directly intosrc[5]viaop_params[1]. Threads astate_outparam through the CUDA GDN kernels. Defaultggml_gated_delta_netbehavior is unchanged. - lucebox-ggml #27 —
perf(cuda): recover qwen dense decode path(c3d44ed4)- Thread-local CUDA-graph props-check skip (
ggml_cuda_set_skip_props_check) to bypass per-step graph re-validation on the stable AR decode path. - Compile-time
WRITE_INTERtemplate specialization of the GDN kernels — the pure-AR path compiles out the intermediate-state store entirely. - Ampere grouped-cols default flip (env
DFLASH_GDN_FORCE_GROUPED_COLS/DFLASH_GDN_NO_GROUPED_COLS); grouped-cols is off by default on Ampere (sm_86 / 3090).
- Thread-local CUDA-graph props-check skip (
Notes
- The two upstream PRs overlap in
gated_delta_net.cu; applied as the union (each kernel/launch signature carries bothstate_outand theWRITE_INTERtemplate, with the runtimeskip_intermediateparam removed). - Scope is ggml only — no consumer changes under
server/src/. The new symbols are additive and unused by current server code. - Commits keep original authorship.
Verification
- All target symbols present in the vendored tree.
ggml.c,gated_delta_net.cu, andggml-cuda.cucompile clean under CUDA 12.6 / sm_86 using the existing build flags.- Diff limited to 4 files under
server/deps/llama.cpp/ggml/; zeroserver/src/drift.