port: clef decision-model support + llama-clef-demo (ABI-compatible with b10453.4)

Merged
SI
18d ago
+1752-3

What

Ports the clef decision-model architecture onto runanywhere-b10453.4 and wires the joint decision head end to end, plus a CLI that exercises it.

A clef GGUF (general.architecture = clef) attaches a joint decision head: one forward pass over a prompt carrying a state + every typed question + every option scores all options at once. The head reads per-token span tags to pool the right runs, and the per-option scores come back as rows of the embeddings output.

Commits

  1. clef model-side port — LLM_ARCH_CLEF registration, arch/KV/tensor tables, llama_model_clef (Qwen3.5 backbone + joint head), rope/memory/graph routing, model saver opt-out, arch test, conversion/clef.py + gguf-py metadata.
  2. compat fixes for this base — arch-table duplicate removal, span array inits, decision-order enum in the public header, the missing build_qkv overload the head calls.
  3. llama_batch_set_decision_order — span array stays NULL after llama_batch_init and is allocated on first setter call. NULL means "no spans", which keeps plain generation on a decision model on the 0.0f status path instead of the all-zeros NaN path.
  4. llama-clef-demo — a CLI that runs a real joint decision: spans tagged, one llama_encode, one score per option, softmax → choice/confidence. Named demos + generic --question/--option.

Verification

Built with -DLLAMA_BUILD_TESTS=ON and run against Clef-Flash-Q4_K_M.gguf:

  • test-llama-archs lists clef; model loads all 587 tensors with no arch errors.
  • llama-clef-demo --demo → A (sqlite-vec) at p=0.92, exit 0.
  • Plain generation (no spans) still takes the clean path.

Notes

  • Model-side fidelity: src/models/clef.cpp and conversion/clef.py are the same content as the upstream clef support, adapted to this base's batch model.
  • The server-side decision endpoints are intentionally not ported; this base has no extended-batch framework and the SDK consumes the head through its own engine op.
  • Maple entries untouched; clef is a dense Qwen3.5-9B, no interaction expected.

Summary by cubic

Ports the clef decision-model architecture to this base and adds a CLI that exercises the joint decision head end to end.

A clef GGUF carries a joint decision head: one forward pass over a prompt with a state, every typed question, and every option scores all options at once, with per-option scores returned as rows of the embeddings output.

  • Registers LLM_ARCH_CLEF (Qwen3.5 backbone + joint head) with arch/KV/tensor tables, rope/kv/graph routing, and model-saver opt-out.
  • Adds conversion/clef.py and gguf-py metadata so Clef checkpoints convert to GGUF.
  • Adds llama_batch_set_decision_order() to tag batch entries with their span roles; the span array stays NULL until the first setter call, so plain generation on a decision model still takes the clean 0.0f status path.
  • Adds llama-clef-demo, a CLI that tags spans, runs one llama_encode, and reads back one score per option for softmax → choice/confidence.
  • Routes the clef output matrix through GET_ROWS weight-buft selection so the CPU repack buffer doesn't claim it and abort scheduling.
  • Fixes build failures: missing <algorithm>/<iterator> includes on libstdc++, positional llama_batch initializers missing the new decision_order member under LLAMA_FATAL_WARNINGS=ON, and POSIX-only popen/mkdir calls in the demo that broke arm64 Linux and Windows builds.

Server-side decision endpoints are intentionally not ported; this base lacks the extended-batch framework, and the SDK consumes the head through its own engine op.