convert: pplx-decider v1.2 joint checkpoints
Merged
SI
4d ago+96+96+01234567890123456789-3-3-0123456789
Converts pplx-decider v1.2 checkpoints, which declare protocol v3 (joint-question-inline-readout-v1) in joint_config.json, to GGUF. v1 conversion doesn't change.
For a v1.2 checkpoint the converter writes:
qwen35.attention.causal = false. The full-attention layers see the whole sequence, asattention_mode: noncausal_full_attentionrequires. The runtime already reads this key and masks only the full-attention layers, so there's no runtime change.decision.type = pplx-decider-joint, plus the marker (<|fim_suffix|>), the 255 answer codes and the special tokens to escape in user text. The codes and escapes are stored as JSON strings because llama.cpp leaves array keys out of the model metadata it exposes.- The joint
systemonetemplate, with one numbered question and one marker per question. - The readout as a padded LM head, as for v1.
It refuses other protocol versions, a non-zero readout_offset and other attention modes.
Tested on perplexity-ai/pplx-decider-flash-v1.2:
- The rendered prompt is byte-identical to the checkpoint's own chat template.
- BF16 logits at the markers match the reference to within 0.02 on Metal and CPU.
- Q4_K_M has no flipped answers, worst probability drift 0.14.
The Q4_K_M is published at runanywhere/decision-models-stable-fork (gguf/pplx-decider-4b, commit cb2ff56). runanywhere-sdks#1145 runs it.
Summary by cubic
Adds GGUF conversion support for pplx-decider v1.2 joint checkpoints, which declare protocol v3 (joint-question-inline-readout-v1) in joint_config.json. v1 conversion doesn't change.
Details
- Writes a joint
systemonetemplate (one marker after each question), a padded LM-head readout, and metadata for the marker, answer codes, and escape tokens. - Sets
qwen35.attention.causal = false; only the full-attention layers are affected, so runtime behavior is unchanged. - Refuses other joint protocol versions, non-zero
readout_offset, and attention modes other thannoncausal_full_attention. - Tested on
perplexity-ai/pplx-decider-flash-v1.2; rendered prompts are byte-identical, BF16 logits match within 0.02, and Q4_K_M shows no flipped answers.