feat(decision): local decision models — ABI v13 + llamacpp + MLX + Swift surface

Merged
SI
1mo ago
+5202-136

Adds the decision primitive end-to-end: state + typed questions (choice / noul / score) scored in one joint forward pass, returning calibrated answers with probabilities and confidence.

Stack

  • ABI v13 (RAC_PRIMITIVE_DECIDE, decision_ops, promoted from reserved_slot_5 at the same binary offset) + idl/decision.proto and the commons service/component.
  • llama.cpp engine op (engines/llamacpp/rac_decision_llamacpp.cpp) — renders the model's bundled decision template, tags span tokens, reads one score per option. Verified on Clef-Flash-Q4_K_M: team=billing 0.985, 810 ms.
  • MLX engine op (engines/mlx) + Swift callback — joint-head runtime via the mlx-swift-lm fork. Verified end-to-end on macOS arm64 with clef-flash-4bit: team=billing 0.9737, refund(true) 0.9454, urgency 1.6997 (matches the reference within bf16 noise).
  • Swift surface — RunAnywhere.decision.decide(state:questions:) + CppBridge component bridge.
  • Lifecycle fix — RAC_PRIMITIVE_DECIDE cases in model_lifecycle.cpp (the ABI commit added the slot but not the load/cleanup/attach arms; without them loads failed with "Unsupported modality").
  • Pins — mlx-swift-lm 3.31.5 → 3.31.8.

⚠️ Draft: one cross-repo dependency

mlx-audio-swift 0.1.5 pins mlx-swift-lm exact 3.31.5, so SPM cannot resolve against 3.31.8 until that repo releases 0.1.6:

  • Proposed: RunanywhereAI/mlx-audio-swift#1 (pin bump to 3.31.8)
  • Needs someone with push on that repo to merge + tag 0.1.6. Then this PR adds the 0.1.6 pin and resolution goes green.
  • sdk-publish's checksum table and the runAnywhereMLXSwiftVersion literal here are hand-edited, per the repo rules.

Testing

  • ./run doctor / local native builds on macOS arm64.
  • Decision E2E harness: local clef-flash-4bit through RunAnywhere.decision.decide.
  • Tokenizer + span + probability parity vs the checkpoint's reference implementation (fixtures captured from the model author's script).

Not in scope

  • Vision/multimodal decisions, Android/Web decision paths, Wally CLI wiring.
  • NeuRT needs a matching ABI-13 release (tracked separately; stub fallback keeps public builds green).

Summary by cubic

Adds the decision primitive end-to-end: state + typed questions (choice / noul / score) scored in one joint forward pass, returning calibrated answers with probabilities and confidence.

What changed

  • ABI v13 promotes RAC_PRIMITIVE_DECIDE / the decision_ops vtable slot from the reserve, plus idl/decision.proto (IDL bumped to 1.5.0, SCHEMA_LOCK refreshed), a lifecycle component, and the RunAnywhere.decision.decide(state:questions:) Swift surface. Lifecycle now maps MODEL_CATEGORY_DECISION back from proto, and the Swift bridge reloads when the model path changes under the same id so stale checkpoints aren't scored.
  • llama.cpp gets a decision engine op (kernel pinned to runanywhere-b10453.6, now reported by the Swift/Kotlin/Dart version literals); MLX gets one on macOS arm64 via the mlx-swift-lm fork. mlx-audio-swift pins to the tagged 0.1.6, which carries mlx-swift-lm 3.31.8 — the previous revision pin no longer needed. RAC_LLAMACPP_HAS_DECISION is defined on the runanywhere_llamacpp carrier so its plugin entry actually lists DECIDE; the llamacpp plugin-entry test derives its expected primitive count from the vtable instead of hardcoding 4.
  • Adds prompt-format versioning (clef.decision.prompt_format_version, default 3): the served version is reported on results and a request pinning a different one is refused. Per-question softmax temperatures come from model metadata (<arch>.decision.temperature.<type>[.<bucket>]) instead of the hardcoded 1.0, with request override winning over bucket then type.
  • Fixes missing RAC_PRIMITIVE_DECIDE lifecycle arms that caused "Unsupported modality" on load.
  • Options render in canonical order per question kind (sorted, noul true-first, score level order) and scores remap to request order, closing the cross-engine parity gap.
  • Malformed requests are rejected instead of scored: NOUL option keys normalize to the reserved true/false pair, and duplicate or blank option keys, empty state, and wrong option counts per question type fail at the boundary via commons validation. Unset answer oneofs fall back to the answer's own type field instead of defaulting to .choice.
  • Adds tests/decision-cli: a Swift CLI to run the canonical ticket through either backend, smoke-test chat, and compare probability parity. Release builds pin the lifecycle dlsym symbols so the CLI doesn't dead-strip into "Decision model not loaded" errors.
  • Adds a linux-arm64 cpp-desktop kit (new CMake preset + workflow job; the only dispatch filter now fails the run on a typo instead of skipping every job green). The release train ships Linux kits (x64 + arm64) on ubuntu-22.04 — official releases previously shipped no Linux kit at all. Pins sherpa-onnx to v1.13.5-rac-desktop.5 so linux-aarch64 links glibc 2.35, and allowlists rac_decision_service.h on the deprecated-surface gate.
  • Pins NeuRT to v0.20.38, rebuilt against ABI v13 so the prebuilt archives no longer hard-fail on the API-version mismatch.
  • Stabilizes a Kotlin fan-out test that could fire an event before both collectors attached, installing a duplicate C registration.

Not in scope: vision/multimodal decisions, Android/Web decision paths, and Wally CLI wiring.

Summary by CodeRabbit

  • New Features
    • Added joint decision scoring for choice, yes/no/unknown, and score questions, including probabilities and confidence.
    • Added Swift APIs for submitting decision requests and retrieving answers, prompt-format version, and token usage.
    • Added a command-line tool to run, compare, and chat with local models.
    • Added Linux x64 and ARM64 desktop kit builds and release packages.
  • Improvements
    • Updated Llama.cpp and MLX runtime support and refreshed bundled model components.
    • Improved tokenizer compatibility with models using unrecognized tokenizer classes.