fix(llm): map StructuredOutputMode across SDKs and update docs (#917)

AD
3d ago
+247-222

Fixes #917

Summary of Changes

  • Protobuf Mapping: Mapped \StructuredOutputMode\ (\CONSTRAINED, \VALIDATION_ONLY, \REPAIR) into proto \StructuredOutputOptions.mode\ across Swift, Kotlin, Flutter, React Native, and Web bindings.
  • Enabled Constrained Decoding by Default: Updated SDK \generateStructured\ APIs to default to \constrained\ mode (which uses GBNF grammar sampling in commons) instead of throwing preflight unsupported exceptions or defaulting to \ alidationOnly.
  • Docs and Comments: Updated stale docstrings, comments, and references across Core C++ and SDK bindings to accurately reflect that constrained decoding is supported via llama.cpp sampler grammar on the LLM generate path.

Summary by cubic

Fixes #917 by mapping StructuredOutputMode (constrained, validationOnly, repair) into the proto StructuredOutputOptions.mode across the Swift, Kotlin, Flutter, React Native, Python, and Web bindings and making constrained the default for generateStructured. This replaces the previous preflight failure and validationOnly default. Constrained decoding now uses schema-derived GBNF grammar sampling via the llama.cpp sampler on the LLM generate path. SDK-side repair retry loops were dropped: repair is passed through as a proto mode, and generateStructured now forces tool choice to none. Comments and docstrings that called constrained decoding unsupported or not yet wired in were updated.

  • Kotlin LlmOptions gains a structuredOutput field whose mode overrides the generateStructured mode argument; strict is not forwarded to the proto.
  • The Python bridge exposes only constrained; other modes raise not_implemented.
  • The branch also picks up merged docs updates, including the rcli to wally CLI rename and corrected streaming-callback docstrings.

Summary by CodeRabbit

  • New Features
    • Structured output now defaults to constrained generation, using the supplied schema to guide output where supported.
    • Structured output settings let you choose an enforcement mode; settings provided in LLM options take precedence over the method’s mode argument.
    • Structured output options now include a mode across supported SDKs.
  • Behavior Changes
    • Structured generation no longer retries invalid results with a repair prompt. Results are extracted and validated after generation.