fix!: send reasoning_effort as reasoning.effort on flex Responses requests

Merged
CR
2d ago
+266-5

What

Since 0.11.0 (#48), text calls without completion_window="24h" go to flex /v1/responses. Two kinds of reasoning control were forwarded in a shape that /v1/responses doesn't accept:

  • responses.create(reasoning_effort=...), top-level or in extra_body, went out as a top-level reasoning_effort. A test asserted this as a supported extension.
  • chat_template_kwargs, thinking and thinking_token_budget in extra_body were copied onto the Responses request from both chat.completions.create and responses.create.

Today the gateway ignores these on /v1/responses, so the caller's reasoning setting is silently lost. control-layer#1999 (DW-974) makes the gateway refuse all four with 400 unsupported_parameter ("use 'reasoning.effort'"). The refused list comes from onwards/src/reasoning.rs (ApiSurface::Responses).

Change (Python and TypeScript, same behaviour)

  • reasoning_effort on the Responses flex path now becomes reasoning: {"effort": ...}. It merges with other reasoning fields (e.g. summary). If reasoning.effort is also set to a different value, the call raises instead of picking one. Chat calls already mapped reasoning_effort this way. A new end-to-end test covers that.
  • chat_template_kwargs, thinking and thinking_token_budget now raise client-side (ValueError / Error) before anything is submitted. The error names the field and points to reasoning_effort (chat) or reasoning={"effort": ...} (Responses).

Why raise, not translate

Mapping enable_thinking: false to reasoning.effort: "none" would be faithful for that one key. chat_template_kwargs can hold arbitrary template keys, though, and thinking / thinking_token_budget have no exact reasoning.effort equivalent. Translating only some of them means guessing effort levels, which changes behaviour silently. Raising is simpler and matches the gateway everywhere else:

  • realtime /v1/chat/completions already returns a 400 for these fields;
  • 24h batch uploads already refuse lines that contain them;
  • flex will refuse them after #1999.

The difference is that the error comes before a background job is created, and it names the replacement. Callers who already use reasoning_effort are unaffected.

The 24h batch path is unchanged.

Breaking change

Flex calls (any text call without completion_window="24h") now raise before submitting in two cases:

  • the request carries chat_template_kwargs, thinking or thinking_token_budget;
  • reasoning_effort conflicts with reasoning.effort.

Before this change, these requests were sent and the fields were silently ignored. After control-layer#1999 they'd fail with a 400 anyway. bump-minor-pre-major is set, so release-please bumps the minor version: Python 0.12.0 and TypeScript 0.7.0.

Tests

  • Python (python/tests/test_flex_polling.py):
    • The old test that expected reasoning_effort in extra_body now checks that the other extensions still go through extra_body.
    • New TestFlexReasoningControls, using public create() with only the upstream responses.create mocked:
      • top-level, extra_body and merged reasoning_effort all become reasoning;
      • a conflicting reasoning.effort raises;
      • each native thinking field raises on both the chat and Responses flex paths, and nothing is submitted;
      • chat reasoning_effort reaches the submitted Responses request as reasoning.effort.
  • TypeScript (typescript/test/client.test.ts): the same cases.

Run locally:

  • pytest tests/test_flex_polling.py tests/test_responses.py: 59 passed (Python 3.11, openai latest).
  • ty check: clean.
  • npm test: 57 passed.
  • npm run typecheck: clean.

Python 3.12 and the openai-version matrix are left to CI.

Known follow-ups (not in this PR; tracked in DW-981)

  • Flex tools without a description get a 400. dwctl's Responses tool type requires description: String (dwctl/src/inference/translation/responses/types.rs:557), but Chat Completions accepts tools without one. This is the default path for the PydanticAI, Agno, Microsoft Agent Framework, Agents SDK and Arize recipes. Confirmed in source, not run against a gateway.
  • autobatcher drops cache_control on content parts and assistant reasoning_content on the flex path. Neither translator references them. As a result, the cache_control option on ChatDoublewordAsync / DoublewordLLMAsync has no effect, and interleaved reasoning isn't replayed on tool turns.
  • Docs pages call this path "batch". The PydanticAI, Agno, Microsoft Agent Framework, Agents SDK and Arize recipes say BatchOpenAI() without completion_window="24h" "submits batch jobs", but it runs as flex. This came from a docs read on 21 Oct; I didn't re-check it here.