fix(decisions): include request ID and HTTP status in JSON output

Merged
MU
21d ago
+968-144

Preserve decisions request ID and HTTP status in CLI JSON

Fixes RunanywhereAI/wally#186.

Problem

wally decisions --json previously printed only the service response body.
Scripts could not log the HTTP status or the x-request-id header, and a
refusal only produced a human error on stderr.

Changes

  • Keep existing result fields at the top level; add request_id and
    http_status to the CLI document.
  • Emit one JSON error document on refusal while keeping exit code 1 and human
    diagnostics on stderr. Usage errors still exit 2.
  • Carry the final decisions attempt's metadata through retries, session
    refresh, body-read errors, and decoding failures. Auth/pricing metadata is
    never substituted.
  • Use null metadata for local results and errors before any HTTP response;
    missing, invalid, or credential-echoing request IDs become null.
  • Render only parsed, safe error details. Future error enum values do not
    hide the message or turn into a fabricated error type.
  • Retain the existing string-error session/transport interfaces for unrelated
    callers. Update help, golden fixtures, and hermetic regression tests.

Example success (other response fields abbreviated):

{"object":"decisions","model":"eve","answers":{},"request_id":"example-id","http_status":200}

Example refusal:

{"error":{"message":"Invalid request","type":"invalid_request_error"},"request_id":"example-id","http_status":400}

Test Screenshots:

Success:
image

Failure:

image

Compatibility and ownership

This is a CLI output adapter, not a new service body schema. Existing scripts
reading .answers or .usage keep those fields. Byte-for-byte raw-response
consumers must adjust; service-schema validators should remove the two added
fields before validating. Local /v1/decisions serving is unchanged.

No SDK pin, generated binding, engine, or binary target changes. CMake remains
the build entry, and the Apple MLX host wiring is untouched.

The pinned decisions contract defines RequestId and declares the error
response header. Its explicit 200 response still lacks that header declaration;
declaring the optional success header in InferenceInfra is a contract follow-up.
The adapter's ID bounds and unchanged generated payload are tested against the
current pinned artifact/hash.

Validation

Passed on Windows x64 against pinned kit 0.20.39:

  • cmake --build build (see local dependency note below).
  • Focused Rust suites: 7 decisions tests, 9 contract tests, 47 account/session
    tests. All passed.
  • Expanded Python decisions mock suite and focused CTest help/decisions jobs;
    the account mock CLI job also passed in the full CTest run.
  • Both changed help golden fixtures match the built binary byte-for-byte.
  • Modelless smoke/e2e, including all three public backends. Ten model-backed
    modality tests skipped intentionally; no models were downloaded.
  • cargo fmt --check, contract generation/drift checks, version consistency,
    test ledger, and git diff --check.
  • One short production CLI invocation: HTTP 200, exact response ID present,
    91 tokens. A read-only usage query subsequently matched that ID to the
    served request. Ledger cost: 4 microdollars (the CLI estimate showed 3).

Limits, not a claim that all CI is green:

  • Full CTest: 4/6 jobs passed. The Rust job's only failures were two existing
    symlink tests denied by Windows privilege error 1314; its integration
    suites passed. The local harness job failed an unchanged missing-model text
    assertion after entering an interactive prompt (not installed versus
    expected not downloaded).
  • Strict Windows clippy fails on existing path_reload.rs dead code and
    drop_non_drop for the existing no-op Windows stdout guard. It passed with
    only those baseline lint categories allowed; no lint allowances were added
    to product source.
  • Agent-sync fails because this Windows checkout materializes CLAUDE.md as
    a regular file rather than its committed symlink. Neither skills tree nor
    the agent files changed.
  • Apple MLX host/full macOS golden execution was not available locally and
    still needs normal upstream CI. No Apple build wiring changed.

Local dependency note: the initial Windows link failed on the kit server's
existing OPENSSL_thread_stop dependency. Configure supplied an existing
local import library for Git's installed OpenSSL DLL via
CMAKE_EXE_LINKER_FLAGS; CMake staged the DLLs. This is build-tree-only and
not part of the product diff. Preview origins are reserved .example.test
hosts; the live check used the user's already saved production session.

Production budget was at most three actual inference requests, including
retries. Only one CLI inference invocation was made with --no-retry, bounded
to at most two POSTs if automatic 401 refresh was needed; one billed success
was observed. No further inference calls were made. Regression tests use
dummy credentials and loopback/injected transports.

Summary by cubic

Fixes RunanywhereAI/wally#186: wally decisions --json now includes request_id and http_status in the CLI document, and refusals emit a single JSON error document while keeping exit code 1 and human diagnostics on stderr.

Behavior

  • Add request_id and http_status to success and refusal JSON; existing fields like .answers and .usage stay put.
  • Header-derived metadata overrides same-named fields in the service body; unknown success extensions, explicit nulls, and numeric lexemes are preserved verbatim.
  • Metadata comes only from the final decisions attempt; retries, malformed bodies, and transport errors flow through unchanged, and a failed refresh retains the decisions metadata, not the auth response's.
  • Local results, pre-HTTP errors, and invalid or credential-echoing request IDs render null metadata.
  • Only contract-parsed, terminal-safe error details are output; future error enum values omit the type label instead of fabricating one.

Migration
Byte-for-byte raw-response consumers must adjust; service-schema validators should strip the two added fields before validating. Local /v1/decisions serving and the string-error session/transport interfaces are unchanged.

Summary by CodeRabbit

  • New Features
    • Decisions JSON output now includes a request ID and HTTP status when available, for both successful responses and errors.
    • Decision errors can include refusal details when safe to display.
  • Improvements
    • Successful JSON output preserves additional response fields, explicit nulls, and precise numeric values.
    • Error messages include HTTP status when available, and sensitive credentials are excluded from failure output.
    • The --json help text now describes the result and error details included in the output.