fix(runtime): carry typed pre-stream errors on QUIC response plane (#14750)

PR
1mo ago
+189509-93454

Overview:

QUIC (DYN_RESPONSE_PLANE=quic) could not carry a worker's ErrorType on a pre-stream refusal. The frontend therefore answered HTTP 500 and fault detection quarantined the worker that had correctly refused the request. TCP already maps the same refusal to 400 once the typed prologue exists; this PR extends that path to QUIC with a mixed-version Error-frame encoding.

Details:

QuicResponseSender sent FrameKind::Error as raw UTF-8, so ResponsePublisher::send_prologue_typed had nowhere to put a DynamoError. The requesting side re-synthesized ErrorType::CannotConnect with no cause, which the frontend maps to a sanitized 500 and is_inhibited treats as a worker fault.

This PR:

  • Adds StreamPrologueError and an optional typed_error on ResponseStreamPrologue (skipped when absent so an old sender still handshakes).
  • Encodes a typed QUIC error as JSON ResponseStreamPrologue. Untyped errors stay raw UTF-8. Receivers accept both: JSON with a present error field recovers the type; anything else, including a Python {"message","code"} envelope, is the legacy message. Protocol version stays 3 — a bump would force a lockstep upgrade for a payload the old decoder already accepts as UTF-8.
  • Implements send_prologue_typed on both QUIC and TCP so the worker's type survives whichever response plane is configured.
  • Attaches a non-migration-sensitive typed error as the cause of the existing outer CannotConnect, so HTTP classification sees InvalidArgument while retry classification of the outer type is unchanged.
  • Excludes InvalidArgument / Backend(InvalidArgument) from is_inhibited, so a clean client refusal does not quarantine the worker. An untyped pre-stream failure still does.

TCP untyped behaviour is unchanged: an old worker that sends only display text still yields 500 and is still treated as a worker fault.

Public API: StreamProvider<T> widens its error from String to StreamPrologueError. StreamPrologueError derefs to str, so err.contains(..), err.len(), and &err[..] keep compiling. This overlaps the runtime plumbing in #14467; this PR is self-contained against current main so QUIC can land without waiting on that review. If #14467 merges first, this rebases onto the shared types.

Where should the reviewer start?

  • lib/runtime/src/pipeline/network/quic_response.rs — encode_error_payload / decode_error_payload and send_prologue_typed
  • lib/runtime/src/pipeline/network.rs — ResponseStreamPrologue.typed_error and StreamPrologueError
  • lib/runtime/src/pipeline/network/egress/addressed_router.rs — pre_stream_failure_error
  • lib/runtime/src/pipeline/network/egress/push_router.rs — is_inhibited client-refusal exclude

Validation

  • cargo fmt --all -- --check
  • cargo clippy -p dynamo-runtime --lib --all-targets -- -D warnings
  • cargo test -p dynamo-runtime --lib pipeline::network::quic_response::tests — 26 passed (includes typed Error-frame round trip, live send_prologue_typed → InvalidArgument, and legacy UTF-8)
  • cargo test -p dynamo-runtime --lib pipeline::network::tcp::server::tests — 40 passed (TCP typed prologue survives; untyped/invalid-prologue paths unchanged)
  • cargo test -p dynamo-llm --lib test_pre_stream_refusal_surfaces_as_400 test_pre_stream_connect_failure_still_surfaces_as_500 — typed pre-stream refusal → HTTP 400 with the worker's reason; untyped / engine-shutdown still 500
  • typed_client_refusal_does_not_quarantine_the_worker — CannotConnect + InvalidArgument cause is not inhibited; untyped CannotConnect still is

Related Issues

🔗 This PR is linked to an issue:

  • Fixes #14750
  • Relates to #14467 (TCP typed prologue; this PR is the QUIC follow-up and is self-contained against current main)

AI disclosure

This change was drafted with assistance from Cursor Grok 4.6. I reviewed the wire-format compatibility, the quarantine/is_inhibited interaction, and the test results before opening the PR.

Open in Web Open in Cursor 

Summary by cubic

Carries the worker's typed error on the QUIC response plane so a clean client refusal gets HTTP 400 instead of HTTP 500 and is no longer treated as a worker fault. Untyped pre-stream failures still surface as HTTP 500 and quarantine the worker.

Note: this branch is far behind main and has not been rebased — the diff carries ~150 unrelated commits. Rebase before merging; only the QUIC typed-error work below belongs to this PR.

  • Typed QUIC errors travel as JSON tagged with dynamo.quic.typed_error; anything untagged (legacy UTF-8, Python {"message","code"} envelopes, decoy {"error","code"} objects) stays raw UTF-8 so it can't be misparsed as a prologue. Protocol version stays 3.
  • Client refusals (InvalidArgument / Backend(InvalidArgument)) are excluded from is_inhibited, and NON_MIGRATABLE is pinned to the same set so an attached cause never changes migration behavior.

Migration

  • StreamProvider<T>'s error type widens from String to StreamPrologueError; it derefs to str, so .contains(), .len(), and indexing keep compiling.