fix(chat): keep probing after the turn stops looking in-flight

Closed
SW
4mo ago
+174-3

Row 1 of chat#1923 — the original bug is live on prod. chat#1924 shipped the plumbing but the trigger never fires in the real failure mode.

Reproduced on prod after #1924 merged

Original prompt, prod 7a56bb65:

POST /api/chat19:13:42 → 19:15:45 (123 s), 1,498 chunks
Reconnect requestszero
Run kept writing until19:19:00 — 3¼ min later
Persisted server-side25 parts (6 step-start, 6 reasoning, 3 text, 2 tool-skill, 8 tool-bash)
UIfrozen, no stop control, composer idle

Root cause

shouldRecoverStalledStream gated on IN_FLIGHT_STATUSES = {streaming, submitted}. The failure ends the stream cleanly — [DONE] with no finish chunk — so useChat transitions out of in-flight and the interval stops evaluating. Everything downstream (frame counting, URL building, auth, the api route) was correct and simply never got invoked.

The earlier verification passed for the wrong reason. Its prompt was five sequential 45 s sleeps, which guarantees a stall while still streaming — a resume was already in flight at 17:10:28 before the stream died at 17:11:48. It never exercised the post-drop path. The original prompt is many quick tool calls, so nothing was in flight when the stream ended.

This is the case upstream open-agents covers with status === "ready" → probe. I reviewed that, called it redundant because our resume route self-probes via its 204, and dropped it. Wrong: the route cannot probe itself if nothing calls it.

Fix

A cut-off stream and a completed one are indistinguishable from the client — both end with [DONE]. So stop trusting the appearance and ask the server.

  • POST_TURN_PROBE_MS (5 min): after the in-flight → not-in-flight edge, keep evaluating under the existing cooldown.
  • 204 ends it. createChunkCountingFetch reports a 204 on the resume path — the server saying there is nothing to resume, the only authoritative end-of-turn signal. A normally-completed turn therefore costs one extra request, then stops.
  • Any chunk re-arms, so a resumed stream that drops again is caught.

The window is a backstop, not the control: the 204 is what normally ends probing. It bounds the pathological case where that answer never arrives.

This also subsumes the "reconnect fired before the stream died" waste flagged on #1924, since the trigger is no longer purely silence-based.

Tests

5 new cases on the policy, RED before GREEN — the first one reproduces the prod bug directly:

  • recovers after the turn leaves in-flight, while the window is open
  • stops once the window expires
  • never probes a turn that never ran (so idle chats don't poll)
  • stops once the server has confirmed nothing to resume
  • honours the cooldown on post-turn probes

Plus 2 on the fetch: reports no-active-stream on a 204 resume, and does not for a normal streaming response.

88 chat test files passing; tsc --noEmit clean in every touched file; eslint clean apart from the pre-existing useVercelChat 'authenticated' is assigned but never used, which is on main.

Verification owed

This must be verified against the original prompt on a preview before merge — not a sleep-based scenario. That substitution is exactly what hid the bug last time. Results will be posted here.

Refs chat#1923

🤖 Generated with Claude Code


Summary by cubic

Keep probing for a stalled stream after a turn exits in-flight, and stop immediately when the server returns 204. Fixes the prod bug where streams end with [DONE] (no finish chunk) and the UI freezes while the run continues.

  • Bug Fixes
    • Added a 5‑min POST_TURN_PROBE_MS window after leaving in-flight; respects existing cooldown.
    • Treat 204 from the resume route as the authoritative end-of-turn; createChunkCountingFetch signals no active stream and useStreamRecovery halts.
    • Track the in-flight → not-in-flight edge; any new chunk re-arms recovery. Idle chats never poll; probing stops once the server confirms done.
    • Tests cover post-turn probing, cooldowns, and the 204 path.