fix(workflows): one workflow step per LLM call so runs stop blowing the 800s ceiling

Merged
SW
3mo ago
+828-617

Fixes the duplicate task emails and the failing scheduled runs in chat#1918 by giving the agent loop real step boundaries.

The bug, from the runtime

runAgentStep wrapped the entire agent loop in a single "use step" via stopWhen: CHAT_AGENT_STOP_WHEN (stepCountIs(111)), so one step ran 11-25 minutes. WDK deploys step handlers with maxDuration: max, and on Pro that resolves to 800 s (Vercel duration limits) — not unlimited. Vercel terminated the invocation, the step queue redelivered it (retryAfterSeconds: 5), and each retry was a complete agent run that mailed the customer again.

Step "step//./app/lib/workflows/runAgentStep//runAgentStep"
exceeded max retries (4 retries)        code: USER_ERROR

Step record on wrun_01KYY8X6WKMHQ8DN5P8JQP2H4T: status: failed, attempt: 5, 09:01:31 → 10:13:55 = 72.4 min, error {"message": "Unknown error"} — no exception and no stack, which is what a platform kill looks like. Every other step in that run completed on attempt 1 in under 0.2 s.

45 of the last 100 runs failed this way, 42 of them at 72.4-73.5 min (5 attempts × ~870 s). Completed runs sit at a 4.0 min median — interactive chat turns finish inside 800 s and were never affected. Full evidence in root-cause note v3.

The fix

One journaled step per LLM call, with the loop in the workflow body.

FileChange
runAgentStep.tsDrops stopWhen. The AI SDK default is isStepCount(1) (@default isStepCount(1) in ai@6.0.190), so a step is now one model call plus that call's tool executions. Takes modelMessages / originalMessages, returns responseMessages.
runAgentWorkflow.tsOwns the loop. Appends each iteration's responseMessages so iteration N+1 sees iteration N's tool results; bounded by CHAT_AGENT_MAX_ITERATIONS; breaks on abort or any finish reason other than tool-calls.
sendStreamStart.ts / sendStreamFinish.tsNew. The turn's stream envelope, emitted once at workflow level.
convertMessagesStep.tsNew. Conversion runs once, journaled, before the loop.
buildMessageMetadataCallback.tsOptional seed so usage/cost totals span the whole turn instead of resetting per iteration.
lib/chat/const.tsAdds CHAT_AGENT_MAX_ITERATIONS = 111.

The streaming objection, and why it does not block this

The concern was that per-iteration steps break streaming and per-turn persistence. They do not, and the reference implementation (vercel-labs/open-agents apps/web/app/workflows/chat.ts) shows the shape:

  • Each iteration passes sendStart: false, sendFinish: false to toUIMessageStream, so only the workflow body emits the envelope. Without this the client would render N assistant messages instead of one.
  • Every iteration writes to the same writable. WDK streams are Redis-backed, and per foundations/streaming.mdx: "Stream locks acquired in a step only apply within that step. This enables multiple writers to write to the same stream concurrently."
  • originalMessages threads the in-progress assistant message into each iteration, so responseMessage stays cumulative and each persist overwrites one row.

pipeWorkflowStreamWithStopDetection already passed preventClose: true, so the shared writable already survived a step returning — no change needed there.

Two deliberate calls

  • CHAT_AGENT_STOP_WHEN stays. getGeneralAgent (the non-durable /api/chat route) still runs its tool loop inside streamText. Only the workflow path stops using it.
  • modelMessages is passed as a snapshot ([...modelMessages]), not the live array. A durable step input must describe the conversation as it was at that call, unaffected by later appends. A test caught this.

Tests

app/lib/workflows/__tests__/runAgentWorkflowLoop.test.ts (new, 5 tests) — written first, confirmed RED (4 failing), then GREEN:

  • keeps iterating while a step finishes on tool-calls, stops on stop
  • threads each iteration's responseMessages into the next iteration's modelMessages
  • emits exactly one stream start and one stream finish across a multi-iteration turn
  • stops looping when a step reports the user aborted, even on tool-calls
  • bounds a runaway loop at CHAT_AGENT_MAX_ITERATIONS

Plus 3 new cases in runAgentStep.test.ts: stopWhen is absent (regression guard — if it creeps back the step regrows past 800 s and this bug returns), sendStart/sendFinish are false, and responseMessages is returned. The obsolete prepareStep cacheControl test is replaced by one asserting cacheControl on the messages handed to streamText.

  • Full api suite: 4,319 tests / 795 files passing (re-run against a clean pnpm install --frozen-lockfile).
  • pnpm build: TypeScript step passes; the local run then stops at page-data collection on env vars the worktree lacks (STRIPE_SK), which CI has. Vercel check is green.
  • tsc --noEmit: 203 errors, of which exactly one touches a file in this PR — runAgentWorkflow.test.ts Property 'sandbox' does not exist on type 'never', which is pre-existing on main (line 315 there, 333 here after this diff). No new type errors.
  • eslint clean on all touched files.

Verification still owed before merge

Preview verification against a real run is not done yet and is the gate for this PR: start a long POST /api/chat/runs, then confirm via npx workflow inspect steps --runId=<id> --env preview that every runAgentStep record is attempt: 1 and under 800 s, that the chat renders one assistant message, and that exactly one email_send_log row lands. Results will be posted as a comment here.

Refs recoupable/chat#1918

🤖 Generated with Claude Code

Summary by cubic

Split the agent loop into one workflow step per LLM call to keep each step under Vercel’s 800 s limit and stop duplicate customer emails. Streaming now matches upstream open-agents and preserves tool-call parts across iterations. Fixes recoupable/chat#1918.

  • Bug Fixes

    • Scheduled runs stay under 800 s per attempt; no duplicate emails.
    • Tool-call parts persist across iterations; the UI renders one assistant message per turn via a hoisted stream envelope.
    • Robust abort handling: treat aborted streams as user stops and close open tool-calls on abort.
    • ai@6 path threads via result.response.messages.

  • Refactors

    • Moved the loop to runAgentWorkflow with CHAT_AGENT_MAX_ITERATIONS; runAgentStep runs one model call (no stopWhen).
    • Dropped the outer createUIMessageStream; write result.toUIMessageStream(...) parts directly to the shared writable with sendStart: false and sendFinish: false.
    • Persist from the workflow via persistAssistantMessageStep; runAgentStep no longer takes chatId. Seeded buildMessageMetadataCallback from originalMessages so usage/cost totals carry across iterations.
    • Added convertMessagesStep, sendStreamStart, sendStreamFinish; removed pipeWorkflowStreamWithStopDetection and finalizeAbortedAssistantMessage; added isAbortError and isRunCancelled.
    • Step input is { modelMessages, originalMessages }; step returns responseMessages for threading into the next iteration.

Summary by CodeRabbit

  • Improvements
    • Improved assistant responses involving multiple tool-use steps, with smoother continuation across the full interaction.
    • Streamed replies now begin and finish cleanly as a single response, reducing interruptions during multi-step processing.
    • Usage, cost, and completion details remain accurate across extended assistant interactions.
    • Added safeguards to prevent excessively long automated processing loops.
    • Improved handling of interrupted or cancelled assistant runs for more reliable completion behavior.