feat(chat-workflow): add POST /api/chat/{chatId}/stop

Merged
AR
4mo ago
+2064-22

Summary

Adds POST /api/chat/{chatId}/stop to stop the in-flight workflow for a chat — cancels the Vercel Workflow run behind chats.active_stream_id and releases the slot via CAS. Auth + session-ownership gating mirrors /api/chat/workflow; pending-<uuid> placeholders are released without a cancel call (clearing the slot makes the in-flight start() self-cancel).

Test plan

  • Stopping a chat with a live run returns { success: true, stopped: true } and the stream halts
  • Stopping a chat with no active run returns { success: true, stopped: false }
  • 403 for a chat owned by another account; 404 for an unknown chat

Summary by cubic

Adds POST /api/chat/{chatId}/stop to stop an in‑flight chat quickly, preempt the agent mid‑step, and close the SSE so the UI and DB stay in sync. Stop responsiveness is faster by polling cancellation every 150ms.

  • New Features

    • POST /api/chat/{chatId}/stop (+ OPTIONS CORS): cancels live runs or clears pending-*; waits up to 8s for a terminal status; returns 504 on timeout and 502 on cancel failure; clears the slot via CAS on success (best‑effort).
    • Real‑time preemption: polls getRun().status to abort streamText via an AbortController, passes the signal to pipeTo, and backstops tools with wrapToolsWithAbort so hung tools don’t stall the run.
    • Faster SSE close: wraps the workflow readable to poll status and close promptly on terminal states, synthesizes tool-output-error for unclosed tool‑calls (emitting tool-input-available first if needed), and forwards client disconnects to getRun().cancel().
    • Faster stop: cancellation polling now runs every 150ms (was 750ms) for snappier aborts and fewer wasted tokens.
  • Bug Fixes

    • Robust abort handling: detects runtime‑closed streams, classifies user‑stops correctly, and guards late finishReason rejections; on abort re‑persists the assistant message with closed tool‑call parts (errorText: "Cancelled") so reload doesn’t spin.
    • Validation and auth: 400 for bad chatId; 403/404 mirror /api/chat/workflow ownership checks.
    • Billing and commits: charge credits when a response message exists; skip auto‑commit on user‑stop.
    • Refactor: extracted waitForTerminalRunStatus, trackToolCallChunk, finalizeAbortedAssistantMessage, and pipeWorkflowStreamWithStopDetection into focused modules (no behavior change).

Summary by CodeRabbit

  • New Features

    • Add API endpoint to stop active chat workflows; responses indicate whether the workflow was stopped.
    • Streaming and agent execution now support real-time cancellation so in-progress assistant/tool work can be preempted.
  • Bug Fixes

    • Request validation, authentication and ownership enforced for stop requests.
    • Improved cancellation handling: waits for terminal run status, surfaces cancellation failures, and best-effort clears active stream markers.