fix(session): cancel only the timed-out prompt

Merged
GO
1mo ago
+1262-49

A timed-out request should stop without killing a newer request or leaving that newer request stuck. This adds a prompt-specific abort endpoint for the watchdog in andytown #28290, part of REPL-31317.

What changes

POST /session/:sessionID/prompt/:messageId/abort distinguishes a prompt being prepared or queued from the model call actually running. Canceling a preparing or queued prompt leaves unrelated model and shell work alone. Canceling the hung active prompt starts its already-admitted replacement. A late cancellation cannot stop a replacement that has started.

Each admitted prompt keeps its own cancellation token, so a third request cannot hide an earlier queued request. Canceled queued input stays out of the model context across automatic handoff, including when the next prompt is still being prepared. Superseded preparation cannot reintroduce its persisted input.

Context-only (noReply) messages do not take cancellation ownership. Completed and canceled ownership is cleaned up, and explicit loop resumes continue to work after an abort.

The runner controls terminal notifications for work it owns. Prompt-specific cancellation saves the aborted assistant message but suppresses session-wide terminal notifications that could end a newer request. Direct processor callers retain their existing error/idle behavior, and normal session-wide Stop retains its idle notification.

A prompt admitted after the final history read starts after normal completion. An explicit loop resume without a live prompt token cannot be stopped by a stale prompt-specific cancellation. Normal completion and session-wide Stop still emit their idle notifications.

Before a model step claims its input snapshot, it rechecks whether any included prompt was canceled. An invalid snapshot is reread; once an earlier input is consumed by a newer request, canceling that earlier input returns false. Three gated tests cover selected input, earlier context, and cancellation after the claim.

The SDK adds session.abortPrompt; existing abort callers keep their API.

How to review this PR

  1. Follow registerPrompt through preparation and admission. A token is not proof that its model call is running.
  2. Read cancelPrompt with Runner.cancelIf. Only the matching active model call is interrupted; queued cancellation excludes that message from the current drain.
  3. Follow A running with B queued and C arriving. Canceling B preserves A. Canceling A starts the latest admitted replacement; that replacement retains A’s context but excludes canceled B. Read the tests that inspect the actual model request for both queued and still-preparing C.
  4. Check manual Stop: it invalidates every queued token, so delayed admission cannot restart stopped work.
  5. Follow a prompt arriving after the final history read, then an explicit loop resume after cancellation. The first must run; the second must survive a stale abort. Normal completion still sends its idle event.
  6. Check the shell handoff, ownership cleanup, and processor interruption context. Then read the lifecycle regressions beside those branches.

Testing Done

  • HTTP coverage, authentication, and behavior gates each passed all 214 scenarios, with no missing or skipped routes. The prompt-specific abort route is registered in the exerciser.

  • 163 tests passed, 1 existing skip across prompt, processor, runner, and both HTTP test suites. All 30 package typechecks passed during push.

  • Built all four supported binaries; macOS version smoke test passed. The disposable Daytona sandbox verified the Linux release checksum and version.

  • On a real Daytona sandbox: held A's model response open, admitted B, then released A's delayed cancellation. A stopped, B streamed text, and a repeated A cancellation returned false without stopping B.

  • On that sandbox: aborting a context-only message preserved the running model request; a later request still streamed; aborting a context-only message during a shell command let the shell finish.

  • Real sandbox before/after: .3 returned false for canceling B after C arrived and sent B to C’s model call. .4 returned true, preserved A until its own cancellation, streamed C, and excluded B from C’s model input.

  • Final .5 real-sandbox check: normal completion emits idle, normal completion starts queued work, stale aborts preserve replacements, and the three-prompt/context-only/shell regressions still pass. The precise completion-window and explicit-resume races have deterministic runtime regressions.

  • Fresh independent review of the final ownership and interruption fixes found no remaining blocker.

These tests use the actual sandbox binary and HTTP/SSE transport with a controlled model endpoint. They do not exercise an external model provider.

The local .6 build passed another production-snapshot Daytona run: normal idle and queued handoff, three-prompt cancellation with excluded model input, context-only input, shell preservation, and repeated stale cancellation. The exact snapshot-claim interleavings are covered by deterministic tests. The sandbox was deleted after QA.

Release and rollback

Opt-in prerelease 1.18.32-pr28290.5 is paired with the coordinator SDK and sandbox pins in andytown #28290. The newer snapshot-claim fix is built and tested as 1.18.32-pr28290.6, but publication is still pending; .5 does not include that latest fix. Publish .6 and update the downstream pins before merging. Stable latest remains unchanged. Revert the paired downstream code and pins to roll back. Ownership is process-local, like the runner; unknown IDs are no-ops.