cmux NEXT: notify when a sandboxed Run ends, from one app-level waiter

Merged
TE
8d ago
+930-19

Leo's words: "Likewise, there's some stuff where we wanted to know what the state of affairs with cloud was. Like, we didn't with the ChatGPT work mode. We really, like, that, we want that to be over there. And then we also want to emulate that ChatGPT work mode pretty hard."

This is the notify step of that loop (hand off, remote run, progress, results, steer, notify, come back), for a sandboxed Run sent from a New Tab page (Labs > Show Preview Features, hq#3367).

What changes

  • One app-level waiter per Run (SandboxRunFollower, CmuxNextCloud). Before, each Sandboxed send polled /api/runs/:id/wait inside the pane's own send, under the send's deadline, and nothing survived a relaunch. Now the send hands the created Run to the follower and awaits that same long-poll. The follow:
    • outlives the tab that sent it, and the send's deadline;
    • is saved (UserDefaults; a Run is given up after 6 hours) and resumes after a relaunch for the signed-in user.
  • A native notification when the Run ends (SandboxRunNotices, the App). It posts through the app's existing DesktopNotifier, under the agent source's settings: notifications.desktop, quiet hours, per-source desktop override, sounds and their cooldown, Dock bounce. No banner posts while the person is looking at the Run's own tab, since its card shows the end.
    • Title: the task's first line.
    • Body: the outcome (Run finished, failed, timed out, cancelled). Localized in Runs.xcstrings; the table lands in #26537. Until then the English default shows.
  • A Run this app cancelled tells nothing. runs.cancel notes the id, even before the Run's follow starts, and the note is saved, so neither a race with the send's hand-off nor a quit before the end makes it notify. A Run cancelled elsewhere does notify, as "Run cancelled".
  • The server decides whose Run a follow is. A Run is followed under the account signed in at hand-off. If the create went out under another account (a switch during the send), the wait is refused (403 or 404) and the follow ends without a notification. A sign-out or account switch stops the follows of the other account's Runs.
  • A tab that answered early takes the receipt. If a Run ends while its send is still answering (the send's deadline), the send answers with the receipt. If the send had already answered (a lost wait), the Run's end replaces that answer for the page's next read, and the banner click opens the Run's card in a new tab, since that page may still show the old answer.
  • Exactly once across a crash. An end is saved untold, told, then marked told. An end saved but not marked told is told at the next launch, but only if its banner is not already in Notification Center.
  • A Run ends only with its receipt. A /wait answer that is no Run snapshot is unreadable and never ends the Run.
  • Click reveals the tab that sent the Run. If that tab is closed, the click opens a new workspace whose first chat is seeded with the ended Run's receipt (WorkspaceSpawn.firstChat with AgentPaneSeed.sandboxResult, the path Open Chat uses). That path also opens a window when none is open. The page's sandbox.state read then shows the Run's card, as the original tab did. The follower saves ended Runs (task, key, receipt) for a week, so a banner left in Notification Center still opens the card after a relaunch. When a card is dropped, its banner is withdrawn with it.

Stop conditions and cleanup (the observers this adds)

  • Each Run's poll task (in SandboxRunFollower.polls) ends when:

    • the Run is terminal (a receipt, or status succeeded, failed or cancelled);
    • the account that made it signs out, or another account signs in;
    • the server answers 403 or 404;
    • 6 hours have passed (the server caps a task at 12 minutes);
    • or the app quits (stop() from applicationWillTerminate).

    It is a long-poll, and the server holds each call up to 10 s. A server that answers at once is asked again no sooner than every 2 s, and failures back off up to 60 s. Waiters still out when a poll ends get an error, never a hang.

  • The account observer (SandboxRunNotices.account) is cancelled in stop() at quit.

The hand-off store

The watcher is generic on purpose. A follow-up PR adds a second source to it: the hand-off store's GET /v1/handoffs/:id/wait (cmux#26578). This PR is only the Run API's source.

No push

The Run's end reaches the app through the long-poll on /wait that the app already used. This PR builds no push infrastructure. A push would let a closed app notify too: today a Run that ends while the app is closed notifies at the next launch, once the session loads.

Not covered

  • "Needs the user": the Run API has no such state yet (queued, running, cancelling, succeeded, failed, cancelled), so nothing posts for it.
  • Runs created by the CLI (cmux run --sandboxed) are not followed: the CLI waits and prints the result itself.

Size

Size: about 950 counted lines, about 390 of them the tests that pin exactly-once delivery. Splitting the follower from its tests, or from its one caller, would land code without its proof. The strings are split out into #26537.

Capture

No capture: the app's own views don't change. The new surface is a macOS notification banner, posted when a sandboxed Run ends on the server, and the capture tours run no Cloud Run, so they can't produce it. theBannerReadsTheTasksFirstLineAndTheOutcome pins the banner's words. A click opens the existing Run card, which looks as it did.

Risk

Medium. The pane's Sandboxed send now waits through the follower instead of its own loop. The send's answers stay the same:

  • the receipt snapshot;
  • timedOut on a wait error or a malformed 200;
  • timedOut at the send's deadline (SandboxDeadline.race still bounds it).

The difference is that the follow goes on afterwards for the notification. Everything is behind Labs > Show Preview Features and the sandbox allowlist on the server.

Proof

  • SandboxRunNoticeTests (fleet package lane), with CloudAPIClientRunTests and AgentPaneSandboxRunTests:
    • one notice per terminal Run, also with two waiters, a late follow and a relaunch;
    • none for this app's own cancel, also when the cancel comes before the follow; one for a cancel elsewhere;
    • a body that is no Run snapshot is never an end;
    • an ended Run's card outlives a relaunch;
    • a cancel noted before a quit stays silent after the relaunch; another account's Run is not followed;
    • a send that answered before its Run ended takes the receipt, and so does one still answering;
    • an end saved but not told is told at the next launch, once;
    • 40 Runs in flight are all followed, and cards past their week take their banners;
    • a failed wait answers the send timedOut and the notice still comes;
    • a relaunch resumes the signed-in user's Runs; sign-out and an account switch stop them;
    • the banner's words;
    • a restored page shows the ended send as its own.
  • Release compile through the fleet.

Agent session: agent session

šŸ¤– Generated with Claude Code