feat(api): Slack agent foundation — shared handler, execution guard, thread window, model and output fixes

Merged
AV
2mo ago
+2384-794

Links

Summary

  • Phase 0 of the Slack agent plan: one handler for mentions and assistant DMs, an execution guard so a message's tools run at most once, the thread window fixed to the newest replies, and the model loop made honest about truncation, refusal and time limits.
  • Output moves to Markdown blocks with a channel placeholder for progress; images are capped where the API caps them; job routes get a duration budget.
  • No schema changes. The guard reuses ingest.webhook_events under a slack-agent: key prefix.

Why / Context

The current agent is an @-mention bot with a stateless per-message loop and a set of bugs found in review: thread context read the oldest 20 replies, stop reasons went unhandled so half-answers posted as done, Slack retries produced duplicate runs and side effects, the image cap was 120 MB against a 5 MB API limit, and the two handlers were 300-line near-duplicates. The plan's later phases (channel sessions, thread-scoped working sessions, plugin tools, memory) multiply message volume and blast radius, so this foundation lands first.

How It Works

  • events/process-agent-message/ is the single handler. It claims execution first via events/utils/agent-delivery/ (insert-or-nothing on (provider, event_id)), then runs preflight, the agent, and the reply inside one 240s budget. A duplicate claim exits silently; a pending claim older than six minutes is flipped to failed and the user gets a "lost track" notice instead of a replayed run.
  • run-agent.ts walks conversations.replies pages bounded by the trigger's ts and keeps the newest 20. Sonnet 5 is the default with adaptive thinking on 4.6+ models, a cache breakpoint on the static system prompt, 120s per model call with one retry while the budget allows, and static copy for refusal / max_tokens / turn cap / deadline via SlackAgentError. Full assistant content (thinking and server-tool blocks) is replayed on continuation. A curated tool allowlist is enforced at discovery and execution; MCP isError results go back as errors and never become action records.
  • Progress: DMs use assistant.threads.setStatus; channels post a placeholder that is updated with tool status and deleted after the final reply, since that API is DM-only. The final reply is a new message in a Markdown block, split at Slack's 12k limit.
  • Events route: QStash deduplicationId: event_id, 503 when publishing fails so Slack retries, edited and deleted messages filtered, file shares kept.

Manual QA Checklist

Not yet run against a live Slack workspace; the checklist below is what a preview deploy with a dev Slack app should confirm.

Channel mention

  • 👀 appears, "Thinking..." placeholder posts and updates while tools run
  • Final reply arrives as a new message rendering Markdown; placeholder and 👀 are removed
  • Thread with more than 20 replies answers from the newest context

DM

  • Typing status shows and clears; no placeholder; reply renders

Guard and limits

  • Same delivery twice: second run posts nothing; one slack-agent: row in ingest.webhook_events
  • Aged pending row + redelivery: "lost track" notice, indicators cleared
  • Ask it to delete a task or run a terminal command: refuses, row ends processed
  • Image over 5 MiB: clear size message
  • Second model call of a run shows non-zero cache_read_input_tokens

Testing

  • bun test src/app/api/integrations/slack in apps/api: 46 pass
  • bunx tsc --noEmit -p apps/api/tsconfig.json: clean
  • bunx biome check apps/api/src/app/api/integrations/slack: clean
  • bun run check:i18n reported clean by the implementing agent; no catalog changes

Design Decisions

  • Claim before preflight: a worker killed during image download is then recognised by the retry instead of re-running blind; preflight errors post to the user rather than relying on a retry.
  • Static copy over the Haiku rewrite for stop reasons: the rewrite prompt is for raw API errors; feeding it instructions produced unpredictable text and an extra call.
  • webhook_events over a new table: the unique index already gives insert-or-nothing semantics, the Linear sweep filters by provider, and retention prunes by age, so no migration was needed for Phase 0.

Known Limitations

  • Not an exactly-once outbox: tool results and reply bodies are not persisted, so an unsent reply after a crash is reported, not recovered. The persistent-session phase owns that.
  • Channel progress is a placeholder message, not a native typing indicator; Slack has no channel equivalent of the assistant status today.
  • tasks_update, deletes, terminals_send and automation mutations stay unavailable until the approval flow exists.

Follow-ups

  • Phase 1: thread-scoped sessions with follow-up queueing, Linear/GitHub plugin tool bridge, completion callback into the thread.
  • Phase 2: channel session with the four-outcome reply decision, back-off, channel memory, Approve button.

Risks / Rollout

  • Risk: behaviour change for every Slack mention and DM; the loop budget and guard are new paths against production Slack and Anthropic.
  • Rollout: deploy the API; no flags, no migration, no config.
  • Rollback: revert the commit. The slack-agent: rows in ingest.webhook_events are inert to every other reader and age out under retention.

https://claude.ai/code/session_01YDDSQYG6WSZn4jXWTR3Pid


Summary by cubic

Mentions and assistant DMs previously ran through two near-duplicate handlers with no execution deduplication, oldest-first thread context, and unhandled model stop reasons. This Phase 0 change consolidates them into one handler with an execution guard, a newest-reply thread window, and clear handling of truncation, refusal, and time limits.

What changed

  • Claims each delivery once; duplicates exit silently and stale claims post a "lost track" notice.
  • Thread context walks all reply pages, keeps the newest 20, and stops at the trigger's timestamp.
  • QStash publishes use event_id for deduplication, return 503 when publishing fails, and filter edited and deleted messages.
  • Replies render as Markdown blocks split at Slack's 12k limit; channels get a temporary progress placeholder and DMs use typing status.
  • Image uploads are capped at 5 MiB per file and 20 MiB total.
  • Default model is now Sonnet 5 instead of Sonnet 4.6.
  • Job routes get a 300s duration budget and return 400 for invalid JSON.

Rollout

  • Deploy the API; no flags, migration, or config.
  • The guard reuses ingest.webhook_events under a slack-agent: prefix; those rows are inert to other readers and age out under retention.
  • This changes behavior for every Slack mention and DM; rollback is redeploying the previous build.

Summary by CodeRabbit

  • New Features
    • Slack mentions and direct messages now share a consistent agent experience.
    • Sonnet 5 is now the default Slack model.
    • Long Markdown responses are automatically split into Slack-compatible messages.
    • Thread context, progress updates, images, mentions, and completed actions are handled more reliably.
  • Bug Fixes
    • Duplicate or delayed Slack events no longer trigger repeated responses.
    • Invalid requests and undeliverable messages now receive clearer handling.
    • Image downloads are subject to size, count, and timeout safeguards.