feat(agents): the events route and projection into the transcript store

Merged
CH
5d ago
+17281-16

P09 of the plan "Bringing Codex or Claude Code Max plans into Luke". Builds on P06 (#4236, merged), whose bearer lookup this route reuses, and on P08 (#4255, merged).

What it does

Adds POST /api/agents/runner-events. A native harness's runner (@luke/agent-runner) posts its numbered event batches there, and the route writes them into the existing store writer the way eve's relay does.

  • Auth: the runner's bearer, hashed and looked up against conversations.runner_token_hash through P06's admitRunnerBearer. A wrong, missing or retired bearer gets a 401 before the body is read.
  • Pluggable input: claude-code-moments.ts reads SDK messages into harness-neutral moments (native-moments.ts). native-projection.ts, a pure fold, turns moments into BrainRunEvents plus a few other writes. P22 adds a Codex reader of the same NativeReader shape; it does not need to touch the projection.
  • What it projects:
    • Turns: one turn per launch. The turn opens on the runner's luke_runner_accepted, system/init, or the first output.
    • Steps and text: each model call is a step (keyed by message.id where present). Text is drafted by finished sentence from stream_event deltas and told whole when the step closes.
    • Tool calls and reasoning: tool calls are started and settled, and thinking blocks become reasoning.
    • Turn end: a luke_runner_end ends the turn. turn_ended settles it (failed if the last result errored), stopped cancels it, and failed/input_closed fail it with a detail.
  • Special messages:
    • result usage is summed onto the turn and written with TURN_ENDED, priced on the agent's model. That is today's meter: inputTokens counts cache reads and writes, as model-catalog.ts reads it.
    • compact_boundary goes to recordCodingAgentCompaction (compacted_at).
    • rate_limit_event rejected is kept, and a failed turn carries it in failure_detail (rate_limit_rejected five_hour resets …). limit_reached is P15's.
    • The runner's WAITING event (P05's Stop hook) leaves the turn running, so the agent reads as waiting, not ended.
    • system/init's session_id goes to conversations.native_session.
  • Idempotence: new columns conversations.native_sequence and native_projection (one additive migration). Both are read FOR UPDATE and saved in the same transaction as every write (native-projection-store.ts). Events at or below the saved sequence are skipped, so a duplicate or overlapping POST is a no-op. readNativeSequence(target) lets P11 fill the next launch's firstSequence.
  • Tool names: CLAUDE_CODE_TOOL in @luke/features/agent-tools holds Bash, Read, Edit, Write, NotebookEdit, Glob, Grep, WebFetch, WebSearch, TodoWrite, Agent and Skill, plus ScheduleWakeup and Monitor (the plan's waiting tools, which P00's stream uses). These are registered in CODER_TOOL_SET. mcp__luke__x is stored as x, and its answer is parsed as the tool's JSON. Unregistered names are skipped with a log line.
  • Subagents: messages with parent_tool_use_id fold under their Agent call. Only the call and its answer are stored.
  • Awaiting lines: luke_runner_accepted { uuid } (P08's definition in @luke/agent-runner/protocol) takes the awaiting row whose client id is that uuid into the turn. The writer's takeAwaitingLine gains this client-id form; the text form eve uses is unchanged.
  • Launch end: runnerEventsApp({ ended }) is a hook called once per newly applied END, after commit. The route wires it to P08's runnerEnded through sandboxAtRunnerEnd, which provides FetchHttpClient, skips a conversation with no session, and logs a failed stop by its tag rather than failing the POST. The sandbox then stops at its timeout or the Terminal-lease sweep. runnerEnded itself is covered by P08's tests.

Out of scope: rendering (P10), launching (P08), the MCP server (P06), Codex (P22).

Tests

  • runner-events-app.test.ts (real dialect), driving P00's hermetic stream as the runner posts it:
    • the stored transcript, part by part
    • the line taken into the turn
    • WAITING keeping the turn running, with sentence drafting midway
    • usage on the turn, and native_session recorded
    • a whole or overlapping replay changing nothing
    • wrong, missing or retired bearers refused
    • subagent folding, and stop → cancelled with ended told once
    • compaction, and the rate-limit failure detail
  • native-projection-store.test.ts: save and read back, rollback on failure, account isolation, cleared conversations.
  • Old transcripts read unchanged: the writer's text-form take and every existing transcript or tool-set test pass as before. The tool set only gains names.

Run locally: the new and P06 tests, tsc for service, database, web and agent-runner, oxlint, knip, discarded-effect, scripts/repository-checks.sh, drizzle-kit check, and the web rewrite tests. pnpm check:affected ran out of memory in the sandbox (parallel tsc), so I ran its pieces one by one; CI runs the full set.

The developer waived WORKFLOW.md's per-PR Preview/Mac runs for this rollout. No runtime edge or AGENTS.md change was needed.

Made with Luke


Devin Review