fix(session): retry a failed session title on later turns and report why it failed

Merged
SA
2d ago
+526-54

Issue for this PR

Closes #1689

Field report after the 0.12.8 release: every new TUI session kept the New session - <ISO timestamp> placeholder title.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Why: a session whose first title request fails kept its placeholder title for the rest of its life, and nothing recorded why. ensureTitle ran only when the session had exactly one user message, so there was no second chance. Its log line carried the AI SDK's generic No output generated. Check the stream for errors., because result.text rejects with that and the provider's error reaches only the stream's error part. No telemetry event was emitted. The field report could not be answered from the user's log or from App Insights, and the title path itself is byte-identical from v0.12.7 to v0.12.9, so the trigger is in the user's provider or configuration.

What changed:

  • ensureTitle runs on every turn while the title still matches the placeholder pattern, up to the fifth user message, always with the first message as context. A transient failure heals on the next turn; a permanently broken small model stops costing a failed request after five.
  • The title call reads result.fullStream and records the provider's error in the failed to generate title log line (error, attempt, provider, model) and in a new title_generation_failed telemetry event: provider, model, attempt, error class name, keyword class, HTTP status when there is one. No message text.
  • TestLLMServer.title(...) queues answers for title requests so a test can make title generation fail. The automatic E2E Title remains the default, and the queue is separate from the body matchers because a title request carries the first user message.
  • Docs: title_generation_failed in the telemetry catalogue, and the models page now says what small_model drives and how a value the provider cannot serve shows up.

Not changed: the trigger on the user's machine. This makes the failure self-healing and observable; the cause is read from the next report's telemetry.

How did you verify your code works?

  • Review findings addressed in the second commit: a small_model the provider does not list is now reported instead of thrown (reproduced by the reviewer with zero log lines and zero events on the first commit), http_status is read from RetryError.lastError, non-Error error parts keep their name, one title request is in flight per session, TestLLMServer.reset clears the title queue, and the docs no longer say small_model drives compaction. Two tests cover the first and fourth of these.
  • New test/session/title-retry.test.ts (four cases) through the real prompt loop with the fake LLM server: a 400 on the first title request leaves the placeholder and emits one event with attempt: 1 and http_status: 400; the second turn retries with the first message as context and the retry's answer becomes the title; a title that fails on every turn is requested exactly five times and the sixth turn does not ask. On main the first test fails at the retry (one title request, placeholder kept).
  • test/telemetry/telemetry.test.ts (event-type completeness, 135 pass) and bun run typecheck clean.
  • test/session, test/cli/run, test/tui-journeys: 1,690 pass, 44 skip, 2 fail. The 2 failures are the two --time-budget cases in test/cli/run/json-output.test.ts, and they fail identically on main on this machine, run alone, so they are not from this change.
  • The 18 source-guard and telemetry files that read prompt.ts or the telemetry module (branding merge guard, release adversarial suites, memory injection, plan refinement, classify-provider): 663 pass, 0 fail.
  • The "All fibers interrupted without error" line printed between tests by this harness also appears on main for workspace-turn-boundary-loop.test.ts; it is not from this change.

Cost side effects

Only for a session whose title request failed; a session titled on the first turn makes the same single request as before. A failed session makes at most five title attempts (the first five user messages), and the SDK retries each attempt up to twice on retryable errors, so the worst case is 15 HTTP requests per session, each the size of today's single title request (the first user message). One title request is in flight per session at a time.

Screenshots / recordings

Not applicable.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

🤖 Generated with Claude Code