fix(agent): classify a content-filter refusal as MODEL_REFUSED

Merged
A2
7d ago
+180-9

What and why

  • A model call that finishes with unified reason content-filter and no text was reported as MODEL_PROVIDER_FAILED (model provider failed: No output generated.), category infrastructure, exit 3. generateText still returns that result, and reading output throws NoOutputGeneratedError, which drops the finish reason (packages/e2e/src/agent/model/sdk.ts).
  • The adapter checks the finish reason before reading structured output or parsing text. Unified content-filter, or a raw token of content-filter, content_filter, or refusal, raises MODEL_REFUSED (category test, exit 1, blocked category automation). The message names the step, the model, and the raw token when the result still carries one.
  • agent.extract does not enter its repair round for that error (packages/e2e/src/agent/invocation.ts continues only on ModelOutputInvalidError). agent.act classifies the same finish reason before the tool-choice and context-overflow retries (packages/e2e/src/agent/tool-loop.ts). On act, the AI SDK ToolChoiceViolationError keeps only the unified reason, so the message quotes content-filter.
  • Empty output with finish reason stop stays MODEL_OUTPUT_INVALID. A refusal is not model-unreachable, so the failed step counts as a failure of the flow recording (packages/e2e/src/agent/step-cache.ts).
  • The runner reruns test-category failures when retries is greater than 0, and it does not rerun infrastructure failures (packages/e2e/src/run/retry.ts). The refused call itself is one doGenerate: no transport retry and no repair round.

Verified

Ran it locally: yes

  • Command: pnpm --filter e2e exec vitest run tests/unit/agent-model-refusal.test.ts
  • On this diff (c8a70517afbcb5b7b52b71e53ed6c9b07eddd4beb8cce58e78cf28228c7d5aae, base 34b90ae647cf7989793c858e37e22ffb63594dc0): 5 passed.
  • With the fix files restored to that base: 4 failed, 1 passed. The test file imported and ran. The failures were the old error codes.
main (34b90ae)this branch
Structured extract and agent.extract: MODEL_PROVIDER_FAILED, infrastructure, model provider failed: No output generated.MODEL_REFUSED, category test, one model call, agent.extract: the model anthropic/claude-sonnet-5.5 refused the request (refusal)
Text call, raw reason content_filter: MODEL_OUTPUT_INVALIDMODEL_REFUSED, one call, message quotes content_filter
agent.act: MODEL_PROVIDER_FAILED (Model response did not contain a tool call even though tool choice was required.)MODEL_REFUSED, one call, message quotes content-filter
Finish reason stop and empty outputMODEL_OUTPUT_INVALID on both

Limitations

  • No live provider call. The finish reason is scripted in doGenerate.
  • pnpm test, pnpm check, and scripts/check-error-codes.ts were not run. packages/e2e/tests/unit/step-cache.test.ts was updated and was not part of the command above.
  • retries greater than 0 still reruns a test-category failure. Explore pins retries at 0. A refused planner call leaves the explore loop, as MODEL_PROVIDER_FAILED already did: PLANNER_SHORTCOMINGS is MODEL_OUTPUT_INVALID and ASSERTION_INCONCLUSIVE.

Checklist

  • Changeset added (pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only.
  • Breaking change: title carries !, the changeset body names what breaks and what replaces it, and the deprecation warning landed a window earlier (stability policy in CONTRIBUTING.md).
  • Docs updated in the same PR: the docs/**/*.mdx page for the behavior, and skills/e2e/ if the skill describes it.
  • Contract change: emitted .d.ts reviewed, tests/types/sdk-types.ts updated; wire change edits the schema and both fixtures.
  • Engine contract change (e2e/engine): changesets for e2e, @e2e-dev/web, and @e2e-dev/mobile.

MODEL_REFUSED is a new AgentErrorCode. The report schema already types code as a string, and tests/types/sdk-types.ts does not pin that union. dist/ is produced by the build.

AI-assisted development: an AI coding agent wrote this patch, and an independent AI review checked the diff, the new test, and the recorded vitest results.

AI-assisted development and review; submitted by Ryan Vo.

Validated against upstream commit 34b90ae647cf7989793c858e37e22ffb63594dc0.

Refs #945.