[STG-2463] fix: silence AI SDK "system message in messages" warning on agent.execute()

Merged
SH
3mo ago
+56-8

What & why

Every hybrid/DOM agent.execute() call prints this AI SDK v5 warning to the console:

AI SDK Warning: System messages in the prompt or messages fields can be a security risk because they may enable prompt injection attacks. Use the system option instead when possible. Set allowSystemInMessages to true to suppress this warning, or false to throw an error.

It's cosmetic (a warning, not an error), but it fires on essentially every agent run, and the "prompt injection" wording reads alarmingly enough to generate support tickets — it did (a customer reported it).

Root cause

The agent loop builds its system prompt as a system-role message inside the messages array (prependSystemMessage() in v3AgentHandler.ts) rather than the top-level system param. This is deliberate: it lets the system prompt carry Anthropic ephemeral cache-control via providerOptions (see the function's own comment). AI SDK v5 warns whenever it sees a system message in messages, since it can't distinguish a trusted, library-authored system prompt from untrusted input.

The committed lockfile pinned ai@5.0.133, which predates this warning — so the repo build never saw it, but the ^5.0.133 range means real installs resolve to newer 5.0.x (e.g. 5.0.209) that do warn. That's why users hit it and CI didn't.

Fix

  • Pass allowSystemInMessages: true to the generateText / streamText calls in the agent loop. This is the AI-SDK-sanctioned opt-out for an intentional system-in-messages structure, and it keeps prompt caching intact (vs. moving to top-level system:, which cannot carry per-message providerOptions and would regress Anthropic cache-control).
  • Bump the ai floor ^5.0.133 → ^5.0.185, where allowSystemInMessages is a typed option.
  • Add an explicit return type (LanguageModelV2) to getAISDKLanguageModel. The ai bump pulls a newer @ai-sdk/provider, and without the annotation tsc emits TS2742 ("inferred type … not portable").

E2E Test Matrix

Command / flowObserved outputConfidence / sufficiency
BEFORE — published @browserbasehq/stagehand@3.6.0 (resolves ai@5.0.209), hybrid agent.execute() on a real Browserbase session, console capturedAI SDK system-in-messages warning emitted: YES (1) — full "…security risk…prompt injection…" textReproduces the customer's exact symptom on the version they run.
AFTER — local patched build (this branch, same ai@5.0.209), identical hybrid agent.execute() flow, same model, real Browserbase sessionAI SDK system-in-messages warning emitted: NO (0); run exits cleanlyProves the fix removes the warning without breaking the agent run. Same harness as BEFORE → clean A/B.
pnpm turbo run build --filter @browserbasehq/stagehandTasks: 3 successful, 3 total (fails with TS2742 before the LLMProvider annotation)Typecheck + emit green; confirms the ai bump + annotation compile.
prettier --check + eslint on changed filesAll matched files use Prettier code style!, eslint cleanStyle/lint gates pass.

A/B model: anthropic/claude-haiku-4-5-20251001. Both runs used identical mode: "hybrid" + systemPrompt config (the customer's shape).

Closes STG-2463.

🤖 Generated with Claude Code


Summary by cubic

Silences the AI SDK v5 “system message in messages” warning during hybrid/DOM agent.execute() runs while preserving Anthropic cache-control. Addresses STG-2463.

  • Bug Fixes

    • Pass allowSystemInMessages: true to generateText/streamText so the agent’s system prompt in messages does not warn and caching stays intact.
  • Dependencies

    • Bump ai to ^5.0.185 (lock resolves to 5.0.209).
    • Add explicit LanguageModelV2 return type in getAISDKLanguageModel to avoid TS2742 with newer @ai-sdk/provider.