Fix [Agent] : message processing bug
why
There is currently a bug that occasionally happens when using agent specifically in dom mode, and passing messages from one run to another. The error Invalid prompt: The messages must be a ModelMessage[] is thrown due to two issues:
processMessageswas modifying tool outputs without verifying they conform to the expectedtype: "content"schema format- The
acttool was returningundefinedvalues in its result object, which is not valid JSON
what changed
-
packages/core/lib/v3/agent/tools/scroll.ts: AddedtoModelOutputmethod toscrollTool(DOM mode) to ensure it returns the{ type: "content", value: [...] }format consistent with other vision tools. -
packages/core/lib/v3/agent/utils/messageProcessing.ts: AddedisContentTypeOutputcheck before modifying any tool output. This ensures we only compress outputs that havetype: "content"(array-based values), preventing corruption oftype: "text"ortype: "json"outputs. -
packages/core/lib/v3/agent/tools/act.ts: FixedplaywrightArgumentsbeing set toundefinedwhen no actions exist. Now the field is omitted entirely instead, sinceundefinedis not valid JSON and breaks AI SDK validation. -
packages/core/lib/v3/types/public/agent.ts: AddedScrollToolResultinterface and updatedScrollVisionToolResultto extend it.
test plan
- Run agent in DOM mode with multiple steps on a task that previously caused the error
- Verify no
Invalid prompt: The messages must be a ModelMessage[]error occurs - Verify message compression still works correctly for screenshot/ariaTree/vision tools
- Run agent in hybrid mode to ensure no regression
Summary by cubic
Fixes intermittent “Invalid prompt: The messages must be a ModelMessage[]” in DOM mode by enforcing a consistent tool output schema and safer message compression. Stabilizes multi-step runs and keeps vision-tool compression intact.
- Bug Fixes
- scrollTool (DOM): added toModelOutput to emit { type: "json", value: { success, message, scrolledPixels } }.
- Message processing: added isContentTypeOutput; only compress outputs with type "content" and replace the whole output; leave "text"/"json" untouched; keep vision action text and drop media; use safe placeholders for screenshots/ARIA trees.
- act tool: omit playwrightArguments when no actions to avoid undefined in JSON.
- Types: introduced ScrollToolResult and made ScrollVisionToolResult extend it for a unified result shape.