Fix [Agent] : message processing bug

Merged
TK
10mo ago
+96-43

why

There is currently a bug that occasionally happens when using agent specifically in dom mode, and passing messages from one run to another. The error Invalid prompt: The messages must be a ModelMessage[] is thrown due to two issues:

  1. processMessages was modifying tool outputs without verifying they conform to the expected type: "content" schema format
  2. The act tool was returning undefined values in its result object, which is not valid JSON

what changed

  1. packages/core/lib/v3/agent/tools/scroll.ts: Added toModelOutput method to scrollTool (DOM mode) to ensure it returns the { type: "content", value: [...] } format consistent with other vision tools.

  2. packages/core/lib/v3/agent/utils/messageProcessing.ts: Added isContentTypeOutput check before modifying any tool output. This ensures we only compress outputs that have type: "content" (array-based values), preventing corruption of type: "text" or type: "json" outputs.

  3. packages/core/lib/v3/agent/tools/act.ts: Fixed playwrightArguments being set to undefined when no actions exist. Now the field is omitted entirely instead, since undefined is not valid JSON and breaks AI SDK validation.

  4. packages/core/lib/v3/types/public/agent.ts: Added ScrollToolResult interface and updated ScrollVisionToolResult to extend it.

test plan

  • Run agent in DOM mode with multiple steps on a task that previously caused the error
  • Verify no Invalid prompt: The messages must be a ModelMessage[] error occurs
  • Verify message compression still works correctly for screenshot/ariaTree/vision tools
  • Run agent in hybrid mode to ensure no regression

Summary by cubic

Fixes intermittent “Invalid prompt: The messages must be a ModelMessage[]” in DOM mode by enforcing a consistent tool output schema and safer message compression. Stabilizes multi-step runs and keeps vision-tool compression intact.

  • Bug Fixes
    • scrollTool (DOM): added toModelOutput to emit { type: "json", value: { success, message, scrolledPixels } }.
    • Message processing: added isContentTypeOutput; only compress outputs with type "content" and replace the whole output; leave "text"/"json" untouched; keep vision action text and drop media; use safe placeholders for screenshots/ARIA trees.
    • act tool: omit playwrightArguments when no actions to avoid undefined in JSON.
    • Types: introduced ScrollToolResult and made ScrollVisionToolResult extend it for a unified result shape.