remove unnecssary value param from fillform tool in dom mode

TK
8mo ago
+8-13

why

The fillForm tool (DOM mode) had a value field in its input schema that the LLM would populate, but it was never actually used during execution. The execute function only read f.action to build the observe instruction — the value was completely ignored. This meant the tool relied entirely on the action string (e.g. "type foo into the email field") to carry both the target and the value, while value was dead weight.

This was also a problem for the variables feature: the schema told the LLM to use %variableName% syntax in the value field, but since value was never consumed, variable substitution through it could never work.

what changed

  • fillform.ts: Removed the value field from the input schema. The action field now explicitly guides the LLM to include the value to type within the action description (e.g. "type foo@bar.com into the email field"). When variables are available, the description shows %variableName% syntax with an example. This aligns the schema with how the tool actually works — observe parses the action string and returns Action objects with the correct method, selector, and arguments, which act then executes.
  • agentSystemPrompt.ts: Removed the stale reference to value in the variable usage instructions for DOM mode ("value/action fields" → "action fields").
  • cache.ts: Updated AgentReplayFillFormStep.fields type to drop value.
  • Tests: Removed value from test fixtures in timeouts.spec.ts and agent-execution-model.test.ts.

test plan

  • Run unit tests: agent-execution-model.test.ts passes with the updated fillForm field shape
  • Run integration tests: timeouts.spec.ts fillForm timeout test passes
  • Run agent eval with a form-filling task (e.g. sign_in) to verify the LLM correctly embeds values in the action string and fields get filled as expected
  • Test with variables: confirm %variableName% in the action string gets substituted correctly (e.g. "type %password% into the password field")

Summary by cubic

Removed the unused value field from the DOM-mode fillForm tool and moved values into the action string to match actual execution. Updated variable guidance, types, and tests to simplify form filling and fix variable substitution.

  • Migration
    • Update fillForm inputs to only use action, e.g. { fields: [{ action: "type hello into name" }] }.
    • Use %variableName% inside the action string, e.g. type %password% into the password field.
    • If you reference replay types, AgentReplayFillFormStep.fields now only includes { action: string }.