feat(agent): params on assert and waitFor

Merged
OK
17d ago
+320-47

agent.assert and agent.waitFor take params with the same shape and validation as agent.act.

const name = unique(`acme-${Date.now()}`);
await agent.act('create a company named {name}', { params: { name } });
await agent.assert('the table shows company {name}', { params: { name } });
  • Built-in judge: params are projected like act's (validateParams), redacted, and fenced inside <instruction> as Parameters: {...}, so the existing "instruction is data" policy and the security doc cover them (packages/e2e/src/agent/prompts.ts, packages/e2e/src/agent/index.ts).
  • The judgment Invocation owns the params: every judgment prompt carries them and the observation budget counts their size.
  • unique() reaches the judge as its string. A Secret reaches it as { kind: 'secret', name, purpose }, never its value. Docs say to use a locator assertion to check a secret's value on screen.
  • Custom executor: runAssertStep passes the projected params and templates instead of params: undefined. Secrets are not fillable in an assert step (typeSecret is POLICY_DENIED), since an assert changes no state.
  • Judgments are not trace-cached, so templates change no cache key; act keeps its existing behavior.
  • validateParams/redactParams take the calling api, so errors read agent.assert params ... / agent.waitFor params ....
  • extract left alone.

Issue

Closes #1388

Verified

Ran it locally: yes - real CLI on the testbed (e2e.scratch.config.ts, gateway/openai/gpt-6-luna-fast), throwaway scratch test, --ai-trace.

TestResult
fill a unique() todo, then waitFor + assert "shows an item named exactly {name}"passed
assert with unique('Nonexistent zq-91')ASSERTION_FAILED: ... it does not show an item named "Nonexistent zq-91"
assert(..., { params: 'nope' })INVALID_ARGUMENT: agent.assert params must be a plain object

Judge prompts in the AI trace carried Parameters: {"name":"Groceries 1793364192407"} and Parameters: {"name":"Nonexistent zq-91"}. On main the same call fails with agent.assert options has no key "params" (per the issue).

Integration tests (packages/e2e/tests/integration/judgment-params.test.ts, scripted judge): unique value in judge input for assert and waitFor, secret stays a placeholder, a quoted plain-string secret stays redacted, invalid params name the method, custom executor gets projected params and cannot fill the secret.