fix(workflow-tests): keep the error of a mock that throws

Merged
WA
6d ago
+29-1

Summary

When a workflow-test mock throws, the block it mocks should fail with the thrown message. Today it fails with A mocked block must return an object, like { ok: true } instead.

The sandbox sends a thrown mock as { runId, callId, error }. testReplyArgsSchema is a union whose first branch is { runId, callId, output: z.unknown() }. That branch also accepts an object with no output key, so it matched and stripped error. The session then saw output: undefined and replaced the error with the generic message.

As a result, a workflow that branches on a failed block's error text (for example a 409 from a mocked tool) behaves differently under test than live. The error branch is now tried first.

Found while checking live the Company Sim benchmark, whose hidden tests fail mocked MCP blocks with <status> <error>.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other: ___________

Testing

  • Added lib/workflow-tests/protocol.test.ts. It fails on the old union order (the thrown error is lost) and passes with the fix. Output replies, including null, still parse as output.
  • executor/execution/test-hooks.test.ts still passes.
  • Live check on a local Sim built from staging: a mock that throws 503 … on an MCP block reached the block's error path as A mocked block must return an object, like { ok: true }. I have not yet re-run this live check on the fixed build.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)