chat-open bench: the subagent case clicks subagent lines the chat renders

Merged
TE
5d ago
+50-2

Summary

In the chat-open bench, the subagent case's first open failed in every condition at both refs we measured (b623fbb2, 157bc918): open_reply = {"error": "no visible element matches"}. Opens 2 and 3 measured fine (~900 ms).

This is a bug in the harness's selector, not in the product. Open index clicked Auditor {index}. debug.agent_pane click only matches elements the page has rendered, and the parent chat renders only its newest rows (13 transcript_rows in every failing record). So Auditor 0, the oldest subagent line, was never in the page. A person would have to scroll up to see it as well. Auditor 1 and Auditor 2 were rendered, which is why opens 2 and 3 worked.

Fix: opens click the newest subagents first (Auditor 3, 2, 1 with the default --subagents = runs + 1).

Testing

  • python3 scripts/cmux-next/tests/test_chat_open_bench.py: 14 pass on the head. The new SubagentNameTests fails on the base, because subagent_name does not exist there.

Observed / Not run

  • Observed: the raw records of both bench runs (8 of 8 first subagent opens failed, all with 13 rendered rows) and the stand-in agent, which names subagents Auditor 0..N-1 (chat_open_agent.py).
  • Not run yet: a fleet bench at this head.

Changelog

none

Agent session: agent session

šŸ¤– Generated with Claude Code

Summary by cubic

Fixes the chat-open bench's subagent case so it clicks a subagent line the chat actually renders instead of a fixed name that may be absent from the virtualized page.

  • The bench tries clicking agent-pane lines newest to oldest and stops at the first click that lands; a click that matches nothing answers "no visible element matches" and does nothing, so the next name is tried.
  • Adds SubagentClickTests to cover the click ordering and fallback.