fix(bench): quote the fake agent's path so fleet agent-mode benches open their chats
The memory bench's agent mode (perf-10x-idle.sh --arg=agents=N, bench_memory.py --fake-agent) has opened no chats on any fleet step. The app splits a harness's args into words the way a shell does (AgentHarnessAddRequest.words). A fleet step's checkout lives under /Library/Application Support/cmux-build-worker/..., so the fake agent's path became two words, the agent exited at once, and the first acp new failed with agent.exited. bench_memory then stops after the launch workspace, and every agents=N sample reads agents=1.
The fix quotes the path with shlex.quote, which words() already understands. The same call in markdown-chat-typing-e2e.py had the same bug and gets the same fix.
Evidence: a fleet step at 6687c32f22bf with agents=5 printed acp new: {"error":{...,"errorKind":"agent.exited","exit":1,...}} and sampled idle-5 agents=1. The same fake agent answers initialize, session/new and a prompt when started with its path as one argument, on Python 3.9 and 3.12.
Performance
Same harness both sides: scripts/measure/perf-10x-idle.sh --arg=counts=1 --arg=agents=N --arg=measure=… on one fleet Mac, both runs on the same worker (fresh-home tagged build). This PR changes no app code; the rows show the harness now measures what it is asked to.
| metric | before | after | delta | harness or run link |
|---|---|---|---|---|
workspaces in the idle-20 sample (agents=20) | 1 | 20 | +19 | before: cmux-ci step 7aa4a5e6f2cb1f3b6413623b (9f45a27f); after: step 5d283c1b76d4e8112ee62c50 (9d2e8cc8) |
agent chats opened by --fake-agent | 0 (acp new: agent.exited) | 19 | +19 | same steps; before also step 81f44712f605894f573b79a1 (6687c32f, agents=5) |
cef-renderer processes at idle-20 | 5 | 24 | +19 | same steps |
cef-renderer footprint at idle-20 | 290.7 MB | 1840.0 MB | +1549.3 MB | same steps |
No capture: harness scripts only, nothing visible.
Agent session: agent session
🤖 Generated with Claude Code
Agent session: agent session
![View with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v3-3x.png)
![Autofix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark-3x.png)
Need help on this PR? Tag @codesmith-bot with what you need.
Summary by cubic
Fixes agent-mode memory benches so fleet steps actually open chats instead of sampling agents=1.
- The app splits a harness's
argsinto shell-style words, and fleet checkouts live under a path with a space, so the unquoted fake-agent path became two words and the agent exited at once. - Quotes the path with
shlex.quote(which the word splitting already understands) inbench_memory.pyand applies the same fix inmarkdown-chat-typing-e2e.py. - Harness-only change; no app code or perf numbers are affected.