fix(desktop): accept common computer calls and let input reach attached sheets

Merged
WA
24d ago
+109-35

Summary

This follows #9084 and comes from two benchmark findings (MacAgentBench, Opus 5.5 through the worker loop).

Calls rejected before reaching the app. About one call per task was rejected as invalid input. This mirrors simstudioai/mothership#704, which should merge first so contracts:sync stays in step:

  • Coordinate clicks, scrolls and drags without windowId target the window the snapshot selected.
  • Scroll deltas default to 0, so a one-axis scroll is valid.
  • press_key with an elementId focuses that element first.

Input refused in system panels. #9084's stricter window checks refused input in two common panels:

  • Finder's Go to Folder: the app reports the sheet as its focused window, not the observed one.
  • Out-of-process open and save panels (Notes > Attach File): their contents report no window at all.

A sheet now counts as the window it is attached to. A click on windowless panel content is allowed only inside the app's focused sheet, and only when that sheet belongs to the observed window. Every other unconfirmed target is still refused.

Test plan

  • macOS 26 VM: Finder Cmd+Shift+G, then type a path and press Enter, navigates to the folder; Notes Edit > Attach File, then a click inside the panel and Escape both reach it
  • Helper compiles for arm64 with the macOS 26 SDK
  • 84-task benchmark against the merged #9084 build (running)
  • CI