fix(mobile): recover iOS act from Enter without keyboard and lost app

Merged
OK
18d ago
+161-19

Fixes the iOS agentic flake in act completes Login Form (APP_NOT_OPEN after Safari took the screen).

Root cause (CI run 40408011065 report)

  • Replay of the cached Login trace: first Log in tap closed the keyboard.
  • Next recorded step, press('Enter') on Password, ran with no keyboard up. agent-device's XCTest keyboardReturn failed and reset the runner session.
  • Next observation took 25.6s and saw only application "Safari" (Save Password sheet host). Model then guessed app names: e2eBenchmark, Mobile Benchmark, Benchmark, all APP_NOT_INSTALLED.

Changes

  • press('Enter') on an unfocused field taps it first, as a character key already did (packages/mobile/src/surface.ts, pressKey).
  • open_app with no app reopens the app under test. A wrong name gets an INVALID_ARGUMENT that tells the model to omit app, instead of agent-device's "Run apps..." CLI hint.
  • withHint drops agent-device's per-code default hints (CLI advice), for every translated device error.
// agent tool call after another app takes the screen
open_app({}) // -> "Opened the app under test (dev.e2e.benchmark)."

Not fixed here: act completes Gestures double tap. XCTest can't deliver two taps under RN's 300ms window (count: 2 lands 300-301ms apart; doubleTap() and single-record variants reach RN as one press). Needs a HID touch path in agent-device, separate change.

Verified

Ran it locally: yes (iPhone sim, mobile-benchmark, agent-device 0.21.31)

  • Scratch test: fill, press('Enter'), press('Enter'), tap Log in. Keyboard down for the second Enter.
mainthis branch
fails with the CI keyboardReturn errorpasses 3/3
  • Scratch agentic test: open Safari, then agent.act back to Login Form. Agent called open_app({}), passed 2/2.
  • Deterministic login-form + controls on iOS and Android: 53 passed, 2 skipped. Agentic Login Form, Text Input, Choice Controls: 6/6.
  • Original flake didn't reproduce locally (5/5 under CPU stress on a fresh sim).
  • Self-review: limrun iOS refuses Enter, so the new focus tap lands before that refusal; dismissed, the test fails either way.