ci(mobile): iOS simulator and Android emulator gates for the mobile benchmark, with a Control Inventory scenario

Closed
OK
1mo ago
+1647-31

Stacked on oskar/mobile-engine-fixes (#491), which carries the engine fixes the first iOS run of these suites found. Supersedes #482, which GitHub keeps in a stack with the merged #475 and #476 and will not retarget. With #491 underneath, the iOS run of this tree passes: 6 files, 50 tests, 10 skipped where agent-device or React Native has no answer yet (each skip names why). The Android job runs for the first time with the emulator selected by serial.

flowchart LR
  F[@expo/fingerprint] --> B{cached app?}
  B -- hit --> R[repack JS with @expo/repack-app] --> D[deterministic suite]
  B -- miss --> N[native build] --> D
  D --> A{same repo and key?}
  A -- yes --> G[agentic suite]
  • A Control Inventory scenario (plain controls, every state exposed) plus controls and device-fixture suites exercise every verb, read, matcher, refusal, and device method on a device; the flows gained state matchers.
  • mobile.yml: iOS on blacksmith-6vcpu-macos-26, Android emulator on x64 Blacksmith with KVM. .github/actions/expo-app keys the Expo build by its native fingerprint, keeps it in the actions cache, repacks the JS on a hit, falls back to a native build. scripts/ci-run.sh runs the deterministic suite and then the agentic one when the key is present (the emulator runner executes each script line separately, so the guard lives in the script). The booted simulator is handed to the config as device.
  • Skips that remain, each an upstream limit: agent-device delivers a double tap as one tap on iOS; no focused flag on iOS text fields; the snapshot carries no placeholder or attributes, so getByPlaceholder, getAttribute, and toHaveAttribute match nothing on a device; React Native flattens plain Views, so filter({ has }) into a View with only a test id matches nothing.

Devin Review