#499 ci(mobile): iOS simulator and Android emulator gates for the mobile benchmark, with a Control Inventory scenario

The engine fixes these suites found are merged as #491. Same commit as #498, which GitHub closed when #491's branch was deleted, now rebased onto main. The iOS run of this tree passes on an iPhone 17 Pro simulator: 6 files, 50 tests, 10 skipped where agent-device or React Native has no answer yet (each skip names why). The Android job runs for the first time with the emulator selected by serial.

flowchart LR
  F[@expo/fingerprint] --> B{cached app?}
  B -- hit --> R[repack JS with @expo/repack-app] --> D[deterministic suite]
  B -- miss --> N[native build] --> D
  D --> A{same repo and key?}
  A -- yes --> G[agentic suite]

• A Control Inventory scenario (plain controls, every state exposed) plus controls and device-fixture suites exercise every verb, read, matcher, refusal, and device method on a device; the flows gained state matchers.
• mobile.yml: iOS on blacksmith-6vcpu-macos-26, Android emulator on x64 Blacksmith with KVM. .github/actions/expo-app keys the Expo build by its native fingerprint, keeps it in the actions cache, repacks the JS on a hit, falls back to a native build. scripts/ci-run.sh runs the deterministic suite and then the agentic one when the key is present (the emulator runner executes each script line separately, so the guard lives in the script). The booted simulator is handed to the config as device.
• Skips that remain, each an upstream limit: agent-device delivers a double tap as one tap on iOS; no focused flag on iOS text fields; the snapshot carries no placeholder or attributes, so getByPlaceholder, getAttribute, and toHaveAttribute match nothing on a device; React Native flattens plain Views, so filter({ has }) into a View with only a test id matches nothing.
• The iOS job boots the iPhone 17 Pro on the iOS 26.5 runtime, pinned by IOS_RUNTIME in the workflow. The runner carries four runtimes and the first match was 26.2, whose accessibility bridge reports no selected on a tab or radio and an empty field's placeholder as its value; seven tests that pass on 26.5 failed there. Bump the pin on purpose, after a local run on the new runtime.
• What the first iOS runs found. With the same Xcode 26.6, iOS 26.5 runtime, and Release build as a Mac at a desk, the CI Mac serves agent-device snapshots without the simulator's accessibility bridge: a React Native radio carries checked but no selected, a tab no state, and an empty field reports its placeholder as its value (the failure screens are printed into the job log now, since the artifact quota is full). The engine reads a field that shows its hint as empty; the suites read a radio by checked, a tab by the section it reveals, and the reward claim by the message it leaves rather than the toast, which is gone before a snapshot lands. The microphone alert overruns the runner's watchdog on that host on every run and wedges the runner for whatever follows, so in CI that one test runs on Android only.
• Android is advisory for now (continue-on-error on the job), tracked in #500: on the emulator no scenario opens, every deterministic test times out in openScenario, and the job runs into its limit before the other files start. The three refusal tests, which open nothing, pass, so the engine and the emulator boot are fine; the list scroll or the row lookup is not. No Android AVD exists on the development Mac, so this needs a machine with one. The iOS job is the gate; the script stops after a deterministic failure, so the advisory Android job spends no model calls.
• Second iOS run on CI, deterministic suites: 6 files, 48 passed, 10 skipped, 0 failed. The agentic suite ran live there (no recordings yet in that commit) at 22 of 23; its recordings are committed now (22 entries from a Mac, 14 replay on main's cache code, 8 call the model until #480), Huge Virtualized List is a declared gap (the agent scrolls blind on a list iOS renumbers every observation, #486), and Apple Pay Billing Address is a gap on the CI simulator only, whose Apple Pay sheet has no billing-address step.
• CI time, measured on a Mac at a desk against the Release build. The home list is a two-column grid so a scenario is one tap away instead of five to ten seconds of dragging, the three-second transition budget that existed for the flung list is gone, and the iOS target drives a pool of simulators. Deterministic suite, 50 tests: 1090 s before, 444 s on one simulator, 283 s on two. A deep link was tried first and abandoned: simctl openurl raises the system "Open in" alert on iOS. In CI the iOS job is now two jobs, deterministic and agentic, each with two simulators (the picked iPhone 17 Pro and a clone), so the gate's wall time is one suite, not both.
• CI on the final head (run 35909674621): ios deterministic green in 9 minutes (suites 359 s, two simulators booted in 137 s, app restored from cache in 10 s), ios agentic green in 16 minutes (786 s of suites, 20 scenarios pass, 4 declared gaps, 9 acts still call the model, $0.03), against a single 45-minute-plus job before. The flows file is split in two so the two workers share it, a moved control gets a second and a half to come to rest, and the scenario-open check reads the header title so it works on Android too: the emulator now runs the whole suite (28 pass, 19 fail on Android-only semantics, listed in #500) and stays advisory.
• Android, 2026-09-24, from an emulator on a Mac. Of the 19 failures listed in #500, 18 were the suites' iOS spellings (the navigation bar's identifier as the home marker, role=ScrollView matching react-native-screens' wrapper too, tab buttons without ids, CONFIRM in capitals, a password sheet Android never shows, check() on a switch whose state Android does not report) or agent-device 0.21.6 facts the tests now name and run on iOS only (names from the content description, checked and heading, a WebView field's label, logical pixels, a clipboard read). The last one was an engine bug: the transition wait acted on the ref resolved mid-transition, and Android reports a sliding modal's frames in flight, so the tap went below the screen. The control is found again after the wait now (@e2edev/mobile patch), and AbsListView is a list. Emulator: 39 passed, 1 flaky, 10 skipped; iOS on two simulators: 48 passed, 10 skipped. The agentic suite's 17 Android recordings are committed (Stripe's Compose postal-code field takes no input from agent-device's fill, so that scenario runs on iOS only). The Android job is two jobs now like iOS, and the iOS boot step starts both simulators before waiting. Still advisory: on CI the sequential onboarding fill of the third step's number field failed verification in one run of two (the field stayed on its hint), and the permission test's relaunch after a grant lands on the launcher on its first attempt in every run; both pass locally.
• Launching is explicit now, the Maestro model and the web's: an attempt launches nothing, the worker brings the pinned app up once in prepare, and a test calls app.open() when it wants the app fresh (on a device the runner serves it with the engine's restart; a path there is APP_URL_REQUIRED). A launch: 'once' option was built, measured, and reverted first: with the fixture popping back to the home list it lost time on iOS (308 s against 279 s) and gained nothing on Android, since a pop and the transition budget the first tap then pays cost as much as the launch. agent-device is 0.21.13; probing the skips against it moved four tests (toBeFocused, the two double taps, foregroundApp after home) to Android-only, where they pass, with the iOS runner facts kept as comments. Emulator: 44 passed, 6 skipped; two simulators: 48 passed, 6 skipped. The time audit lives in #500: agent-device's own startup is cheap, the per-attempt cost is the relaunch plus a cold first observation (about 2.3 s), and the CI simulator boots are now concurrent.
• Run 35992430569 on this tree: all four mobile jobs green (Android 44 passed and 16 passed with one flaky, iOS 48 and 20). The judge had failed two Android pixel scenarios by contradicting itself over quote marks; it now explains before it decides and treats quotation marks as delimiters (e2e patch). The Android jobs gate from here; the remaining known items are the iOS runner episodes (#502) and tap_at recordings inside anonymous containers that never replay (#503).
• The run script is gone: each mobile job runs one e2e run line whose || ... --last-failed gives the failed tests a second pass before the job goes red, since agent-device's runtime has episodes of its own on the CI machines (#502), and a step prints the failure screens on failure (the artifact quota is full). Run 35999168302 on this shape: all four jobs green, the Android jobs gating; the iOS agentic first pass lost Modal Flow to app 'dev.e2e.benchmark' is not running and the second pass replayed it.

---