feat(ios): experimental Swift iOS engine over the engine protocol

Closed
OK
12h ago
+2037-1

Stacked on #1234. A second language behind the engine protocol: an iOS engine written in Swift and hosted by an XCUITest bundle on the simulator. It measures what driving the app straight from XCUITest buys over mobile() on agent-device.

How it works

flowchart LR
  H[harness: connectEngine] -- JSON-RPC over TCP 127.0.0.1 --> S[EngineServer.testServe<br/>Swift, in the XCUITest runner]
  T[transport.ts] -- xcodebuild test-without-building --> S
  S -- XCUIApplication.snapshot, XCUICoordinate --> A[app under test]
  • packages/ios/server/E2EServer.xcodeproj is a host app plus the E2EServer UI test bundle. It's hand-written, so there's no generator dependency, and it builds with build:server.
  • EngineServer.testServe listens on a localhost port, which the simulator shares with its Mac. It answers the protocol on the main thread, one connection at a time, and honors $/abort and abandonment as the Android server does.
  • Device.swift builds the tree from one XCUITest snapshot. It masks secure fields in pixels and screenshots, and acts with XCUITest gestures at the observed frames.
  • transport.ts installs appPath and starts the test with xcodebuild test-without-building on a free port. It waits for E2E_SERVER_READY, then connects with a plain socket.
  • ios({ waitForIdle: false }) replaces XCUITest's private idle wait with a no-op, as WebDriverAgent can. It's opt-in and exists to measure what that wait costs.
  • apps/mobile-benchmark/e2e.ios.config.ts runs the deterministic selection on it. BENCH_ENGINE=agent-device runs the same selection on the default engine.

Results

iPhone 17 Pro simulator, iOS 26.5, mobile benchmark app, agent-device and the Swift engine both with the benchmark's transition: 1500, one worker. Both pass all 41 suite tests, twice in a row.

Suiteagent-deviceSwiftSpeedup
controls164.5s138.5s1.2x
flows69.5s57.3s1.2x
flows-lists233.3s231.2s1.0x
login-form47.1s28.1s1.7x
smoke8.9s6.4s1.4x
total523s461s1.13x

Engine calls (p50):

Operationagent-deviceSwiftSwift, no idle wait
observe94.4 ms18.0 ms17.6 ms
locate76.2 ms14.6 ms14.4 ms
screenshot671 ms105 ms104 ms
fill (19 chars)4852 ms1989 ms1538 ms
keyboard.type (10 chars)1213 ms429 ms428 ms
tap484 ms422 ms390 ms
back, list up1055 ms1710 ms1034 ms
swipe558 ms1433 ms1423 ms
restart app710 ms1705 ms1714 ms

To match the mobile engine where the platform gives no signal, the Swift engine:

  • Reads React Native's checkbox and radio roles out of the accessibility value.
  • Taps a uniquely addressed node as an XCUITest element. That scrolls a covered one clear (the sticky footer) and reaches another process's sheet (Save Password).
  • Makes a control that arrived with the last action wait out the transition budget, because a presenting view drops touches while it reports its final frame.
  • Relaunches from the Mac with simctl for restart and clearState, and returns once the app answers.

The gain is much smaller than on Android (2.4x there). agent-device already drives iOS through an XCTest runner, so going direct removes only the daemon and HTTP hop:

  • Reads are 5-6x faster (snapshot, locate, screenshot).
  • Gestures are paced by XCUITest itself: its idle wait, and over a second per press(forDuration:thenDragTo:) swipe.

The next lever is XCTest's private event synthesis for gestures (what WebDriverAgent uses).

Verified

Ran it locally: yes

  • Suite: apps/mobile-benchmark, e2e.ios.config.ts on simulator stab-1, 41/41 on two consecutive runs (461s, 462s), against BENCH_ENGINE=agent-device (41/41, 526s).
  • Micro-bench: E2E_IOS_UDID=<udid> node bench/bench.ts --rounds 10, run twice with matching numbers.
  • Clean exit: the runner's xcodebuild process ends on dispose, with no orphan left after a run.