feat(android): experimental direct Android engine over the engine protocol
Stacked on #1232. An Android engine that is an engine protocol server written in Java, running on the device: the proof that an engine can live outside the runner's process and language, and a measure of what driving the device directly buys over agent-device.
@e2e-dev/android (private, experimental)
A Java engine server (packages/android/server/java/) that runs on the device as an instrumentation. It listens on an abstract socket and drives the device in process through UiAutomation: the tree comes from accessibility nodes, and touches and keys are injected events. There is no adb process, no daemon hop, and no XML dump per call. The host half (src/transport.ts) installs the APK, starts the instrumentation, and forwards a port. The APK builds in seconds with javac, d8, aapt2, and apksigner, no Gradle.
Mobile benchmark, deterministic Android suite (apps/mobile-benchmark/e2e.android.config.ts, 42 tests, Pixel 7 API 36 emulator, one worker):
| Suite | agent-device | direct | Speedup |
|---|---|---|---|
| controls | 130.9s | 37.1s | 3.5x |
| flows | 47.7s | 11.9s | 4.0x |
| flows-lists | 110.1s | 72.8s | 1.5x |
| login-form | 28.7s | 8.6s | 3.3x |
| smoke | 3.8s | 1.4s | 2.6x |
| total | 321s | 131s | 2.4x |
Engine calls on the same screens (packages/android/bench/bench.ts, nearest-rank p50, 5-20 samples per row, init one cold sample):
| Operation | agent-device | direct | Speedup |
|---|---|---|---|
| init (cold) | 1593 ms | 168 ms | 9.5x |
| restart app | 887 ms | 156 ms | 5.7x |
| observe | 15.3 ms | 2.0 ms | 7.6x |
| tap | 132 ms | 30.4 ms | 4.3x |
| fill (19 chars) | 1028 ms | 4.4 ms | 231x |
| screenshot | 1459 ms | 65.7 ms | 22x |
| swipe | 402 ms | 465 ms | 0.9x |
| back, list up | 320 ms | 416 ms | 0.8x |
| keyboard.type (10 chars) | 67 ms | 333 ms | 0.2x |
It wins wherever agent-device spends an adb process or a daemon hop per call. It loses where the app sets the pace:
swipewaits for the list to come to rest.backis held up by the pop animation, which blocks accessibility queries for about 450 ms.keyboard.typesends one key event per character, and React Native re-renders on each. agent-device commits the whole string through its IME.
fill sets text with ACTION_SET_TEXT; agent-device types it and reads it back.
Both engines pass all 42, twice in a row. The direct engine declares the same device surface as the mobile engine: press takes Enter, Space, or one character, the keys a soft keyboard has, and refuses modifiers and other named keys with the mobile engine's reasons.
Not there yet (in its README): one device per run, a device fixture with back() only, no video or state capture, no selector locators, no launchArguments or permissions (refused at config load).
Verified
Ran it locally: yes
- Android: the mobile benchmark selection above on both engines (
node node_modules/e2e/dist/cli/bin.js run --config e2e.android.config.ts, withBENCH_ENGINE=agent-devicefor the baseline) andnode bench/bench.ts --rounds 20. - Review rounds: after the cubic pass, the Android suite still runs 40/42 in 134s, and no server process is left on the device after the run.
- Reviewed together with #1232: the fresh-context review and four cubic rounds covered both layers, and the Android findings (cancellation, masking, one session at a time, safe package names, device cleanup) are fixed here.