fix(mobile): start the iOS keyboard-dismissing drag clear of controls

Merged
KR
6d ago
+284-35

On an iPhone, the keyboard has no dismiss key, so device.dismissKeyboard() sends it away with a short drag. That drag started at the exact centre of the screen and travelled 3% of the screen width, which is still within touch slop. If a control sat at the centre, the drag pressed it. In the user report it selected a different filter button; in the benchmark repro below it checks a checkbox. When a control took the touch, the keyboard also stayed up, because the scroll view never got the tap.

Fixes ENG-910.

  • Each drag now starts at the point nearest the centre that no control covers, with 8 pt of clearance (packages/mobile/src/keyboard.ts).
  • When the screen has a scroll view, the start must be inside one: a scroll view is what turns the drag into a dismissal, and a drag anywhere else does nothing.
  • A control is a node with a control role (bars included), or a leaf that isn't plain content. A React Native Pressable with no role shows up as an unnamed leaf, while text and images don't count.
  • A leaf over half the screen counts as a flattened tree's merged content, not a control. Those screens drag at the centre as before, and the committed iOS recording for Flattened Registration Form depends on that.
  • The start point is picked again from the latest capture before each drag (horizontal, then vertical).
  • A screen with no clear point fails with the existing UNSUPPORTED_CAPABILITY instead of touching a control.
  • New benchmark test on Choice Controls (apps/mobile-benchmark/tests/device.e2e.ts): it sets a valid order, fills the notes field, dismisses the keyboard, and places the order. A toggled control would leave Place order disabled.

Known gaps, written up in the JSDoc: a Text with its own onPress reads as plain text, and a control inside a flattened tree can't be seen. Both get the same drag main makes today.

Verified

Ran it locally: yes

  • mobile benchmark on an iPhone 17 Pro simulator (iOS 26.5), e2e run tests/device.e2e.ts --target ios-simulator --grep "choice controls" --video
  • main: both drags start at (201, 437), on the Mushrooms checkbox. It gets checked, the keyboard stays up, and the test fails: 
    × dismissing the keyboard over choice controls leaves them as they were 25.91s
UNSUPPORTED_CAPABILITY: keyboard dismissal failed: the keyboard has no dismiss key and stayed up through a horizontal and a vertical drag at the centre of the screen.
  • this branch: one horizontal drag from (201, 517), in the gap between "Rush delivery" and its switch, dismisses the keyboard and changes no control: 
    ✓ dismissing the keyboard over choice controls leaves them as they were 16.88s
  • whole iOS deterministic suite on this branch: Tests 56 passed | 1 skipped (57), including Sequential Onboarding's dismissals
  • iOS agent replay, --config e2e.agent.config.ts --grep "Flattened Registration Form": replayed all 11 recorded actions, no model call, dismissKeyboard included. The run had no model key, so only the judgment afterwards couldn't run.
  • the start point on every committed golden snapshot (iOS and Android): flattened screens and text-heavy scroll views start at the centre as before, Choice Controls steers clear of its checkbox, and the benchmark home list (all rows) is refused
  • Android not run: it presses the keyboard's own dismiss key and never reaches this path
mainthis branch
mainfix

The videos below show main first, then this branch.

Changes made by Claude Opus 5.5 in Claude Code (T3 Code).

https://github.com/user-attachments/assets/4fcc3f62-05f9-424d-bdb8-584be75b1ef3

https://github.com/user-attachments/assets/c243f156-3493-4e5c-881c-fd9c1802d759