fix(mobile): keep the known viewport when the app reads smaller than the screen

Merged
KR
5d ago
+98-19

On iOS, a swipe that starts at a sheet's grabber (y≈52) can land in the system's top-edge zone and pull down Notification Center instead of dismissing the sheet. With Notification Center up, agent-device's XCTest snapshot backend reads the app behind it at a third of the screen: 130x281.33 on a 390x844 iPhone, marked healthy. project() stored that as the known viewport, so every rect and tap after it was measured against the wrong screen (ENG-914).

The triage blamed a sparse mid-transition frame. On an iOS 26 simulator the shrunk tree was healthy and lasted as long as Notification Center stayed down, so a sparse-only guard would never fire.

  • A snapshot replaces the known viewport only when it is the same size or that size rotated (sameScreen, packages/mobile/src/surface.ts).
  • Any other size is checked once against the simulator screenshot's logical size. If they disagree, the known viewport stays. The screenshot reports portrait even in landscape, so it confirms a size but not an orientation; rotations are taken from the tree.
  • A real device's screenshot carries no logical size, so there the tree's size still wins, as on main. Android reports a constant screen size and never produced this.
  • viewportFor accepts a rotated size itself, so the keyboard dismissal, which projects its own snapshot, still follows a rotation since the last observation.
flowchart LR
  S[snapshot size] --> K{known?}
  K -- no --> A[adopt]
  K -- yes --> R{same or rotated?}
  R -- yes --> A
  R -- no --> P{screenshot logical size}
  P -- agrees --> A
  P -- none --> A
  P -- disagrees --> KEEP[keep known]

Verified

Ran it locally: yes

  • iPhone 16e simulator (iOS 26), Calendar. Through agent-device directly, a swipe from the Calendars sheet's grabber pulled Notification Center and the snapshot came back as Application "Calendar" 130x281.33, healthy, with the screenshot at 390x844.
  • A scratch e2e test against the built branch packages: portrait, landscape, back to portrait, open the Calendars sheet, swipe from its grabber. Passed. The engine's viewport (logged from dist) was 390x844, then 844x390 in landscape with no screenshot probe, then 390x844 through the swipe.
  • On this simulator the engine read through agent-device's AX backend, which reported 390x844 even with Notification Center down, so main did not misbehave end to end. The live 130x281.33 tree is replayed in a unit test instead.
mainthis branch
viewport after the 130x281.33 tree (keeps the known viewport for an app drawn smaller than the screen)130x281.33390x844

Fixes ENG-914

Made with Claude Code (Opus 5.5); reviewed by Claude Fable 5.1.

https://github.com/user-attachments/assets/8760d40e-71ee-4091-bad7-59d76cdf3aae