android: A51 — the scan-out image keeps the pool buffer's row stride

Merged
JO
11d ago
+95-1

What this fixes

A composited frame was sheared whenever the session's scan-out width was not a multiple of 320 px.

The scan-out image is allocated from the session's pool, whose rows are gralloc's — padded for widths gralloc does not like (1024 px → 5120 B on Adreno), where a LINEAR Vulkan image is otherwise laid out at width * 4. Created through a DRM format-modifier list, Turnip picks the tight pitch. The compositor reads the buffer at the buffer's stride, so every row the GPU writes lands further along than the row read back: the frame is sheared, repeated across the width, with the tail of the buffer never written.

Only a composited frame writes this image. Direct scan-out (one screen-sized layer, no filter that needs the shader, no look, no ReShade) is the guest's own buffer — which is why the fault appeared exactly when NEAREST/FSR/NIS/SGSR forced vulkan_composite. Measured on a 1024x768 session into a 1280x960 panel: SurfaceFlinger reported the layer's stride as 5120 B while the composite wrote at 4096. At 1280 and 1920 wide the two pitches coincide, so no output this stack has run before could show it.

The change

The pitch is not derivable from the geometry, so A51 asks for it:

  • the shim owns the pool and answers gamenative_drm_scanout_pitch(w, h) (declared weak, resolved from gamenative-drm's libgndrm_shim.so);
  • when it answers, the image is created with an explicit LINEAR modifier at that stride instead of the modifier list — which also makes the AddFB2 export report the stride the compositor reads;
  • the symbol is weak, so a real KMS session keeps the driver's own layout;
  • the create is gated to output images with no imported dma-buf and no memory to reuse, and the modifier list is unlinked so the create chain carries exactly one layout authority (the external-memory and format-list nodes stay as the chain's tail).

android/patches/gamescope/0049-A51-scanout-image-at-the-pool-row-stride.patch applies on top of the 48-patch train, and the train re-applies cleanly around it.

android/check-payload-closure.sh also gains libnativewindow.so in its system allowlist: the ICD dlopens it for the guest's AHardwareBuffer receive path and it exists on every Android this payload runs on. That check was failing on a pre-existing edge, not on anything added here — RESULT: CLEAN again.

Verification

  • Cross-build for aarch64-linux-android succeeds with zero warnings; the train applies cleanly onto a clean 48-patch tree, and clean-train + A51 reproduces the working tree byte for byte.
  • In the built binary gamenative_drm_scanout_pitch is WEAK UND (as intended), and GLOBAL FUNC in the shim.
  • check-payload-closure.sh: RESULT: CLEAN.
  • On device (Retroid Pocket Nova, 1280x960@120Hz, Adreno 740 + Turnip 26.3.0-devel), every session logs the pitch it lays its scan-out image out at:
    • 1024x768 sessions: output image 1024x768 at the pool's row stride 5120 (tight 4096) — the padded case, and the frame is now coherent with SGSR forcing the composite;
    • 1280x960 / 1280x720 sessions: … 5120 (tight 5120) — unchanged, as they must be.
  • SurfaceFlinger agrees on the whole chain for the padded case: layer geomBufferSize=[0 0 1024 768], buffer stride:5120 bytes, layer transform 1.2500 → 1280x960, composition=DEVICE.

Six device cases (container screen size × Upscaling Resolution × filter) were run end to end through the app — the full matrix and screenshots are in utkarshdalal/GameNative#2283, which is the app side of this feature.

Companion PRs

  • utkarshdalal/GameNative#2283 — the app hands the Upscaling Resolution to the daemon as the session's scan-out size, and defaults it to Off.
  • GameNative/gamenative-drm#32 — the emulated connector advertises that size as the mode, the presenter fits the composite into the window, and the shim gains the gamenative_drm_scanout_pitch export this patch consumes.

Depends on the shim export landing in gamenative-drm; without it the symbol stays unresolved and this patch is a no-op (the driver's own layout, i.e. the previous behaviour).