display: the gamescope vs X11 CPU investigation, and the present-path work it justified

Merged
JO
4d ago
+6136-57

What this is

The record of one investigation — where the gamescope display path costs CPU over the in-app X11 path — plus the four code changes that survived being measured.

The finding is not a fix: payload 20261110.1 had already collected the win, and four of the eight candidate causes did not survive being read against the source. The durable output is the measurement discipline, the refutations, and the tooling.

What this PR fixes

Two commits, both on the daemon's present path. Concretely:

43268c5 — per-frame churn in the poll loops and the JNI publish

  • pushes::fds_except allocated a Vec on every call. It is now fds_except_into(&mut Vec, except), and both callers (asc.rs scan-out channel, lib.rs unserved-listener loop) own their buffers outside the loop. Same rows, same order, same revents semantics — only the heap object is reused.
  • call_app_static paid a GetStaticMethodID plus two CString constructions on every publish. It now caches JStaticMethodID keyed by method name against the cached CALLBACK_CLASS, which is never unloaded. (JStaticMethodID, not the raw jmethodID: *mut _jMethodID is not Send and does not compile.)

e4634a7 — the presenter's retire signal, and the atomic decode

  • The presenter rebuilt a HashSet<u64> of live inodes by walking both object tables on every commit. Stores now keeps one count per inode plus a membership generation, and live_inos() returns Arc<HashSet<u64>>, rebuilding only when membership actually changed, so an unchanged commit pays an Arc clone. Every mutation site is hooked: both GEM-creation paths, GEM_CLOSE (moves to the orphan table without changing the count), the PRIME re-dup path, and the orphan sweep in destroy-FB.
  • parse_atomic materialised four scratch arrays (objs, counts, prop_ids, values) per commit. It now computes each section's offset from the header and count_objs and reads straight from the payload, with one length test replacing four element-wise ones — checked before anything is resolved, so EINVAL/ENOENT precedence is unchanged.

Measured effect: 6.04 → 1.04 allocations per commit, 915 → 695 bytes (−83 %). No CPU% or frame-rate change is claimed, because none is resolvable — see Before / after, counted.

Verification: cargo test --workspace 343 passed, 0 failed; clippy -D warnings clean on host --all-targets and on aarch64-linux-android; tools/build-android.sh green through the export allowlist and undefined-symbol audit. The debug_assert recount of gems vs gems_closed is proven live — breaking one count on purpose makes the golden suite fail with ino_refs disagrees with gems ∪ gems_closed, so a green run means something. Device check of the built .so (installed copy verified byte-identical in every content section; the md5 gap is only lld's .shstrtab ordering): two sessions, ~10 minutes of churn under gamescope, no panic, no assertion, no corrupt frames, publishes == vsyncs, late=0.

What this PR does not fix

  • No CPU or power win. At the 242 commits/s the daemon serves, the entire commit path is ~0.5 ms of CPU per second. These patches remove allocations, not time you can observe.
  • The remaining +5.1 pp inside the guest — under gamescope its Vulkan WSI is our ICD shim, running in Hades2.exe's address space. The next instrument is simpleperf record on Hades2.exe, not another plan item.
  • Injected pointer clicks — found while checking the build: drags move the game cursor correctly, but a tap activates nothing under gamescope or X11. Unrelated to this branch, deliberately out of scope.

The uncapped matched pair — full dataset

Method, because the numbers only mean anything with it. Two arms, same container except displayServer, same save slot, same idle spot, same scene. Both arms saturate cpuss at ~102 °C within three minutes, so they are matched by temperature as well as by scene — which is what makes a pair comparable, and why non-adjacent runs are superseded rather than averaged. Container 1280x720, panel 120 Hz, 8 cores, uncapped (fpsLimiterEnabled: false) in both: the guest rendered ~2.8x the frames the display can show.

Instruments: /proc/stat and per-core cpufreq at 1 Hz; kgsl-3d0/gpubusy and devfreq/cur_freq at 1 Hz; package power at 1 Hz; thermal_zone matched by type (cpuss-0-0, gpuss-0, aoss-0); top -n 1 -b -m 14 averaged over each window; fps from five MangoHud screencap reads per arm.

Table 1 — Before: pre-20261110.1, ~210 s per arm (source: 3588fa9)

metricX11gamescopedelta
fps (5 MangoHud reads)344.6339.2−1.6 %
CPU busy (8 cores)45.8 %47.3 %+1.5 pts
CPU user21.5 %21.0 %−0.5 pts
CPU sys20.3 %21.9 %+1.6 pts
GPU busy / clock83.8 % / 760 MHz84.5 % / 759 MHz+0.7 pts
package power19.18 W19.59 W+0.41 W (+2.1 %)
cpuss / gpu / soc101.7 / 76.8 / 78.6 °C101.9 / 76.3 / 78.0 °C—
RAM used6.48 GB6.60 GB+0.12 GB

Table 2 — After: on payload 20261110.1, adjacent pair, 240 s per arm

metricX11gamescopedelta
fps (5 MangoHud reads)338.8342.5+1.1 %
CPU busy (8 cores)45.9 %46.2 %+0.3 pts
CPU user20.9 %20.0 %−0.9 pts
CPU sys20.8 %21.8 %+1.0 pts
GPU busy / clock83.7 % / 760 MHz84.4 % / 747 MHz+0.7 pts
package power19.44 W19.40 W−0.04 W
cpuss / gpu / soc101.8 / 76.2 / 78.0 °C101.9 / 76.6 / 78.3 °C—
RAM used6.56 GB6.42 GB−0.14 GB

Step between the tables: +1.5 pts → +0.3 pts of 8-core CPU busy, at equal GPU load. An older still measurement — payload 20261105.4, both arms on the original build — survives only as one aggregate, +3.8 pts, with no instrument matrix. Full trajectory: +3.8 → +1.5 → +0.3.

Three caveats, all load-bearing:

  1. Table 1's power column is the charge-path form (usb_in − battery_net), which the Retraction section shows is unreliable — the offline USB node reports 0.26 W of phantom load. The +0.41 W → −0.04 W step is therefore not a clean result; the only trustworthy power pair is the on-battery one, and it does not cover the old payload.
  2. Table 1's provenance is the weakest thing here. 3588fa9 labels its gamescope column 20261110.1 while its own caveat says the X11 row came from the APK preceding that update, and its per-process header calls both arms "the older build". cbdfe3a reclassifies it: non-adjacent runs, this pair being the APK-only step (new APK, old payload on the gamescope arm), "superseded rather than averaged". The numbers are quoted unchanged because it is the only full matrix in the branch for a pre-20261110.1 gamescope arm — not because the provenance is clean.
  3. Table 1 is ~210 s per arm, Table 2 is 240 s; both are thermally saturated by the end of the window, which is what makes the step worth reading at all.

Per-process attribution (Table 2's pair)

top -n 1 -b -m 14 truncates to 14 rows and quantises to 3.7 pp per row per sample with ~20 samples per arm, so a row is trustworthy to roughly ±5 pp. The per-process sum does not reconcile against /proc/stat (+95 ms/s from the rows vs +24 ms/s measured overall) — a sampling artefact, so do not quote this as a partition of the total.

processX11gamescopedelta (pp of one core)
Hades2.exe (guest, incl. the ICD shim)289.8294.9+5.1
app.gamenative19.318.0−1.3
gamescope05.7+5.7
surfaceflinger10.910.90.0
wineserver10.710.5−0.2
vendor.qti.hardware.display.composer5.66.0+0.4

Two rows carry the story. The compositor costs about what the X server cost: in X11 mode the X server runs inside app.gamenative (started in-process, frame leaves via VulkanRenderer/ASurfaceRenderer in the same process), so that 19.3 includes an X server — under gamescope the app is only the gndrm daemon at 18.0 and gamescope appears as 5.7. A component moving, not a cost appearing. The guest's +5.1 pp is the only real delta, down from +20.8 before the payload, and it buys the entire present path at ~1.8 % of the guest's own CPU.

Rate census at the same time

Guest presents 341/s, gamescope commits 242/s (0.71 per present), panel 120/s, daemon vsync grid 121/s. Four different rates that the first pass conflated; separating them is what killed causes B and C.

What the cap is worth

The container was uncapped — 341 fps into a 120 Hz panel. On battery, where battery I x V is system power:

armfpsCPUpackage power
X11 uncapped35825.5 %7.38 W
X11 capped12017.8 %4.38 W
gamescope capped12116.1 %4.53 W ±0.18
gamescope, cap off12125.1 %4.32 W

Capping saves 3.00 W on the identical path. And at the same 121 fps, letting the guest do vsync costs 9 points of CPU more than the limiter does, for zero watts.

Retraction

An earlier pass concluded gamescope was ~1 W dearer. That was an artefact: the charger form usb_in − battery_net folds in taper and a phantom 0.26 W on the offline USB node (3000 µA at 86000 µV with usb/online=0). On battery the difference is +0.15 W ±0.21 (0.7 σ) — no measurable gap. power_now is a frozen constant on this device and must not be used.

The four refutations

Recorded because the wrong model is what would otherwise get built:

  • Per-slot fence reap — x11_reap returns immediately when present.is_null(); that route is DRI3-only and unused here.
  • VK_KHR_present_wait spinning — nothing chains VkPresentIdKHR; zero vkWaitForPresentKHR measured.
  • Acquire starvation — arithmetically bounded at ~46 µs/frame.
  • UBWC / tiled scan-out — an AHardwareBuffer cannot express UBWC.

Three plan items closed outright by measurement (present-wait, slot reaping, present-mode re-declaration — the last is load-bearing every commit, declared_fifo 0 of 25 009). rt_cap cannot work for an app uid on Android; the obtainable form is nice -20, and it is a latency item with no CPU claim.

Before / after, counted

Opinions about allocations in hot paths are cheap, so the commit path has an instrument that counts instead: modules/gndrm-daemon/tests/present_alloc_bench.rs drives the real Device::dispatch with a linearized ATOMIC flip request (ten plane properties, alternating between two imported framebuffers, 4 000 flips after a 64-flip warm up) behind a counting global allocator. Run in worktrees at each revision so the harness is byte-identical.

revisionallocs/flipbytes/flipns/flip (min of 3)
53a0aa5 — before all four6.049151770
43268c5 — +P3/P46.049151734
e4634a7 — +P1/P21.046951831

Six allocations per commit become one — −83 %, and −220 bytes. The five that go are the four scratch arrays in parse_atomic and the HashSet the presenter collected from both object tables every frame. The one that remains is the sets array the design deliberately kept (docs/performance/05 §9).

The time is not claimable, and the PR should not be reviewed as if it were. Minimums 1770 / 1734 / 1831 ns, medians 2448 / 2498 / 1860 — the spread between repeats of one build exceeds the spread between builds. At 242 commits/s the whole commit path is ~0.5 ms of CPU per second, so no device arm could resolve it either.

The flat P3/P4 row is not evidence against them: the pollfd hoist lives in the event-loop poll and the method-id cache in the JNI publish path, and the commit dispatch enters neither. Their case is the code argument.

The docs

docs/performance/, numbered 01-08, one document per plan item, each with file:line evidence, the change, the invariants, and the measurement that decides it. Evidence captures under docs/performance/evidence/. The top-level docs/01-23 series stays architecture/subsystem; this is its own series.

Tooling that came out of it and is reusable: a byte-anchored container editor (tools/container-env-flip.py), a one-command census arm (tools/gamescope-cpu-arm.sh), its parser (tools/gamescope-cpu-parse.py), and a scene fingerprint that proves two captures are the same place (tools/scene-fingerprint.py).

Not in this PR

  • nice -20 for gamescope — designed (07), unstarted; a frame-time p99 item, not a CPU one.
  • The remaining guest +5.1 pp — needs simpleperf record, not another plan item. The two witnesses that would have justified more shim work are exactly the ones step 1 refuted.
  • The pointer-click defect — the next step there is instrumented logging, not a speculative patch.

Reviewing it

The two code commits are self-contained and reviewable on their own. 01 is the investigation summary, 05 is what the code implements, 02 is how every number was taken.