display: the gamescope vs X11 CPU investigation, and the present-path work it justified
What this is
The record of one investigation — where the gamescope display path costs CPU over the in-app X11 path — plus the four code changes that survived being measured.
The finding is not a fix: payload 20261110.1 had already collected the win, and four of the eight candidate causes did not survive being read against the source. The durable output is the measurement discipline, the refutations, and the tooling.
What this PR fixes
Two commits, both on the daemon's present path. Concretely:
43268c5 — per-frame churn in the poll loops and the JNI publish
pushes::fds_exceptallocated aVecon every call. It is nowfds_except_into(&mut Vec, except), and both callers (asc.rsscan-out channel,lib.rsunserved-listener loop) own their buffers outside the loop. Same rows, same order, samereventssemantics — only the heap object is reused.call_app_staticpaid aGetStaticMethodIDplus twoCStringconstructions on every publish. It now cachesJStaticMethodIDkeyed by method name against the cachedCALLBACK_CLASS, which is never unloaded. (JStaticMethodID, not the rawjmethodID:*mut _jMethodIDis notSendand does not compile.)
e4634a7 — the presenter's retire signal, and the atomic decode
- The presenter rebuilt a
HashSet<u64>of live inodes by walking both object tables on every commit.Storesnow keeps one count per inode plus a membership generation, andlive_inos()returnsArc<HashSet<u64>>, rebuilding only when membership actually changed, so an unchanged commit pays anArcclone. Every mutation site is hooked: both GEM-creation paths,GEM_CLOSE(moves to the orphan table without changing the count), thePRIMEre-dup path, and the orphan sweep in destroy-FB. parse_atomicmaterialised four scratch arrays (objs,counts,prop_ids,values) per commit. It now computes each section's offset from the header andcount_objsand reads straight from the payload, with one length test replacing four element-wise ones — checked before anything is resolved, soEINVAL/ENOENTprecedence is unchanged.
Measured effect: 6.04 → 1.04 allocations per commit, 915 → 695 bytes (−83 %). No CPU% or frame-rate change is claimed, because none is resolvable — see Before / after, counted.
Verification: cargo test --workspace 343 passed, 0 failed; clippy -D warnings clean on host --all-targets and on aarch64-linux-android; tools/build-android.sh green through the export allowlist and undefined-symbol audit. The debug_assert recount of gems vs gems_closed is proven live — breaking one count on purpose makes the golden suite fail with ino_refs disagrees with gems ∪ gems_closed, so a green run means something. Device check of the built .so (installed copy verified byte-identical in every content section; the md5 gap is only lld's .shstrtab ordering): two sessions, ~10 minutes of churn under gamescope, no panic, no assertion, no corrupt frames, publishes == vsyncs, late=0.
What this PR does not fix
- No CPU or power win. At the 242 commits/s the daemon serves, the entire commit path is ~0.5 ms of CPU per second. These patches remove allocations, not time you can observe.
- The remaining +5.1 pp inside the guest — under gamescope its Vulkan WSI is our ICD shim, running in
Hades2.exe's address space. The next instrument issimpleperf recordonHades2.exe, not another plan item. - Injected pointer clicks — found while checking the build: drags move the game cursor correctly, but a tap activates nothing under gamescope or X11. Unrelated to this branch, deliberately out of scope.
The uncapped matched pair — full dataset
Method, because the numbers only mean anything with it. Two arms, same container except displayServer, same save slot, same idle spot, same scene. Both arms saturate cpuss at ~102 °C within three minutes, so they are matched by temperature as well as by scene — which is what makes a pair comparable, and why non-adjacent runs are superseded rather than averaged. Container 1280x720, panel 120 Hz, 8 cores, uncapped (fpsLimiterEnabled: false) in both: the guest rendered ~2.8x the frames the display can show.
Instruments: /proc/stat and per-core cpufreq at 1 Hz; kgsl-3d0/gpubusy and devfreq/cur_freq at 1 Hz; package power at 1 Hz; thermal_zone matched by type (cpuss-0-0, gpuss-0, aoss-0); top -n 1 -b -m 14 averaged over each window; fps from five MangoHud screencap reads per arm.
Table 1 — Before: pre-20261110.1, ~210 s per arm (source: 3588fa9)
| metric | X11 | gamescope | delta |
|---|---|---|---|
| fps (5 MangoHud reads) | 344.6 | 339.2 | −1.6 % |
| CPU busy (8 cores) | 45.8 % | 47.3 % | +1.5 pts |
| CPU user | 21.5 % | 21.0 % | −0.5 pts |
| CPU sys | 20.3 % | 21.9 % | +1.6 pts |
| GPU busy / clock | 83.8 % / 760 MHz | 84.5 % / 759 MHz | +0.7 pts |
| package power | 19.18 W | 19.59 W | +0.41 W (+2.1 %) |
| cpuss / gpu / soc | 101.7 / 76.8 / 78.6 °C | 101.9 / 76.3 / 78.0 °C | — |
| RAM used | 6.48 GB | 6.60 GB | +0.12 GB |
Table 2 — After: on payload 20261110.1, adjacent pair, 240 s per arm
| metric | X11 | gamescope | delta |
|---|---|---|---|
| fps (5 MangoHud reads) | 338.8 | 342.5 | +1.1 % |
| CPU busy (8 cores) | 45.9 % | 46.2 % | +0.3 pts |
| CPU user | 20.9 % | 20.0 % | −0.9 pts |
| CPU sys | 20.8 % | 21.8 % | +1.0 pts |
| GPU busy / clock | 83.7 % / 760 MHz | 84.4 % / 747 MHz | +0.7 pts |
| package power | 19.44 W | 19.40 W | −0.04 W |
| cpuss / gpu / soc | 101.8 / 76.2 / 78.0 °C | 101.9 / 76.6 / 78.3 °C | — |
| RAM used | 6.56 GB | 6.42 GB | −0.14 GB |
Step between the tables: +1.5 pts → +0.3 pts of 8-core CPU busy, at equal GPU load. An older still measurement — payload 20261105.4, both arms on the original build — survives only as one aggregate, +3.8 pts, with no instrument matrix. Full trajectory: +3.8 → +1.5 → +0.3.
Three caveats, all load-bearing:
- Table 1's power column is the charge-path form (
usb_in − battery_net), which the Retraction section shows is unreliable — the offline USB node reports 0.26 W of phantom load. The +0.41 W → −0.04 W step is therefore not a clean result; the only trustworthy power pair is the on-battery one, and it does not cover the old payload. - Table 1's provenance is the weakest thing here.
3588fa9labels its gamescope column20261110.1while its own caveat says the X11 row came from the APK preceding that update, and its per-process header calls both arms "the older build".cbdfe3areclassifies it: non-adjacent runs, this pair being the APK-only step (new APK, old payload on the gamescope arm), "superseded rather than averaged". The numbers are quoted unchanged because it is the only full matrix in the branch for a pre-20261110.1gamescope arm — not because the provenance is clean. - Table 1 is ~210 s per arm, Table 2 is 240 s; both are thermally saturated by the end of the window, which is what makes the step worth reading at all.
Per-process attribution (Table 2's pair)
top -n 1 -b -m 14 truncates to 14 rows and quantises to 3.7 pp per row per sample with ~20 samples per arm, so a row is trustworthy to roughly ±5 pp. The per-process sum does not reconcile against /proc/stat (+95 ms/s from the rows vs +24 ms/s measured overall) — a sampling artefact, so do not quote this as a partition of the total.
| process | X11 | gamescope | delta (pp of one core) |
|---|---|---|---|
Hades2.exe (guest, incl. the ICD shim) | 289.8 | 294.9 | +5.1 |
app.gamenative | 19.3 | 18.0 | −1.3 |
gamescope | 0 | 5.7 | +5.7 |
surfaceflinger | 10.9 | 10.9 | 0.0 |
wineserver | 10.7 | 10.5 | −0.2 |
vendor.qti.hardware.display.composer | 5.6 | 6.0 | +0.4 |
Two rows carry the story. The compositor costs about what the X server cost: in X11 mode the X server runs inside app.gamenative (started in-process, frame leaves via VulkanRenderer/ASurfaceRenderer in the same process), so that 19.3 includes an X server — under gamescope the app is only the gndrm daemon at 18.0 and gamescope appears as 5.7. A component moving, not a cost appearing. The guest's +5.1 pp is the only real delta, down from +20.8 before the payload, and it buys the entire present path at ~1.8 % of the guest's own CPU.
Rate census at the same time
Guest presents 341/s, gamescope commits 242/s (0.71 per present), panel 120/s, daemon vsync grid 121/s. Four different rates that the first pass conflated; separating them is what killed causes B and C.
What the cap is worth
The container was uncapped — 341 fps into a 120 Hz panel. On battery, where battery I x V is system power:
| arm | fps | CPU | package power |
|---|---|---|---|
| X11 uncapped | 358 | 25.5 % | 7.38 W |
| X11 capped | 120 | 17.8 % | 4.38 W |
| gamescope capped | 121 | 16.1 % | 4.53 W ±0.18 |
| gamescope, cap off | 121 | 25.1 % | 4.32 W |
Capping saves 3.00 W on the identical path. And at the same 121 fps, letting the guest do vsync costs 9 points of CPU more than the limiter does, for zero watts.
Retraction
An earlier pass concluded gamescope was ~1 W dearer. That was an artefact: the charger form usb_in − battery_net folds in taper and a phantom 0.26 W on the offline USB node (3000 µA at 86000 µV with usb/online=0). On battery the difference is +0.15 W ±0.21 (0.7 σ) — no measurable gap. power_now is a frozen constant on this device and must not be used.
The four refutations
Recorded because the wrong model is what would otherwise get built:
- Per-slot fence reap —
x11_reapreturns immediately whenpresent.is_null(); that route is DRI3-only and unused here. VK_KHR_present_waitspinning — nothing chainsVkPresentIdKHR; zerovkWaitForPresentKHRmeasured.- Acquire starvation — arithmetically bounded at ~46 µs/frame.
- UBWC / tiled scan-out — an
AHardwareBuffercannot express UBWC.
Three plan items closed outright by measurement (present-wait, slot reaping, present-mode re-declaration — the last is load-bearing every commit, declared_fifo 0 of 25 009). rt_cap cannot work for an app uid on Android; the obtainable form is nice -20, and it is a latency item with no CPU claim.
Before / after, counted
Opinions about allocations in hot paths are cheap, so the commit path has an instrument that counts instead: modules/gndrm-daemon/tests/present_alloc_bench.rs drives the real Device::dispatch with a linearized ATOMIC flip request (ten plane properties, alternating between two imported framebuffers, 4 000 flips after a 64-flip warm up) behind a counting global allocator. Run in worktrees at each revision so the harness is byte-identical.
| revision | allocs/flip | bytes/flip | ns/flip (min of 3) |
|---|---|---|---|
53a0aa5 — before all four | 6.04 | 915 | 1770 |
43268c5 — +P3/P4 | 6.04 | 915 | 1734 |
e4634a7 — +P1/P2 | 1.04 | 695 | 1831 |
Six allocations per commit become one — −83 %, and −220 bytes. The five that go are the four scratch arrays in parse_atomic and the HashSet the presenter collected from both object tables every frame. The one that remains is the sets array the design deliberately kept (docs/performance/05 §9).
The time is not claimable, and the PR should not be reviewed as if it were. Minimums 1770 / 1734 / 1831 ns, medians 2448 / 2498 / 1860 — the spread between repeats of one build exceeds the spread between builds. At 242 commits/s the whole commit path is ~0.5 ms of CPU per second, so no device arm could resolve it either.
The flat P3/P4 row is not evidence against them: the pollfd hoist lives in the event-loop poll and the method-id cache in the JNI publish path, and the commit dispatch enters neither. Their case is the code argument.
The docs
docs/performance/, numbered 01-08, one document per plan item, each with file:line evidence, the change, the invariants, and the measurement that decides it. Evidence captures under docs/performance/evidence/. The top-level docs/01-23 series stays architecture/subsystem; this is its own series.
Tooling that came out of it and is reusable: a byte-anchored container editor (tools/container-env-flip.py), a one-command census arm (tools/gamescope-cpu-arm.sh), its parser (tools/gamescope-cpu-parse.py), and a scene fingerprint that proves two captures are the same place (tools/scene-fingerprint.py).
Not in this PR
nice -20for gamescope — designed (07), unstarted; a frame-time p99 item, not a CPU one.- The remaining guest +5.1 pp — needs
simpleperf record, not another plan item. The two witnesses that would have justified more shim work are exactly the ones step 1 refuted. - The pointer-click defect — the next step there is instrumented logging, not a speculative patch.
Reviewing it
The two code commits are self-contained and reviewable on their own. 01 is the investigation summary, 05 is what the code implements, 02 is how every number was taken.