renderer: one render target per gesture (ghostty_surface_set_host_capacity)

Closed
LA
10h ago
+955-188

Problem

cmux contract m14: in every frame, a moving terminal presents a frame drawn at its pane's size. A held-key resize of 14 terminals therefore makes about one present per moving terminal per frame (805-1022 per round; 84 with the switch off). That count is the contract, so each present must cost less.

Today each new host size makes FrameState.resize -> api.initTarget: a new IOSurface and a new MTLTexture for each swap-chain frame at each new size. Time Profiler on cmux's held keys-stress, switch on (per round):

switch offswitch on
IOSurfaceCreate7 ms57 ms
newTextureWithDescriptor:iosurface15 ms36 ms
IOSurface client callsabout +36 ms
__mmap30 ms104 ms

Each new IOSurface as layer contents is also a new surface for WindowServer.

Design

ghostty_surface_set_host_capacity(surface, width_px, height_px) (macOS; 0, 0 clears; default none, so no change in behavior without it):

  • One target per swap-chain frame per gesture. While a capacity is set, Metal.fitTarget (called by FrameState.fit from drawFrame) keeps each target at least the capacity (Metal.targetSize). A frame at a new size that fits allocates nothing. A size past the capacity grows that target once, to that size (a target never shrinks while the capacity is set). Cleared, the next frame of each target gets a target of exactly its size. The floor is one allocation per swap-chain target (3), not one per terminal: a target in flight or shown cannot be redrawn.
  • Drawn at the host size, top-left. Target.drawn_width/drawn_height is the frame's size. RenderPass.begin sets the viewport and the scissor to it for a target larger than its frame (no encoder call otherwise). The projection, the uniforms and the fragment positions already use the screen (host) size, with the origin at the top-left. Custom shader textures keep the frame's size (they are plain textures and still follow each size).
  • The layer shows only the drawn pixels. Every present goes through IOSurfaceLayer.showLocked, which sets contents and contentsRect (drawnRect) in one [CATransaction lock] span: the four present paths (setSurfaceSync, setSurface/setSurfaceCallback, setSurfaceOffMain, setSurfacePresented). setSurfaceSync now always takes the lock for this (before, only in embedder mode). contentsGravity stays top-left, so bounds ahead of the frame still show it at the top-left, unscaled. The unit rect's y: macOS unit coordinates start at the bottom, so a top-left sub-rect is (0, 1 - h/H, w/W, h/H); the CARenderer test proves it (see Tests).
  • One source of truth for sizes. A frame carries its drawn size (IOSurfaceLayer.Drawn, Metal.fresh.width/height). The shown frame's size is read from the layer itself (shownSizeLocked: contentsRect x the IOSurface size, under the CA lock), so the size can never disagree with what the layer shows, and a contents change by the embedder still counts. No ivar copy.
  • Counters, appended at the end of ghostty_surface_render_counters_s (C and Zig, same order): host_target_allocs (all causes: a new swap chain, a new size, a capacity, a config change) and host_target_bytes (peak bytes of the live target IOSurfaces). Target.live_bytes lowers the live count when a target is freed; the swap chain now gets the API by pointer so initTarget counts on the renderer's own Metal, not on a copy.
  • ghostty_surface_shown_frame_size reports the shown frame's drawn size (the IOSurface can be larger). See "cmux follow-up".

Every size comparison changed, and why

All compared the IOSurface's size; with a capacity target that is the capacity, not the frame. Each now compares the drawn size:

WhereBeforeNow
IOSurfaceLayer.changesShownSizeshown surface vs new surfaceshown drawn size vs the frame's drawn size
IOSurfaceLayer.shownSize (used by Metal.hostFrame, displayWhileEmbedderPresents, the C API)shown surface sizeshownSizeLocked (contentsRect x surface)
IOSurfaceLayer.setSurfaceOffMain host checksurface vs host sizedrawn size vs host size
IOSurfaceLayer.offMainDecisionLockedshown surface vs frame surfaceshown drawn vs frame drawn
IOSurfaceLayer.embedderDrops (and embedderShowsFrameLocked)shown surface vs new surface; bounds vs new surfacedrawn sizes for both
IOSurfaceLayer.setSurfaceCallback wrong-size discardhost/bounds vs surfacehost/bounds vs drawn size
Metal.takeFresh, Metal.hasFreshfresh surface sizefresh drawn size (fresh.width/height)
Metal.presentHostSizedWaitshown surface vs host (unlocked read)layer.shownSize() (drawn, locked) vs host
Metal.embedderShowsKeptFramepassed the surfacepasses the drawn size (identity check still on the surface)
drawFrame target checktarget size == screen sizeFrameState.fit -> Metal.targetSize (fits, with a capacity)

Unchanged: Metal.setHostDrawableSize checks only whether contents is null; Overlay.zig and the overlay check in generic.zig read z2d surfaces, not IOSurfaces; test helpers testShownWidth (tests without a capacity).

Tests

All named host present: (the workflow's existing filter). New:

  • Metal.zig "a held-key resize makes at most one render target per swap-chain frame with a host capacity": a real Metal device (Device.initForTest), 3 swap-chain targets at the pane's size, capacity 160x60, 24 frames at new sizes (81..150 px): each target is replaced at most once and target_allocs grows by at most 3 (one per target, not 24); each frame's drawn size is its host size; the layer shows exactly the drawn pixels (shownSize, contents, contentsRect = top-left drawn/target). Then a size past the capacity: exactly 1 more allocation, none at the next smaller size. Then clearing: the next frame's target has exactly the host size, contentsRect is the unit rect. Peak and live bytes checked.
  • Metal.zig "without a host capacity each new size makes a target of exactly that size" (the guard): one allocation per new size, exact size, unit rect; same size again allocates nothing.
  • Metal.zig "a frame in a capacity target is drawn only in its top-left drawn pixels": a real render pass (RenderPass.begin, the real bg_color pipeline) into an 8x8 capacity target for a 6x4 frame; the IOSurface read back has the background in the top-left 6x4 and clear elsewhere (viewport and scissor).
  • IOSurfaceLayer.zig "a capacity frame shows only its drawn pixels, top-left": Core Animation renders the layer (CARenderer into a Metal texture). A reference render of the whole surface calibrates the output orientation; the capacity frame (6x4 of 8x8) must show exactly the reference's pixels for surface rows 0..3, columns 0..5: none of the padding, unscaled, the right way up; also in bounds larger than the frame (top-left). This proves the contentsRect y orientation.
  • IOSurfaceLayer.zig "a frame that fills its surface keeps the whole surface as its contents rect" and "an off-main frame of a capacity target compares its drawn size, not its surface's".

Every existing host present: test runs unchanged except for the call syntax (.whole(surface)).

Runs (fork CI, workflow cut to the host present: filter; scratch branches never merge)

  • Red, scratch/host-capacity-red-2 = red commit fd3f326bb + the cut: run 41971754811. Fails to compile, as expected: the API does not exist (src/renderer/metal/IOSurfaceLayer.zig:1097:34: error: use of undeclared identifier 'Size', and at 1140:34; Build Summary: 92/98 steps succeeded (1 failed)). Zig stops at the first missing declaration; Drawn, setHostCapacity, fitTarget and the counters are missing too.
  • Green, scratch/host-capacity-green-2 = fix commit 362a355ac + the cut: run 41971758033: Build Summary: 98/98 steps succeeded; 117/117 tests passed (every host present: test, the 6 new ones included) and lib-vt 46/46 steps succeeded; 97/97 tests passed.
  • Earlier pair, before the draw test was added to the red commit: red 41969943131 (IOSurfaceLayer.zig:1097:34: error: use of undeclared identifier 'Size'), green 41970220734 (116/116, lib-vt 97/97).
  • Probe 41970106668 (scratch/host-capacity-probe): the CI runner has a Metal device ("Apple Paravirtual device") and CARenderer renders there; for an IOSurface, contentsRect y = 0 selects the surface's bottom rows, y = 0.5 its top half.

What the tests do not cover: drawFrame itself (it needs a full renderer: font grid, terminal state); its wiring to FrameState.fit is read, not run. The terminal's own cell pipelines at a capacity size (only bg_color is drawn). Custom shaders with a capacity. The C entry points (ghostty_surface_set_host_capacity, ghostty_surface_shown_frame_size, the two counters) are compile-checked only. No cmux end-to-end run.

cmux follow-up

cmux reads the shown frame from layer.contents (TerminalSurfaceView.presentedFrame, an IOSurface) and takes its width/height as the frame's pixel size. With a capacity that is the capacity. cmux must:

  1. Call ghostty_surface_set_host_capacity(surface, w, h) at the gesture's start for each terminal that will move (the largest pixel size the gesture can give it), and ghostty_surface_set_host_capacity(surface, 0, 0) at the landing, before the landing size's ghostty_surface_set_size, so that frame is drawn in a target of its own size.
  2. Read the shown frame's size with ghostty_surface_shown_frame_size, not presentedFrame.width/height: TerminalSurfaceView+MotionSample.swift (shown?.width, shown?.height), TerminalSurfaceCommitSizeTests (sample()'s frame.width/height and shownPixels).
  3. Anything that shows or pictures presentedFrame elsewhere must show only the drawn part: TerminalFrameHold.HeldFrameView.show copies contents, gravity and scale from the source layer; it must also copy contentsRect. TerminalLastFrame / showLastFrame (a kept IOSurface set as another layer's contents) must carry the contentsRect with it. TerminalSnapshotRenderer (CIImage(ioSurface:)) must crop to the drawn rect (CIImage's origin is bottom-left: (0, H - h, w, h)). A frame drawn after the capacity is cleared is whole again, but the shown frame stays a capacity frame until the next frame is drawn.
  4. layer.contents = nil (a purged frame) needs no change.

Risks

  • Memory: during a gesture each moving terminal keeps 3 targets at the capacity (today about 3 at the current size, plus the churn). A capacity much larger than the gesture's real sizes costs memory until the landing; host_target_bytes reports the peak. After clearing, each target shrinks at its next frame; a surface that draws nothing after the landing keeps the capacity targets until it draws or is hidden.
  • GPU: the render pass clears the whole target (load action clear), not only the drawn part: a little more bandwidth per frame while a capacity is set.
  • setSurfaceSync now takes [CATransaction lock] in the default mode too (it sets contents and contentsRect together). It never commits under it; the lock is recursive.
  • An embedder that sets layer.contents itself while the layer's contentsRect is a sub-rect would make the read size wrong (see follow-up 3).
  • host_target_allocs counts all causes, including the 3 1x1 placeholder targets of every new swap chain.
  • Not ported to the sync line (release/cmux-sync-20261104).