PR #82 - manaflow-ai/ghostty-next

Replaces #81: the red commit now fails on behavior (it compiles), and the landing gives the capacity-sized targets back.

Problem

cmux contract m14 says that in every frame, a moving terminal presents a frame drawn at its pane's size. So a held-key resize of 14 terminals makes about one present per moving terminal per frame (805-1022 per round; 84 with the switch off). That count is the contract, so each present must cost less.

Today each new host size runs FrameState.resize -> api.initTarget, which makes a new IOSurface and a new MTLTexture for each swap-chain frame at each new size. Time Profiler on cmux's held keys-stress, per round:

	switch off	switch on
IOSurfaceCreate	7 ms	57 ms
newTextureWithDescriptor:iosurface	15 ms	36 ms
IOSurface client calls		about +36 ms
__mmap	30 ms	104 ms

Each new IOSurface set as layer contents is also a new surface for WindowServer.

The bound: 1 allocation per swap-chain target per gesture

With a capacity, each swap-chain target allocates at most once per gesture: once, to reach the capacity. With swap_chain_count 3, that is 3 per terminal per gesture, not 1. It cannot be 1 per terminal: of the 3 targets, one can be in flight (the GPU draws it, and its completion presents it) and one is on screen. Neither can be redrawn, so a second size needs a second target, and so on. The Lead's red criterion said "at most 1 surface allocation per terminal per gesture". The test asserts exactly 1 per swap-chain target (3 in total for 24 frames at new sizes; today it is 24).

Design

ghostty_surface_set_host_capacity(surface, width_px, height_px) (macOS; 0, 0 clears; default none, so behavior is unchanged without it):

Targets. While a capacity is set, Metal.fitTarget (called by FrameState.fit from drawFrame) keeps each target at least the capacity (Metal.targetSize). A frame at a new size that fits allocates nothing. A size past the capacity grows the target it is drawn into once, to that size (a target never shrinks while the capacity is set).
Drawn at the host size, top-left. Target.drawn_width/drawn_height is the frame's size. For a target larger than its frame, RenderPass.begin sets the viewport and the scissor to it; otherwise it makes no encoder call. The projection, the uniforms and the fragment positions already use the screen (host) size with the origin at the top-left. Custom shader textures keep the frame's size.
The layer shows only the drawn pixels. Every present goes through IOSurfaceLayer.showLocked, which sets contents and contentsRect (drawnRect) in one [CATransaction lock] span. The four present paths are setSurfaceSync, setSurface/setSurfaceCallback, setSurfaceOffMain and setSurfacePresented. contentsGravity stays top-left, so bounds ahead of the frame still show it at the top-left, unscaled. The y flip: for an IOSurface, a contentsRect y of 0 is the surface's bottom row (probe run 41970106668, and the CARenderer test below), so the top-left sub-rect is (0, 1 - h/H, w/W, h/H).
One source of truth for sizes. A frame carries its drawn size (IOSurfaceLayer.Drawn, Metal.fresh.width/height). The shown frame's size is read from the layer itself (shownSizeLocked = contentsRect x the IOSurface size, under the CA lock), so it cannot disagree with what the layer shows, and a contents change by the embedder still counts.
The landing gives memory back, and never waits (revised after review comment 6652213651). A clear arms a trim (Metal.trim_armed), marks the landing frame due (Metal.landing_due) and wakes the renderer thread (draw_now, one renderer_draw_now). In its next draw, Renderer.SwapChain.trim never waits: it gives back at once every target larger than the host size that no frame is drawing (Target.in_flight, set by Metal.frameCommitted at the commit and cleared by Metal.frameDone at the completion; Target.release leaves the target empty until the next frame drawn into it makes one of its size). That draw then draws the landing frame at the host size. A target whose frame is in flight goes back when that frame completes: Metal.frameDone runs on the GPU completion thread after the present (outside the Core Animation lock) and before the swap-chain slot is released, and releases the target if it is larger than the size the next frame draws at. No later draw is needed, so a terminal that never draws again (no output, a hidden cursor) keeps no capacity-sized target. The trim disarms only when no frame is in flight. A new capacity before then disarms it (the next gesture reuses the targets).
Counters, appended at the end of ghostty_surface_render_counters_s (C and Zig, same order): host_target_allocs (new render targets, all causes), host_target_bytes (the peak bytes of live target IOSurfaces, as agreed with frame-owner-s2), and host_target_live_bytes (live target IOSurface bytes now). Target.live_bytes lowers the live count when a target is freed. The swap chain now gets the API by pointer, so initTarget counts on the renderer's own Metal, not on a copy.
ghostty_surface_shown_frame_size reports the shown frame's drawn size (the IOSurface can be larger). See "cmux follow-up".

Every size comparison changed, and why

Each of these compared the IOSurface's size. With a capacity target that is the capacity, not the frame. Each now compares the drawn size:

Where	Before	Now
`IOSurfaceLayer.changesShownSize`	shown surface vs new surface	shown drawn size vs the frame's drawn size
`IOSurfaceLayer.shownSize` (used by `Metal.hostFrame`, `displayWhileEmbedderPresents`, the C API)	shown surface size	`shownSizeLocked` (contentsRect x surface)
`IOSurfaceLayer.setSurfaceOffMain` host check	surface vs host size	drawn size vs host size
`IOSurfaceLayer.offMainDecisionLocked`	shown surface vs frame surface	shown drawn vs frame drawn
`IOSurfaceLayer.embedderDrops` (and `embedderShowsFrameLocked`)	shown surface vs new surface; bounds vs new surface	drawn sizes for both
`IOSurfaceLayer.setSurfaceCallback` wrong-size discard	host/bounds vs surface	host/bounds vs drawn size
`Metal.takeFresh`, `Metal.hasFresh`	fresh surface size	fresh drawn size (`fresh.width`/`height`)
`Metal.presentHostSizedWait`	shown surface vs host (unlocked read)	`layer.shownSize()` (drawn, locked) vs host
`Metal.embedderShowsKeptFrame`	passed the surface	passes the frame (sizes drawn; identity still on the surface)
`drawFrame` target check	target size == screen size	`FrameState.fit` -> `Metal.targetSize` (fits, with a capacity)

Unchanged: Metal.setHostDrawableSize only checks whether contents is null. Overlay.zig and the overlay check in generic.zig read z2d surfaces, not IOSurfaces.

Default-path changes (no capacity, cmux with the switch off)

setSurfaceSync now takes [CATransaction lock] in renderer mode too. Before, it took the lock only while the embedder presents host sizes. Now it always sets contents and contentsRect together under the lock, because an off-main present reads both under it. It never commits or flushes under the lock (the lock is recursive). It runs on the main thread (display pass, ghostty_surface_draw) and in the completion block of a sync draw.
Every present also writes contentsRect, but only when it differs (showLocked compares first). With no capacity it stays the unit rect, so no property is written.
The swap chain gets the API by pointer, and FrameState.fit replaces the size check in drawFrame with the same rule as before when no capacity is set. SwapChain.trim returns at once while no trim is armed.

Tests (red on behavior, then green)

All new tests are named host present: (the workflow's existing filter). The Metal.zig ones use a real Metal device (Device.initForTest) and the renderer's real swap chain (Renderer.SwapChain, nextFrame, FrameState.fit, trim, releaseFrame), without the GPU (testDraw presents and completes each frame).

Test	Red (stubs, today's behavior)	Green
a held-key resize makes one render target per swap-chain frame with a host capacity: 24 frames at new sizes inside the capacity; each drawn at its host size and shown exactly (contentsRect, top-left); past the capacity, the target grows once, then nothing	`expected 3, found 24`	pass
clearing the host capacity at the landing leaves one target, at the pane's size: capacity set, 12 sizes, clear, one landing draw: live bytes = the on-screen target at the pane's size, both spares empty	`expected 3404800, found 13529600`	pass
the landing gives back the idle targets at once and never waits for a frame in flight (review F2): the trim returns in < 4 ms with a frame in flight and gives back the idle targets; that frame's completion gives back its own, no draw	first pair: n/a; second pair: `the landing trim took 9043417 ns`, `TestUnexpectedResult`	pass (`12417 ns`)
a frame in flight at the landing gives back its capacity target when it completes, with no other draw (review F1)	second pair: `expected 3404800, found 14694400`	pass
without a host capacity each new size makes a target of exactly that size (the guard; nothing given back)	pass	pass
a frame in a capacity target is drawn only in its top-left drawn pixels (real render pass, `bg_color` pipeline, IOSurface read back)	`surface pixel (6, 0): alpha 255`	pass
a capacity frame shows only its drawn pixels, top-left (Core Animation renders the layer with `CARenderer`; proves the y flip)	`expected 6, found 8`	pass
a frame that fills its surface keeps the whole surface as its contents rect	`expected 120, found 160`	pass
an off-main frame of a capacity target compares its drawn size, not its surface's	`TestUnexpectedResult`	pass

Commit 1 is the red: it adds the C API declarations, the entry points, ghostty_surface_shown_frame_size (returning today's IOSurface size), the counters (they count), the seams with today's behavior (setHostCapacity does nothing, fitTarget uses the exact-size rule, SwapChain.trim gives nothing back, the layer still shows and compares the whole IOSurface), and the tests. Commit 2 adds the behavior.

Runs (fork CI, workflow cut to the `host present:` filter; scratch branches never merge)

Red, scratch/host-capacity-red-3 = red commit 2460a557d + the cut: run 41977016390: Build Summary: 96/98 steps succeeded (1 failed); 112/119 tests passed (7 failed). It compiles; the 7 failures are the behavior lines in the table. All 33 existing host present: tests pass in it.
Green, scratch/host-capacity-green-3 = fix commit 4a4182c22 + the cut: run 41977364169: Build Summary: 98/98 steps succeeded; 119/119 tests passed (the 33 existing and the 8 new host present: tests) and lib-vt 46/46 steps succeeded; 97/97 tests passed.
Probe 41970106668: the CI runner has a Metal device ("Apple Paravirtual device") and CARenderer renders there. For an IOSurface, contentsRect y = 0 selects the surface's bottom rows.

Review fixes (comment 6652213651): runs

Red, scratch/host-capacity-red-4 = red commit 47c05f27c (test only; Metal.frameCommitted/frameDone are no-op seams called from Frame.zig) + the cut: run 41987264371: 118/120 tests passed (2 failed). The failures: the landing trim took 9043417 ns with a frame in flight (the 8 ms wait), and expected 3404800, found 14694400 (two capacity-sized targets kept after the landing with a frame in flight).
Green, scratch/host-capacity-green-4 = fix commit f867bd2bf + the cut: run 41987511810: 98/98 steps succeeded; 120/120 tests passed, lib-vt 97/97. The trim took 12417 ns; live bytes after the landing with a frame in flight: 3404800 (the landing target at the pane's size), peak 19801600.

Live bytes (the landing test, 1000 x 700 px pane, capacity 2000 x 700, BGRA)

	red (today)	green
live target bytes during the gesture	14,873,600	16,934,400 (3 targets at the capacity)
live target bytes after the landing	13,529,600 (the 2 spares keep their sizes)	3,404,800 = the on-screen target at the pane's size (1200 x 700)
peak (`host_target_bytes`)	19,443,200	19,801,600

What the tests do not cover: drawFrame itself, which needs a full renderer (font grid, terminal state). Its calls to FrameState.fit and SwapChain.trim were read, not run. Also not covered: the renderer thread wake on clear (draw_now, read only); the terminal's own cell pipelines at a capacity size (only bg_color is drawn); custom shaders with a capacity; and the C entry points (compile-checked only). No cmux end-to-end run.

cmux follow-up

cmux reads the shown frame from layer.contents (TerminalSurfaceView.presentedFrame, an IOSurface) and takes its width/height as the frame's pixel size. With a capacity, that is the capacity. cmux must:

Call ghostty_surface_set_host_capacity(surface, w, h) at the gesture's start for each terminal that will move, with the largest pixel size the gesture can give it. Call ghostty_surface_set_host_capacity(surface, 0, 0) at the landing, before the landing size's ghostty_surface_set_size, so the landing frame is drawn in a target of its own size and the spares are given back.
Read the shown frame's size with ghostty_surface_shown_frame_size, not presentedFrame.width/height: in TerminalSurfaceView+MotionSample.swift (shown?.width, shown?.height) and in TerminalSurfaceCommitSizeTests (sample()'s frame.width/height and shownPixels).
Anything that shows or pictures presentedFrame elsewhere must show only the drawn part. TerminalFrameHold.HeldFrameView.show copies contents, gravity and scale from the source layer; it must also copy contentsRect. TerminalLastFrame / showLastFrame (a kept IOSurface set as another layer's contents) must carry the contentsRect with it. TerminalSnapshotRenderer (CIImage(ioSurface:)) must crop to the drawn rect (CIImage's origin is bottom-left: (0, H - h, w, h)). After the landing frame is drawn, the shown frame is whole again.
Optional: read host_target_live_bytes and host_target_allocs in the held-key bench to confirm 3 allocations per terminal per gesture and the landing's memory.
layer.contents = nil (a purged frame) needs no change.

Risks

Memory during a gesture. Each moving terminal keeps 3 targets at the capacity (today about 3 at the current size, plus the churn). A capacity much larger than the gesture's real sizes costs memory until the landing; host_target_bytes reports the peak. At the landing, the idle targets go back in the landing draw and a target in flight goes back when its frame completes.
Completion thread. Metal.frameDone now releases a target (an IOSurface and a texture) on the GPU completion thread after a clear, outside the CA lock. It calls surfaceSize there; for a surface that is not host-sized that reads the layer's bounds off the main thread (the renderer thread already does). Target.in_flight hands the target between the threads (release/acquire); the renderer thread never touches a target in flight.
One more frame at the landing. A clear makes the renderer thread draw one frame at the host size (one renderer_draw_now), even when nothing else changed.
GPU bandwidth. The render pass clears the whole target (load action clear), not only the drawn part, so there is a little more bandwidth per frame while a capacity is set.
The lock. setSurfaceSync now takes [CATransaction lock] in the default mode too. It never commits under it.
Embedder contents. An embedder that sets layer.contents itself while the layer's contentsRect is a sub-rect would make the read size wrong (see follow-up 3).
Counter scope. host_target_allocs counts all causes, including the 3 1x1 placeholder targets of every new swap chain.
Not ported to the sync line (release/cmux-sync-20261104).

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}

Summary by cubic

Reduces held-key resize cost on macOS by giving each swap-chain render target one allocation per gesture instead of one per frame per new size. A new ghostty_surface_set_host_capacity(surface, w, h) keeps every swap-chain target at least the gesture's capacity, so frames at new sizes that fit allocate nothing, and 0, 0 clears the capacity and gives the memory back at the landing. Without the call, behavior is unchanged, including cmux with its switch off.

Each swap-chain target (3 per surface) allocates at most once per gesture: one can be in flight and one is on screen, and neither can be redrawn, so it cannot be one per surface.
Frames are drawn at the host size from the target's top-left; the render pass sets viewport and scissor, and the layer sets contents and contentsRect together under the Core Animation lock at every present, with the IOSurface y flip.
Clearing the capacity wakes the renderer thread, which never waits: it releases at once every target larger than the host size that no frame is drawing and draws the landing frame at the host size; a target whose frame is in flight is released when that frame completes, with no further draw.
ghostty_surface_shown_frame_size reports the drawn frame's size (the surface can be larger), the render counters gain host_target_allocs, host_target_bytes (peak), and host_target_live_bytes, and all present-mode size comparisons now use drawn sizes instead of the surface's.

Migration

cmux should set the capacity at a gesture's start, clear it at the landing before the final ghostty_surface_set_size, and read the shown size via ghostty_surface_shown_frame_size instead of presentedFrame.width/height.
Embedders that set layer.contents themselves or copy contents properties into other layers must carry contentsRect while a capacity is set.