#24598 Mac Screen: stream at up to 120 fps when both displays can

Mac Screen now streams at up to 120 fps when both Macs' displays can show it. Before this change, it was always capped at 60.

How the rate is agreed

• The viewer reports the most frames per second its display can show: NSScreen.maximumFramesPerSecond for the pane's window. It sends this in a new fps input message, and sends it again when the window moves to another display or a display changes mode.
• For Auto and Smooth Motion, the host streams at the lower of that rate and its captured display's rate, clamped between 60 and 120. Fixed presets stay at their own rates.
• At the agreed rate, the host sets ScreenCaptureKit's minimum frame interval and queue depth (6 at 120 fps) and the sender's maxFramerate.
• The bitrate ceiling scales with the rate so each frame keeps its quality: 12 Mbps at 60 fps, 24 Mbps at 120 (200 kbit a frame).
• When either side's rate changes, the host re-applies all of this. It reports the agreed rate in the displays message (fps).
• The toolbar stats line shows the measured and target rates, for example "118/120 fps". The string is localized.

Compatibility. Peers that predate this change stay at 60 fps, in both directions:

• An older host passes the fps message to its source, which ignores it.
• A viewer that never sends it (the iOS demo apps, older cmux builds) gets 60.
• A viewer that predates the displays fps field ignores the field.

The Screen Sharing loopback source and the cursor drawn in the picture are unchanged. AVSampleBufferDisplayLayer needs no rate hint on macOS: frames are pushed and shown on the next refresh.

Why the first build reached only 60, and the fix. Three caps, measured:

1. The viewer reported 60. The user's laptop had only its two LG monitors online (both 60 Hz); the built-in ProMotion panel wasn't. With either Mac at 60, the agreed rate is 60, by design.
2. libwebrtc's H.264 level cap. RTCVideoEncoderH264 caps the frame rate at what the negotiated level allows for the frame size and sets it as VideoToolbox's expected frame rate. Level 5.2 at 3456×2234 gives 68 fps; libwebrtc knows no higher level. ScreenEncoderFactory now opens the encoder at VideoToolbox's AutoLevel, which has no cap. The SDP still says 5.2; the decoder reads the real level from the stream.
3. VideoToolbox's H.264 throughput. The hardware encoder is bound by pixels per second, whatever the level or settings: about 400 Mpixel/s on an M1 Max and 500 on an M4 Pro, so 55 and 66 fps at 3456×2234. Above 60 fps, Auto and Smooth Motion now scale the picture to at most 360 Mpixel/s: a 16" MacBook Pro streams at about 2154×1392 at 120 fps, and a 14" at about 2149×1396. At 60 fps (and with every older peer) the full size is kept. The fixed presets are unchanged.

Trade-off: at 120 fps, text on a 16" MacBook Pro's screen is sent at 62% of its pixel size (the Retina 2x picture becomes about 1.25x). For full sharpness, choose Fixed High (60 fps, full size). HEVC would hold about 98–115 fps at full size, but the bundled libwebrtc has no H.265 encoder.

DEV-only tools added for this check: debug.mac_screen display_fps {fps?} makes a pane report a refresh rate instead of its display's (a 60 Hz viewer can test the 120 fps path), and quality {quality} picks the Quality menu's choice. Rows now show target_fps, display_fps, and WebRTC's frames received, decoded and dropped. The host's 3 s encoder log line adds capture fps, source fps, encode ms, bytes, RTT and the target rate. In DEV builds, CMUX_MAC_SCREEN_H264_LEVEL=negotiated restores the level-5.2 encoder for comparison. None of these is in Release.

Verification

Red, then green, in one process on a quiet M4 Pro (mini-cmux15, under the fleet host lock). HighFrameRateStreamTests runs the host's real path (ScreenStreamHost, hardware H.264, the sender parameters) to a viewer that reports 120 Hz. The source is a synthetic 3456×2234 display scrolling at 120 fps; the test passes at ≥110 decoded fps.

commit	what	Auto: decoded fps (size)	host encoder
2ef08c74 (red)	level 5.2	39 (3456×2234)	11–46 fps, limited-by none
780b1e12 (red)	AutoLevel, full size	67 (3456×2234)	67 fps, encode 98 ms, limited-by cpu
ad1b4850	scaled to 360 Mpixel/s	119.9 (2154×1392), 0 dropped	120 fps, encode 23 ms, limited-by none

Smooth Motion decoded 119–120 fps in every row: maintainFramerate lets WebRTC lower the resolution (to 538×348 here), as it already does at 60.

Encoder throughput (VideoToolbox directly, with libwebrtc's settings, 120 fps input):

size	M1 Max (the 16" host)	M4 Pro
3456×2234 at level 5.2	44 fps	66
3456×2234 at AutoLevel	55 fps	66
3024×1964	70	85
2304×1489	115 (87 ms)	120
2160×1396	120 (28 ms)	120

Tests (fleet, at the head unless noted):

• FrameRateTests.aboveSixtyThePictureFitsTheEncoder and fixedPresetsScaleAsBeforeAtAnyRate failed at 780b1e12 and passed at ad1b4850. ScreenEncoderFactoryTests passed (VideoToolbox encodes at AutoLevel at 3456×2234@120 and 1080p60). Both ran on mini-cmux15.
• MacScreenFrameRateTests.aDevOverrideIsReportedInsteadOfTheDisplaysRate: covers the DEV override.
• App compile-check passed at 494a4434 (fleet job f3fd2193). At f0cb6650 on the fleet: CmuxVNCWebRTC 13/13 suites passed (step a6b476a6), CmuxScreenKit passed (65bff010), all 31 MacScreen and RemoteView suites passed (dd3b4ef4), and the Release compile succeeded (f3aecfae). Since f0cb6650 only the two tooltip commits changed.
• iOS demos: CmuxVNCiOS and CmuxVNCRustiOS built (Release, iOS Simulator, Xcode 26.6) at 5febeebb on mini-cmux12s. Since then, only tests, comments and DEV-only code changed.
• Correctness review (subagent) found no high or medium issues. Its lows are fixed in 0be614fd.
• Base red fixed here: NativeTooltipGuardTests flagged four Mac Screen lines. MacScreenShareIndicator.swift:48 set a raw toolTip; it now sets themedToolTip (70ba946c), as #24532 does. MacScreenToolbar.swift:115/129/162 already set themedToolTip, but through a private helper named setToolTip(, which the guard's scan matches; it's renamed setThemedTip (b7a4bc95). The guard failed with those 4 lines plus MenuIdentityHeaderView.swift:267 at f0cb6650 (step 782c0572) and 70ba946c (b2606cf2); at b7a4bc95 (step 26bda0c3) its only offender is MenuIdentityHeaderView.swift:267. MenuIdentityHeaderView.swift:267 is #24532's to fix.

Observed / Not run

• Not yet run: the two-Mac session (the other MacBook's built-in ProMotion display as host → a mini viewer with display_fps 120). The new host build has no Screen Recording or Accessibility permission on that Mac, and an agent may not grant them. Until it runs, 120 fps on real hardware across two Macs is measured only in the loopback test above, which uses the same encoder.
• Presentation on a real ProMotion viewer display hasn't been measured; the minis' displays are 60 Hz.
• The 360 Mpixel/s budget is one value for every Mac. It was measured on M1 Max and M4 Pro only.

Changelog

• Added: Mac Screen streams at up to 120 fps when both Macs' displays support it (Auto and Smooth Motion; above 60 fps the picture is scaled to what the Mac's hardware encoder sustains). The stats line shows the measured and target rates.

🤖 Generated with Claude Code