Run the gate on macOS in CI too

Merged
JA
3mo ago
+222-31

Why

The WSL port merged with every macOS branch marked "unchanged by construction". That is an argument, not a test result — the plan that produced it said so in as many words. The Linux job proves the port; nothing proves the thing it was ported around.

This is the test result.

What

A macos job on macos-latest (arm64). Sibling of linux, same conventions: pinned actions/checkout SHA, persist-credentials: false, and nvim pinned to the same v0.11.0 tarball checked against the same release shasum.txt the Linux job's checksum came from — so "which nvim did the gate run against" keeps one answer, and a macOS-only failure is the code rather than the editor.

brew install fzf and nothing else. Every omission is argued in the file; the load-bearing one is coreutils — installing it would put gtimeout, not timeout, on PATH, so the shim would not find it anyway, and taking the perl leg is the entire point.

Not one Intel leg: those runner labels are on GitHub's deprecation path, so it would be a check that needs re-pinning rather than one that keeps working.

This job reaches code the Linux job structurally cannot

only ever runs on macOS
tests/bin/timeoutits perl alarm fallback. On Linux the shim finds a real timeout(1) and execs it — so the branch that file was written for has never executed in CI
every #!/bin/bash suitebash 3.2.57
drill.sh, suite_timer.shBSD pgrep/ps, and BSD sed/awk for the secs() check that exists to drop bc

tests/run.sh — probe the pasteboard, don't assume it

The preflight skipped Darwin outright, on "macOS always has pbcopy, so this is silent there". True about presence, and says nothing about capability: the pasteboard lives in a per-session pbs, so over ssh or on a CI runner pbcopy is right there on PATH with nothing to talk to. The six suite_config.sh cases that assert register + would then fail looking like six unexplained mapping regressions.

That is precisely the mistake the xclip branch already stopped making ("a display name is not a display"). Now it is made once, in one shape, on both platforms: write with pbcopy, read back with pbpaste, believe the exit code.

The probe needs timeout, which macOS has not got — so run.sh now hoists tests/bin onto PATH the way run_test.sh and every suite already do for themselves. Side effect: it makes docs/testing.md's existing claim that run.sh does this true.

Drive-by: the case count was wrong

556 in six places; it is 558. suite_options.sh went 84 → 86 when the port added the wsl_quit_fallback_* assertions and no count moved with it. Measured from a run, not counted by hand — the docs table now sums to 558 as well.

Verified locally

  • ./tests/run.sh → ALL SUITES PASSED, 558, 0 failures, 14/14 suites

  • workflow YAML parses; bash -n on all 8 inline run: blocks

  • the Darwin arm proven able to fail — the real preflight cut out of run.sh (not a copy pasted into a test) run under a stubbed uname/pbcopy/pbpaste, four ways:

    stubresult
    pasteboard answerssilent ✅
    pbcopy failswarns, with the macOS-specific text ✅
    pbcopy ok, pbpaste deadwarns ✅
    pbcopy hangs 300swarns after 5s, not 300 ✅
  • the perl timeout leg exercised on a PATH with no real timeout: stdin passes through the pipe, and a hang is still bounded

What is not verified: macOS itself. There is no Mac here — that is the whole reason this job exists. The first macos run is partly discovery. Three things are most likely to come back red and are called out so they get recognised rather than re-diagnosed:

  1. the pasteboard (6 cases) — the new preflight will name it
  2. BSD pgrep -f vs the timer's ~2KB argv (36 cases) — _drill_timer_pids rests on "pgrep matches the whole command line, while ps truncates", measured on Linux; BSD pgrep reads KERN_PROCARGS2 bounded by kern.argmax
  3. pty driver timing — fixed sleeps, on a slower shared runner. Per the port's own rule: raise the constants, do not add retries

Merging only at linux 558 and macos 558.


Summary by cubic

Adds a macos-latest CI job to run the full gate on macOS and hardens the pasteboard probe to be non-destructive, value-checked, and save-gated. Fixes a bash 3.2 parsing bug in the timer suite; suite count is 558.

  • New Features

    • Added a macos job on macos-latest (arm64) alongside linux.
    • Pinned actions/checkout and the nvim v0.11.0 tarball (checksum-verified); persist-credentials: false.
    • Installed only fzf via Homebrew; skipped coreutils to exercise the perl timeout path.
    • Validated demo.sh secs() under BSD sed/awk and bash 3.2.
  • Bug Fixes

    • tests/run.sh probes pbcopy/pbpaste on -pboard find, gates the write on a successful save, restores it, and checks round-trip by value to avoid clobbering the clipboard and prevent data loss on slow pasteboards.
    • Hoisted tests/bin onto PATH so timeout is available during preflight; clearer “no working clipboard” warnings (including macOS session guidance).
    • Fixed suite_timer.sh on bash 3.2 by balancing the case-pattern ( inside $(...), unblocking all 36 timer cases on macOS.
    • Updated docs: total cases 558; suite_options.sh is 86; corrected WSL clipboard flake math and guidance to read results as rates and consistency, not load or single-run verdicts.

Summary by CodeRabbit

  • New Features

    • Added macOS coverage to automated testing, including clipboard validation and platform-specific tooling checks.
    • Improved test setup with bundled tool discovery and clearer clipboard-provider diagnostics.
  • Bug Fixes

    • Fixed timer test compatibility with older Bash versions.
  • Documentation

    • Updated testing documentation and project references to reflect 558 total tests, including 86 option-suite cases.
    • Documented Ubuntu and macOS CI coverage, clipboard requirements, and troubleshooting guidance.

Greptile Summary

Adds a macOS CI gate alongside Linux, exercising platform-specific shell, utility, clipboard, and timeout behavior against the pinned Neovim release.

  • Adds a checksum-verified Neovim installation and full test run on macos-latest.
  • Probes the macOS pasteboard with bounded, non-destructive round-trip checks.
  • Makes the timer suite parse under macOS Bash 3.2.
  • Updates the documented test count from 556 to 558.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

FilenameOverview
.github/workflows/tests.ymlAdds the macOS CI job, pinned Neovim installation, dependency setup, full gate, and BSD utility compatibility check.
tests/run.shAdds the bundled timeout shim to PATH and introduces a bounded macOS pasteboard capability probe with restoration.
tests/suite_timer.shBalances the case-pattern parenthesis so the command substitution parses under Bash 3.2.
docs/testing.mdUpdates suite counts and documents dual-platform CI and revised WSL clipboard-flake interpretation.

Reviews (8): Last reviewed commit: "Correct the flake arithmetic: isolation ..." | Re-trigger Greptile