PR #22557 - manaflow-ai/cmux

Summary

Every cmux INTERNAL upload from the continuous iPhone builder now carries a readable TestFlight "What to Test" note, written with no manual step. Before this, the note was git log --format='- %s' over iOS paths: engineering subjects, cut at 40 lines, and nothing at all when base was empty (the newest build, 9218b994f0, says only its first line).

Content. scripts/ci/ios_internal_notes.py collects the ## Changelog lines of the PRs whose commits in BASE..SHA change the app. App paths are cnx-ios/, AppIcon.icon/, cmux-wg, cmux-transport, and every local package cnx-ios/CnxKit/Package.swift names at SHA; builder scripts don't count. Lines are grouped under Added / Changed / Fixed / Removed. The parser drops none, dedupes, strips markdown and template comments, and ignores footers after the section (bot badges, "Generated with", trailers). A PR with no Changelog section contributes its title. A commit pushed straight to the branch contributes its subject under "Other changes", without its area label (cnx-ios:, cmux NEXT perf-x:) or trailing references ((#N), (GhosttyNextKit abc1234, cx-…)). Test-only, CI, docs and red commits are dropped.

PR lookup. Batched GraphQL in two reads per batch of 25 (GitHub quota is shared). The first read takes links only: associatedPullRequests(first: 30) plus the (#N) in the subject. A light first read is needed because for a non-default branch GitHub also lists open PRs that branched from the commit, and a 40-commit query with bodies returned 502. The second read fetches the bodies of the chosen PRs. Preference order: the merged PR whose merge commit is the commit, then one merged into feat-cmux-next, then any merged one. A refused batch is split in half and retried. If lookup fails entirely, commit subjects stand in.

Shape. The first line is cmux-next iPhone <version>, feat-cmux-next <12-hex>. Internal testers only., then the range, then the groups. The note stays under App Store Connect's 4000 characters, counted in UTF-16 units. Items that don't fit become …and N more; no line is cut.

Never lose the range. base comes from plan (the last successful builder run). When it is empty, a new previous-build job reads the newest processed (VALID) cmux INTERNAL build and takes the commit its note's first line records. Both the old …continuous build of feat-cmux-next <10-hex> line and the new one parse. testflight-build.sh already stamps CNXSourceCommit into Info.plist, but App Store Connect doesn't expose Info.plist, so the note's first line is the readable record. If that fails, the newest successful run of this builder or of the retired cnx-ios-testflight.yml lane (its head_sha; 3e6a2b56eff was 1.2.0) is used. If that fails too, the note says "First continuous build" with the last 20 Changelog lines.

Secrets.

notes runs on ubuntu-24.04 with contents, pull-requests and actions read only, no environment. It runs the builder branch's script and reads feat-cmux-next history as data.
previous-build runs only when base is empty and holds only the App Store Connect key (no certificate, no profile). It signs the JWT with openssl, so it needs no Python package.
upload (the signer) downloads the note artifact and passes it on. If the artifact is missing, it sends the first line alone, which still records the commit for the next build.
Both new jobs are continue-on-error, because the trigger reads run conclusions as "uploaded or not".

Admin step for this to take effect

The builder runs its workflow file and scripts from the admin-only branch rc/cnx-ios-builder (the release environment admits rc/**; feat-cmux-next pushes can't change what runs with the signing key). After this PR merges into feat-cmux-next, an admin fast-forwards that branch to the merge commit (or any later feat-cmux-next commit):

git fetch origin feat-cmux-next
git push origin <merge-commit-sha>:refs/heads/rc/cnx-ios-builder   # fast-forward only

Set CMUX_WEB_LINT_SKIP_REASON for that push as plans/cmux-next/ios-testflight-builder-DESIGN.md (Operations) says. rc/cnx-ios-builder (200f04ec933) is an ancestor of feat-cmux-next, so this is a fast-forward. It also carries e319a4a763f (set-test-notes.sh, the 5xx retry for the notes update), which the frozen copy does not have yet.

Why the fast-forward is needed: the new notes and previous-build jobs, the signer's download step and scripts/ci/ios_internal_notes.py must all be on that branch, because the dispatch runs ios-internal-nightly.yml@rc/cnx-ios-builder. Until then, uploads keep the old subject-line note. Nothing needs copying to cmux23: the mini runs only ios_internal_builder.py, which this PR leaves unchanged.

Testing

python3 tests/test_ios_internal_notes.py: 35 tests, run on mini-cmux12s (Python 3.14 and the system 3.9). They cover PR body parsing (missing section, template-only section, none variants, multiple lines, markdown noise, category headings, code fences, CRLF, footers, long lines), grouping and dedupe, subject fallback and internal-commit filtering, truncation (including UTF-16 counting and whole lines), the unknown-base path, base recovery (input, then App Store Connect ref, then run history), ASC response parsing, the ES256 JWT (verified with openssl dgst -verify against a generated key), git log over a scratch repo, and the workflow's secret boundaries. tests/test_ios_internal_builder.py still passes. Both are now wired into the cmux-next script tests and ci-guards.
I ran the App Store Connect query (/v1/builds?filter[processingState]=VALID&sort=-uploadedDate&include=betaBuildLocalizations) read-only with asc api. newest_build_ref read 22eeb2baa2 from the live newest build.
actionlint 1.7.7 reports only the existing macos-26 label warning.

Observed / Not run

Not run: the workflow itself. It can only run from rc/cnx-ios-builder after the admin fast-forward. The asc_newest_build HTTP call with the openssl-signed JWT has not run against Apple; the same query ran through asc.

Dry run

The note for 3e6a2b56eff (1.2.0) up to feat-cmux-next ad45b0bc121, from ios_internal_notes.py notes --sha ad45b0bc121… --base 3e6a2b56eff…, is in the first comment.

Changelog

none

Proof

CI and builder only; the dry-run note is in the first comment.

🤖 Generated with Claude Code

^{Need help on this PR? Tag @codesmith-bot with what you need.}

Summary by cubic

Replaces the TestFlight "What to Test" note's git-log subjects with PR Changelog lines, so internal testers see user-facing changes instead of engineering commit messages. A new notes job writes the note with read-only tokens and no signing secret; the upload job only downloads it, and sends just the first line if the artifact is missing.

When the dispatch has no base, a new previous-build job reads the commit recorded in the newest processed build's note in App Store Connect, then falls back to the latest successful builder run.
If both fail, the note says "First continuous build" with the last 20 Changelog lines.
A refused PR lookup no longer drops every Changelog line: GraphQL field errors keep the other fields' data, and a single refused item falls back alone.
Grouped lines are deduped; a PR without a Changelog section contributes its title, plain bullets inherit the last inline category, and collapsed <details> are skipped.

Migration

Fast-forward rc/cnx-ios-builder to this merge commit; the builder runs its workflow and scripts from that frozen branch, so feat-cmux-next pushes can't change what runs with the signing key.