ci: run pull_request workflows on every base branch

Merged
JM
1mo ago
+244-62

Description

Most pr-*.yml workflows only ran on PRs into main (or release/**). A PR into any other branch got no test CI. Native GitHub stacks (gh stack) already matched the filter for every PR in the stack, but manually stacked PRs, PRs linked into a stack after their last push, and ad-hoc PRs into feature branches got only quality-checks, linear, and audit.

This PR:

  • Drops the branches: filter from the pull_request trigger in 19 pr-*.yml workflows. merge_group, paths:, types:, and tag or release push triggers are unchanged.
  • Fixes the three places that read the base branch:
    • pr-helm-chart-version.yml keeps comparing against main (where the chart publishes from) and skips release/** PRs via branches-ignore.
    • pr-helm-chart-testing.yml runs ct list-changed against the PR base, so a stacked PR only lists its own chart changes.
    • pr-playwright-tests.yml falls back to the main visual-regression baseline when the base has none. Today a stacked PR compares against nothing and reports every screenshot as "Added".
  • Adds a merge-queue guard for the two checks that are enforced on pull_request only (required, playwright-required). The pull_request run records the base it tested against as a commit status (ci/tested-base/integration, ci/tested-base/playwright). The same-named jobs in merge-group.yml accept the queue entry only when that base landed in the target branch: directly, or as the tip of a parent PR that merged into the target or is still open against it (a stack in flight). Either way the merged tree differs from the tested tree only by what the target gained since the test, which the queue already accepts. It rejects a base that never landed, such as a release branch after a retarget to main, or an abandoned parent branch. Logic lives in .github/actions/check-tested-base.

Concessions

  • Rollout: every open non-fork, non-Dependabot PR has no record yet. Its first merge attempt is dequeued with a message to push a commit or close and reopen. "Re-run jobs" does not help because it reuses the old base.
  • Retargets stay silent: retargeting a PR fires only edited, which we deliberately do not run on (it would cancel in-flight runs through the cancel-in-progress concurrency groups and can overwrite a real failure with a skipped-green check). The queue guard covers the retarget case at the point that matters.
  • Fork and Dependabot PRs are exempt from the guard. Their runs get a read-only token and cannot record. Same posture as today.
  • Checks on PRs into feature branches are advisory: no ruleset targets feature branches. Prefer gh stack, which enforces main's ruleset on every PR in the stack and rebases children after a merge.
  • Cost: each PR in a stack runs its own CI on every push to its head branch. A restack of N branches is N runs. The changes jobs still gate the expensive lanes.
  • Release-branch PRs now also run the lanes that were main-only (craft, terraform, connector tests, helm chart testing). None is required by the release ruleset.
  • Connector tests (real external APIs) now run on stacked PRs that touch backend/**.
  • Helm chart version on a stacked child compares against main only, so a child that inherits the parent's bump passes until the parent publishes it.
  • Visual regression on a stacked child shows parent + child changes until the parent lands.

How Has This Been Tested?

  • pre-commit on every touched file: check-yaml, actionlint, zizmor, shellcheck.
  • The queue script was dry-run locally against the live repo: PR-number parse from the queue ref and from the squash-message fallback, slash-containing release targets, the Dependabot exemption, and the missing-record error.
  • With a gh shim supplying canned records, the guard ran against live PRs: accept via the target, accept via an open parent PR (one hop, and a four-PR chain), accept via a squash-merged parent PR, reject a release tip after a retarget to main, reject an abandoned parent branch, reject a tip that the open parent no longer contains, and reject an unknown SHA.
  • This PR itself exercises the record step: both ci/tested-base/* statuses landed on the first commit with base main. Next: a throwaway child PR based on this branch to confirm the lanes run on a stacked PR and the baseline fallback, then enqueue it after retargeting to main to see the guard in a real merge_group run.

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.
  • [Optional] Override Linear Check

🤖 Generated with Claude Code


Summary by cubic

Most pr-*.yml workflows only ran on PRs into main (or release/**), so stacked PRs and PRs into feature branches got no test CI. This drops the branches: filter from the pull_request trigger in 19 workflows so every PR runs the same test lanes.

  • The required and playwright-required merge-queue jobs now pass only when the PR-run verdict was computed against a base that landed in the target or is the tip of a parent PR merged into or open against it; Fork and Dependabot PRs are exempt because they cannot record it.
  • Helm chart version still compares against main and skips release/**; helm chart testing diffs against the PR base; the Playwright visual-regression baseline falls back to main when the base has no baseline.

Rollout

  • Existing open PRs have no tested-base record, so their first queue attempt fails; push a commit or close and reopen. Re-running jobs reuses the old base and won't help.
  • Retargeting a PR doesn't re-run tests — the queue guard rejects the stale verdict at merge time.
  • Release-branch PRs now run the previously main-only lanes (craft, terraform, connector, helm chart testing), and connector tests run on stacked PRs touching backend/**.
  • Visual regression on a stacked child shows parent + child changes until the parent lands, and a stacked child's helm chart version passes until the parent publishes it.