[CI/CD;DevEx]: Scope sample and source-change workflows to skip unnecessary jobs

Draft
PI
2mo ago
+2841-884

Motivation

Previously, any file change to a sample group would trigger rebuilding all samples in that group (up to 96 matrix jobs). Additionally, source-unrelated PRs (e.g., new sample config, sample regeneration, CI workflow changes) would still run expensive full Maven unit test suites on Linux, Windows, and CircleCI. This change introduces per-sample and per-change-type scoping to skip unnecessary jobs.


Changes

1. New reusable composite action — .github/actions/compute-matrix/action.yaml

A new composite action that computes a filtered matrix of only the changed samples for a given push or pull request event. It uses git diff against the base SHA to determine which sample directories were actually modified.

Features:

  • Detects unreachable commits after force-push/rebase and safely runs all samples when base SHA is unavailable
  • For pull_request events: uses git merge-base to isolate exactly the PR author's changes (avoids over-triggering when base branch advances)
  • For push events: diffs against github.event.before with fallback to HEAD~1

Inputs:

  • all_samples — newline-separated list of all sample directories the workflow covers

Outputs:

  • matrix — {"sample":["path/a","path/b"]} JSON, used directly with fromJson() for single-dimension matrices
  • samples — ["path/a","path/b"] JSON array, used for multi-dimensional matrices (e.g., os + sample)
  • Returns [] when no samples changed; calling jobs use an if: != '[]' guard to skip entirely

2. All 88 sample workflows — dynamic per-sample matrix

Every samples-*.yaml workflow with a sample: matrix dimension was updated:

  • Added a setup job that calls ./.github/actions/compute-matrix and outputs the filtered matrix
  • The build job now uses matrix: ${{ fromJson(needs.setup.outputs.matrix) }} instead of a static list
  • An if: needs.setup.outputs.matrix != '[]' guard skips the entire build job when nothing relevant changed
  • The static matrix: sample: list is replaced by the all_samples: input — same number of lines to maintain, no duplication
  • fetch-depth: 0 added to the setup job checkout to enable git diff against the base commit

Special cases handled:

  • Multi-OS workflows (samples-swift5.yaml, samples-dart-build-test.yaml) — os: stays static, only sample: dimension is dynamic
  • Multi-version workflows (PHP, Node, Python) — php-version/node-version/python-version stays static
  • Maven-profile workflows (samples-dart.yaml, samples-julia.yaml) — unchanged; they're all-or-nothing by design

3. openapi-generator.yaml — skip unit tests and doc check for non-source PRs

A new setup job detects whether source files changed (checks: modules/**, pom.xml, .mvn/**):

  • test (Unit tests) and documentation (Docs up-to-date) jobs are skipped when only samples or .github/ changed
  • build and samples (Samples up-to-date) continue to run unconditionally — samples verifies sample freshness
  • The workflow trigger has no paths: filter — it fires on every push/PR so required status checks are always satisfied (skipped jobs count as passing)

4. linux.yaml and windows.yaml — skip tests for non-source changes

Both platform-specific test workflows now have the same source-change detection:

  • New setup job detects if modules/, pom.xml, .mvn/ changed
  • build job is skipped (if: needs.setup.outputs.source-changed == 'true') when only .github/ or sample files changed
  • Prevents running expensive platform-specific tests for CI config-only PRs

5. CI/circle_parallel.sh — smart per-node change detection

CircleCI nodes now check if relevant samples changed before running integration tests:

  • Safe defaults: If origin/master unavailable or git diff fails → run tests (assumes change is relevant)
  • Decision logic:
    1. If generator/config changed (modules/, bin/configs/, pom.xml, .mvn/) → run all node tests
    2. If node's specific sample directories changed → run that node's tests
    3. Otherwise → skip
  • Clear logging explains the decision (e.g., "Skipping perl tests — no relevant changes")

Prevents wasted CI resources on pure .github/ changes or unrelated sample changes (e.g., don't run perl tests if only ruby sample changed).

6. Bug fixes applied

Seven workflows had push: branches: instead of push: paths::

  • samples-kotlin-client.yaml
  • samples-kotlin-wiremock.yaml
  • samples-cpp-oatpp-client.yaml
  • samples-cpp-oatpp-server.yaml
  • samples-cpp-qt-client.yaml
  • samples-zapier.yaml
  • samples-dart-build-test.yaml

Additional bugs fixed:

  • samples-ruby.yaml — Removed literal single-quote wrapping from all_samples: entries (YAML block scalars treat quotes as literal chars, breaking prefix matching)
  • samples-typescript-nestjs-server.yaml — Restored node-version: [20, 22] as a static matrix dimension (was accidentally moved into all_samples:)
  • samples-java-petsore-client-jdk11.yaml, samples-java-petsore-client-jdk17.yaml — Restored services: petstore-api container block and removed stray 80:8080 entry from all_samples:
  • samples-clojure.yaml — Removed stray 80:8080 entry from all_samples:
  • samples-java-helidon-v4.yaml — Replaced undefined ${{ matrix.version }} with hardcoded '21'
  • samples-go-client.yaml — Removed single quotes from two all_samples: path entries
  • samples-cpp-httplib-server.yaml — Removed workflow file from paths: trigger (was triggering on own changes)
  • samples-dotnet-standard.yaml — Removed unityWebRequest from paths: (commented out in all_samples:)
  • samples-java-client-echo-api-jdk11.yaml, samples-java-client-echo-api-jdk17.yaml — Replaced broad glob paths: with per-sample explicit entries

7. Documentation comments added to all workflows

  • All paths: sections have a comment explaining /** glob requirement
  • All all_samples: inputs have a comment explaining format (no trailing /**, one per line, prefix-matched)
  • All setup job comments explain the source-change detection logic

Developer experience — adding a new sample

Before and after, exactly 2 additions per workflow file are needed:

BeforeAfter
Line 1paths: entrypaths: entry (unchanged)
Line 2matrix: sample: entryall_samples: entry (replaces matrix entry)

What was intentionally left unchanged

  • openapi-generator.yaml — "Samples up-to-date" job always runs (verifies sample freshness and regeneration status)
  • samples-typescript-typecheck.yaml — runs a whole-TS-samples script, no per-sample matrix
  • samples-dart.yaml, samples-julia.yaml — Maven-profile based, all-or-nothing by design
  • misc-test.yaml — already fast (~30s), not worth adding complexity
  • Maven-profile and non-matrix sample workflows — left unchanged

Performance impact

Before:

  • Sample-only PR: 88 workflows × ~2-50 jobs each = potentially hundreds of jobs triggered
  • .github/ change: Full unit tests + docs checks + platform tests run unnecessarily
  • Any push: All CircleCI integration tests run regardless of what changed

After:

  • Sample-only PR: Only changed samples' workflows are triggered (e.g., 1 sample change = 1 job instead of 58)
  • .github/ change: Sample workflows still trigger (gated by paths:), but unit tests/doc checks/platform tests skip
  • CircleCI: Only relevant integration tests run per change (e.g., perl tests skip if only ruby sample changed)

Estimated savings:

  • Multi-sample workflows: ~70% fewer jobs (most PRs only touch 1-3 samples)
  • Source-unrelated PRs: Full skip of unit tests, doc checks, platform tests (easily 30+ minutes of CI time)
  • CircleCI: ~60% fewer resource-hours on average

Commits in this PR

  1. [Phase 1 - prior] Create compute-matrix action and refactor all 88 sample workflows
  2. [Phase 1 - prior] Fix branches: → paths: bugs in 7 workflows
  3. Add openapi-generator.yaml source-change detection — skip unit tests on sample-only PRs
  4. Fix bugs: petstore services, helidon version, etc. — restore containers, remove stray entries
  5. Fix ruby quotes and nestjs node-version — restore proper matrix dimensions
  6. Use merge-base diff for PR events — fixes stale base SHA over-triggering
  7. Fix force-push detection — run all samples when base SHA is unreachable
  8. Skip linux/windows tests for non-source changes — add source-change detection
  9. Add smart change detection to CircleCI — skip irrelevant integration tests
  10. Fix echo-api broad glob paths — per-sample explicit paths: entries

Testing

All changes are workflow/CI infrastructure only — no application code changes. Validation approach:

  1. Sample workflow matrix: Manually verify one multi-sample workflow executes correctly with filtered matrix
  2. Source-change detection: Verify openapi-generator.yaml skips tests on .github/ PR, runs on modules/ PR
  3. CircleCI change detection: Monitor logs to confirm nodes skip when unrelated samples change
  4. Backward compatibility: Existing workflow triggers (paths:) remain unchanged

PR checklist

  • Read the contribution guidelines.
  • Run the following to build the project and update samples:
    ./mvnw clean package || exit
    ./bin/generate-samples.sh ./bin/configs/*.yaml || exit
    ./bin/utils/export_docs_generators.sh || exit
    
    (For Windows users, please run the script in WSL)
    Commit all changed files.
    This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
    These must match the expectations made by your contribution.
    You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
    IMPORTANT: Do NOT purge/delete any folders/files (e.g. tests) when regenerating the samples as manually written tests may be removed.
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Summary by cubic

Scopes CI to only run what changed. Adds a reusable matrix action, gates sample/OS builds, tightens workflow triggers, skips irrelevant CircleCI nodes, and always runs full CircleCI on master.

  • New Features

    • Added ./.github/actions/compute-matrix to compute changed samples; workflows add a setup job (actions/checkout@v5, fetch-depth: 0) and gate builds with needs: setup + if: checks.
    • openapi-generator.yaml: detect source vs sample/config changes and skip unit tests/docs for sample-only or bin/configs-only changes; bin/** changes do not trigger tests.
    • linux.yaml and windows.yaml: detect modules/**, pom.xml, .mvn/** changes to skip OS tests when non-source files changed.
    • CircleCI: per-node change detection in CI/circle_parallel.sh with clear logs; feature branches skip irrelevant tests, master always runs all.
  • Bug Fixes

    • compute-matrix: use git merge-base for PR diffs to avoid over-triggering; run all samples when the base SHA is unreachable after a force-push; removed ${{ }} from composite action output descriptions.
    • Triggers/matrices: fixed push.branches → push.paths; tightened Java Echo API paths: to per-sample entries; restored static matrix dimensions where needed (Python uses a static python-version matrix; restored node-version for TypeScript NestJS and three TS workflows: samples-typescript-axios-echo-api.yaml, samples-typescript-encode-decode.yaml, samples-typescript-node-encode-decode.yaml).
    • Path/list issues: unquoted Ruby/C/OCaml/Protobuf all_samples entries so prefix matching works; removed stray 80:8080 entries and restored services: petstore-api; removed a workflow file from its own paths: trigger; cleaned up Helidon version placeholders.
    • samples-java-helidon-v3.yaml: hardcoded java-version: '17' to replace a removed matrix.version reference (mirrors the v4 fix).