ci: test what a change reaches, not everything on every push
Closes #224
CI now tests what a change reaches instead of everything on every push.
What a run tests
| Event | Packages | Firefox and WebKit | Slow browser files |
|---|---|---|---|
| Pull request | the ones it changes and their dependents | when the player, the runtime or the demo changed | left out |
| Push to main | the same, from where main was | the same | included |
| Weekly (Mon 05:00 UTC), by hand | all | yes | included |
Only design/, docs/, root .md, LICENSE | no run | no |
- A new
scopejob reads the change: pnpm's...[<commit>]filter for packages and apps, plus the author package andapps/skillsforskills/. Any other file outside the packages that is not docs (the lockfile, a workflow, the root'spackage.json) tests everything. pnpm typecheckstill covers every package: each package's check writes the types the next one reads, and it takes seconds.- The slow files are
stalling-host,worker-updateandlarge-media, listed in the player'svite.config.tsand left out underH5P_TEST_SKIP_SLOW. They take 115 s of test time in Chromium and 120 s in Firefox. Main's run after the merge has them, sodesign/testing.mdnow says to wait for it before publishing the player. - The VS Code job runs when the extension, the player, the runtime or the libraries changed.
to-scormis not among the slow files: it runs on every pull request that testsapps/skills, being the one check of the h5p-to-scorm skill's text against the embed README.
Testing
-
The scope step, simulated on a fresh shallow clone with a fake commit per kind of change, run as a pull request's run:
Change Tested Engines VS Code packages/authorauthor, demo, skills no no skills/h5p-author/SKILL.mdauthor, skills no no design/author.mdpluspackages/authorauthor, demo, skills no no packages/playerplayer, verify, embed, author, extension, demo, skills yes yes packages/runtimethe same, and the runtime yes yes apps/demopagedemo the app's tests only no packages/embedembed, demo, skills no no packages/verifyverify, author, demo, skills no no packages/cmi5cmi5, demo no no packages/vscodethe extension no yes pnpm-lock.yaml, another workfloweverything yes yes It found one bug, fixed before this commit: pnpm counts a file outside the packages as the root project's, and the root's
testscript tests everything. The step now leaves the root out. -
vitest list --project browser: 36 files, and 33 withH5P_TEST_SKIP_SLOW=1; the three named are the ones missing. -
The player's
typecheckpasses. The workflow parses, with each job'sneedsandifas intended.
Not verified
- This pull request's own run tests everything, because it changes a workflow, and leaves out the three slow files, as any pull request's run does. Selecting fewer packages shows up first on the next pull request after the merge.
- A push to main fetches the commit main was at, to compare with. GitHub serves a fetch by commit; a forced push falls back to testing everything.
Release
None: no package's published files change.
Summary by cubic
Closes #224: CI now tests what a change reaches instead of everything on every push. A new scope job picks the packages a change touches and their dependents via pnpm's ...[<commit>] filter. Pull requests test only those, skip the player's three slow browser files, and run Firefox and WebKit only when the player, runtime, or demo changed. Pushes to main run the same selection including the slow files; weekly and manual runs test everything. Docs-only changes (design/, docs/, root markdown, LICENSE) start no run.
pnpm typecheckstill covers every package, in seconds.- The VS Code job runs when the extension or its dependencies changed.
- Non-doc files outside the packages (lockfile, workflows, root
package.json) test everything.
Testing
- Scope selection verified on a fresh clone with a fake commit per change kind; one bug found and fixed: pnpm counts such files as the root project, whose test script tests everything, so the step now excludes the root.
- This PR's run tests everything (it changes a workflow) and skips the slow files as any PR does.
Release
- No published files change.