ci: publish waits for npm to serve what a package depends on
Closes #253.
Release 1.7.0 took four runs of publish.yml. npm takes a version in and processes it before it serves it, and each check that a dependency is on npm failed on one published moments before in the same run. npm held most packages for 1 to 4 minutes, and the 31 MB library pack for 14 (the table is in #253). mcp and verify had no check at all, so mcp 0.3.0 went out while the author 0.14.0 it needs was still being processed.
What changes
scripts/published.mjsis new.served(url, label)asks again every 30 s while the answer is a 404, a server error, or a request that failed, timed out or could not be read. It reads a success's JSON body inside that loop, so a body cut or stalled after the headers is asked again too. A request, its body included, is cut at 30 s: one to npm hung for over 5 minutes while this was being written.- The wait belongs to the script, not to each request.
waitUntil()takesH5P_PUBLISH_WAITminutes once, every check in the script shares them, and when they run outserved()returns the last answer. stopUnlessAnswered()stops a check whose request failed with anything but a 404. Only a 404 says a version is not served, so the check says it could not be checked, not to publish what may already be there.- Run from a package's folder, it checks every dependency the package names by
workspace:, at the version the workspace has.dependencies,optionalDependenciesandpeerDependenciescount;devDependenciesdo not.
prepublishOnly: verify, embed, author and mcp run it right after the pnpm guard. mcp now checks author and verify, and verify checks the player and the runtime.- The three pin checks ask through
served(): the player's pack check, embed's player and normalizer check, and the author's pack, player and normalizer check. That covers both npm and jsDelivr. Each script shares one wait and stops throughstopUnlessAnswered(). The author's now says when jsDelivr has noparts/index.json, as the player's does. Their other checks are unchanged: the index matches this checkout's, and./planand./crc32are exported. publish.ymlsetsH5P_PUBLISH_WAITto 30, twice the longest seen. A dry run sets 0: it publishes nothing, so a dependency it would publish never appears, and the check stops at once, as before. By hand it is 0 too, so a dependency npm does not serve is reported at once, whether it was never published or is still being processed.- Docs:
design/build.mdhas the rule and the numbers, anddesign/layout.mdnames the rootscripts/. The trusted-publishing paragraph indesign/build.mdgets back its last two sentences, which the new paragraph had split off.
Tested
packages/author/tests/published.test.ts(9 tests):- it asks again after a 404, a 503 and a failed request, until a 200;
- with no wait, it asks once;
- a 403 is an answer, not a wait;
H5P_PUBLISH_WAITsets the wait, and the wait runs out;- a request with no answer is cut and asked again;
- a success whose body cannot be read is asked again;
- a check goes on after a success or a 404, and stops on a 403, a 503, a DNS failure or a timeout without saying to publish;
- dependencies share one wait: one that is missing leaves the next no wait of its own;
- only
workspace:dependencies that are not dev ones are asked for, at the workspace's versions; - for mcp, those are the author and the verifier at their versions here.
- Type check:
pnpm typecheckpasses in the author, player, embed, mcp and verify packages. The author's covers the new script, which its tests import. - Against npm as it is now: each package's dependency check passes (mcp 2, verify 2, embed 4, author 3), and so do the three pin checks.
- A dependency missing by hand: a test folder naming h5p-author 99.0.0 fails in 0.65 s with "is not on npm (404). Publish it first".
- Failures, with
fetchstubbed: each check exits 1 and says what happened. A DNS failure gives "could not be checked (ENOTFOUND) … run the publish again" in all four scripts. A success whose body is cut gives "could not be checked (terminated)". A jsDelivr 404 gives "has no parts/index.json, unlike this checkout" in both pack checks.
Summary by cubic
Fixes release 1.7.0's four failed runs of publish.yml (#253). npm processes a new version before it serves it — most packages took 1 to 4 minutes, the 31 MB library pack 14 — so each check that a dependency published moments earlier was on npm failed. mcp and verify had no check at all, and mcp 0.3.0 went out while the author version it needed was still being processed.
- New
scripts/published.mjsruns in theprepublishOnlyof verify, embed, author, and mcp, checking everyworkspace:dependency at the workspace's version. served()retries every 30 s while the answer is a 404, a server error, or a failed or timed-out request, and reads a success's body inside the loop so a cut body is retried too. All checks in a script share one wait ofH5P_PUBLISH_WAITminutes; each request is cut at 30 s.publish.ymlsets the wait to 30; by hand and in a dry run it is 0, so a 404 is reported at once.- The three pin checks (player's pack, embed's player and normalizer, author's pack/player/normalizer) wait the same way, for npm and jsDelivr. A failure other than a 404 stops a check with "could not be checked", never "publish it first".