fix(author): the build page route hands over the source file, only the tool writes the .h5p

Merged
AL
33m ago
+41-6

Closes #194.

Three changes to the h5p-author skill's text (skills/h5p-author/SKILL.md), each from what ChatGPT did with it, and their record in design/author.md. No code changes, and the build is unchanged.

What changes

  • The source file in the build page route. Where nothing is built in the chat and the chat can give the user a file, it hands over the source as <name>.source.json beside the link. The page opens that file, and a link copied wrong into the reply builds nothing. Step 5's opening says the same.
  • Only the build tool writes the .h5p. The skill's opening says never to put one together from h5p.json and content.json, since one made by hand carries no libraries and nothing checks it.
  • A hung install has failed. Step 3's list of npm failures adds "an install that hangs until the command times out".
  • design/author.md:
    • The hand-over rule records the source file in the build page route.
    • A new rule, Only the build tool writes the .h5p.
    • The npx rule records the hang, and that an uploaded plugin's files never reach ChatGPT's code.

SKILL.md grows from 14,670 to 15,154 characters, under the 20,000 that Open WebUI reads at once.

Tested

All lab runs used Haiku, the weakest model first. There were 22 runs, each making the same three-question Question Set, with an author and licence given.

PositionOld text (0.9.6)New text
Sandbox told it has no network, files given as downloads3 of 3: exact link and the source file3 of 3 the same, with npm cut off
The same, the user asking for "the .h5p file itself … not a link and not JSON"1 exact link and file; 1 JSON block, page address and file; 1 neither link nor file, the npx command for the user2 exact link and file; 1 file and the npx command for the user
Terminal whose npm registry takes connections and never answers2 of 2 moved on to the two files and built2 of 2 the same
A .h5p made by hand0 of 60 of 6
  • No gain on Haiku. It already hands over the file, moves on after a hang, and makes no package by hand. The case for the change rests on ChatGPT's runs:
    • 2 of its 3 links broke, while its source file built every time.
    • It zipped a package by hand. With the sentence in a test plugin, it said it had written no .h5p (one run each way).
    • Its npx timed out.
  • Where the new sentence leads. In a sandbox only told it had no network, 3 of 6 new-text runs ran npx anyway, against 0 of 6 old-text runs. This computer's npm answered, so they built. With npm cut off, every run handed over as the table shows.
  • The commands: the two link one-liners are unchanged.
  • The package: pnpm --filter @missing-elements/h5p-author run test:unit (183 tests) and typecheck pass, and sync:skill changes nothing.

Not covered

  • A user who says "not a link" sends a weak model to the npx command for themselves, under the old text and the new (one run in three each). The skill says never to end with commands for the user, and that does not reach this case.
  • ChatGPT and Gemini themselves are not measured with the new text. They get it with the next author release, the site's pin bump, and its zips (missing-elements.github.io#41).

Release

This is a patch, h5p-author 0.9.7, skill text only. After it come the org site's pin bump, which also refreshes both zips, npx skills update -g, and a new Claude app zip.

Summary by cubic

Tightens the h5p-author skill's hand-over rules after ChatGPT failed them in tests; this is skill text and its design record only, no code changes (closes #194).

  • In the build page route, the chat now hands over the source as a <name>.source.json file beside the link; ChatGPT wrote 2 of 3 links wrong by hand while its source file built every time.
  • The skill now says only the build tool writes the .h5p; ChatGPT had zipped one by hand with no libraries and its content wrapped in a params key.
  • A hung npm install now counts as a failed install, since ChatGPT's npx timed out.