Add prerelease-test skill for release QA campaigns
All Submissions:
- Have you followed the guidelines stated in CONTRIBUTING.md file?
- Have you checked to ensure there aren't any other open Pull Requests for the desired changed?
Type of change
- New feature (non-breaking change which adds functionality)
Tooling only — no framework code changes, nothing shipped in any package.
New Feature Submission:
- Does your submission pass the tests?
- Have you linted your code locally prior to submission?
Description
Captures the pre-release QA campaign run against the 0.9.9 train as a repeatable skill, so the same analysis can be kicked off before any release instead of being reconstructed from scratch each time. That campaign found 29 verified issues, 11 of which were fixed before 0.9.9 shipped (#6971, #6984, #6985, #6986, #6987, #6989, #6990, #6991, #6992, #6994) with the rest filed as #6972–#6983.
.claude/skills/prerelease-test/SKILL.md lays out the phases: discover what shipped from the pre-release branch changelogs and confirm every package published to PyPI, de-risk with a smoke test, fan out feature clusters that build real apps and drive them in a browser, upgrade-test reflex-examples apps from the previous stable, regression-test reflex-enterprise demos, audit wheel/sdist packaging, then triage into fix-before-release vs file-as-issue.
Three habits are what made the campaign's output actionable, so they are written in explicitly rather than left to chance:
- Install only published packages. The checkout has unreleased code and a different dependency graph, so testing it says nothing about what users get — and a package that failed to publish is itself a blocker you would otherwise miss.
- Baseline against the previous stable. It is the only way to tell a regression from a pre-existing bug, which is the distinction the release decision turns on.
- Verify adversarially before reporting. An independent agent reproduces each claimed issue from the written repro alone and tries to refute it. On the 0.9.9 run this refuted or reclassified a meaningful share of claims, and it doubles as a test that the repro is good enough to hand to a fix agent.
references/ holds the brief given to each test agent (hard rules, environment traps like checkout shadowing of import reflex, and the known-benign console noise list), cluster decomposition guidance, the Workflow explore/verify pipeline with schemas and a port map, and report templates.
scripts/ bundles three helpers the test agents kept rewriting independently:
| Script | Purpose | Validated against 0.9.9 |
|---|---|---|
check_release_versions.py | changelog versions → PyPI publication check | 16 packages, all published |
audit_pyi.py | stubs present in wheel and sdist, byte-identical, no foreign stubs, counts vs pyi_hashes.json | 121 stubs, clean |
drive_app.py | Chromium driver: console/network capture, small JSON action language, benign-noise filtering, exit 0/1 | clean and failure paths both exercised |
Note on the pyproject change
The one non-skill change is two lines in pyproject.toml. Ruff selects ALL with no .claude exclude, so these scripts are linted by CI. Everything ruff flagged is fixed in the scripts themselves (Args/Returns docstring sections, annotations, EM102, and an action-loop restructure that avoids both PERF401 and PERF203 while still preserving the record of actions that succeeded before a failure). The exception is T201 — printing a report is these scripts' entire purpose — which is whitelisted exactly the way scripts/check_min_deps.py already is.
Test Plan
uvx ruff@0.16.3 check .claude/skills/prerelease-test/scripts/andruff format --check— both clean, using the version locked inuv.lock.- All three scripts run end-to-end against the released 0.9.9 train, with the results in the table above;
drive_app.pywas additionally checked for its failure path (a failing action reports the error, keeps the already-successful actions, skips the rest, exits 1) and its noise filter (--all-consoleshows the lines it normally suppresses). - No framework code is touched, so the unit suite is unaffected by this diff.
🤖 Generated with Claude Code
https://claude.ai/code/session_01EMjBXPozsNeQNSBZecNH8x
Generated by Claude Code