feat(feedback): send bugs without a repro to GitHub, refuse outdated reports
Merged
OK
12d ago+290+290+012345678901234567890123456789-42-42-01234567890123456789
What and why
e2e feedback reports are mostly agent-written. Most of the low-quality ones were sent from old versions, were flakes that need a repro, or repeated bugs we already knew about. 61 of 90 recent reports came from an older version than the latest release. Reports that need a repro die in Linear, because PostHog feedback is anonymous and we can't ask the reporter.
- A
--type bugreport needs--repro, steps that reproduce it every time. Without it, the command sends nothing and prints a prefilled GitHub bug-form link (whatandversionsfilled in, redacted, under 8000 chars). The user adds a minimal repro and submits it themselves. The agent never files it. - Before sending, it asks the registry for
e2e@latest. If the installed version is older, it exits 2 and sends nothing.--allow-outdatedoverrides. A failed or slow registry check (3s timeout) is skipped. UnderE2E_TELEMETRY_DISABLED/DO_NOT_TRACKno request is made. - New
--error-code <CODE>field (error_codeon the event) so reports group by code.--task/--expected/--actual/--approach/--reproare capped at 1500 characters. - Skill Feedback section:
- never report app behaviour, one-off flakes, or outdated versions;
- search GitHub issues first and comment on a match;
- a bug that needs the app goes to GitHub.
- Docs: CLI reference, telemetry, security outbound connections (
registry.npmjs.org).
Verified
Ran it locally: yes
Built CLI against the live registry (latest 0.20.0):
$ e2e feedback --type bug -m "tap lands on the wrong tab" --actual "TARGET_NOT_FOUND on second run" --error-code TARGET_NOT_FOUND --dry-run
Not sent: a bug without --repro goes to GitHub, where the e2e team can ask for a reproduction. Open this prefilled issue, add a minimal reproduction, and submit it:
https://github.com/tester-army/e2e/issues/new?template=bug_report.yml&labels=bug&title=tap+lands+on+the+wrong+tab&what=tap+lands+on+the+wrong+tab%0A%0AActual%3A+TARGET_NOT_FOUND+on+second+run%0AError+code%3A+TARGET_NOT_FOUND&versions=e2e+0.20.0%2C+node+24.19.0%2C+darwin+27.0.0
exit=0
# package version set to 0.19.0
$ e2e feedback -m x --dry-run
feedback not sent: e2e 0.19.0 is older than the latest release 0.20.0, which may already fix this. Upgrade e2e, check the problem still happens, and send it again; --allow-outdated sends it anyway.
exit=2
$ e2e feedback -m x --error-code nope
error: option '--error-code <code>' argument 'nope' is invalid. must be an e2e error code such as LOCATOR_AMBIGUOUS, at most 64 characters
exit=2
packages/e2eunit tests: 2609 passed. New cases cover GitHub routing and redaction, the URL length cap, the outdated refusal and--allow-outdated, a registry failure,--error-codevalidation, andolderThan.- lint, typecheck and docs:check pass.
- check:dead-code fails on 20 unused class members, and plugins:check fails because
plugins/e2e/skills/migrate-to-e2eis missing. Neither comes from this PR.
Checklist
- Changeset added (
pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only. - Breaking change: title carries
!, the changeset body names what breaks and what replaces it, and the deprecation warning landed a window earlier (stability policy in CONTRIBUTING.md). - Docs updated in the same PR: the
docs/**/*.mdxpage for the behavior, andskills/e2e/if the skill describes it. - Contract change: emitted
.d.tsreviewed,tests/types/sdk-types.tsupdated; wire change edits the schema and both fixtures. - Engine contract change (
e2e/engine): changesets fore2e,@e2e-dev/web, and@e2e-dev/mobile.