fix(decision): leave nothing out when the loaded config changes
Merged
OK
4d ago+100+100+012345678901234567890123456789-16-16-01234567890123456789
decisionFilter (#1206, new in #1210) runs every test when a run-all file changes, but it recognized the config only by the names e2e.config.*. A run with --config e2e.ci.config.ts sent a change to that file to the decision model like any other file. In the pre-release bug bash, 3 of 4 decision models left out all 78 testbed tests for a config-only change.
- e2e hands filters
context.configFile, the config the run loaded, relative tochanges.rootaschanges.filesspells it (packages/e2e/src/run/test-filters.ts). The directory is resolved on disk and the file keeps its own name, so a symlinked config matches the path git lists. It isundefinedfor a config object or a config outside the repository. decisionFilterleaves nothing out when that file changes, whatever its name, compared in NFC (packages/decision/src/filter.ts). With an older e2e that does not set the field, it falls back to the name list.configFileis optional on the publicTestFilterContext, so a hand-built context in a user's filter tests still compiles.
Not covered: files the config imports still go to the model.
Verified
Ran it locally: yes
- A scratch git repo with three tests and a custom
e2e.ci.config.tsusingdecisionFilter({ model: gateway.decisionModel('typesafe-ai/jev') }), with only that config changed. Built CLI,e2e list --config e2e.ci.config.ts --changed-since HEAD~1:
| main | this branch |
|---|---|
| model call, 2 of 3 tests left out | no model call, all 3 kept |
# main
judging 1 file changed since 42794d4
decision: asking typesafe-ai/jev about 3 tests in 3 files
decision kept 1 of 3 tests, left out 2 (1 call, 867 tokens, <$0.0001)
tests/cart.e2e.ts › cart page shows its heading [web] (left out by decision: 15% likely affected)
tests/login.e2e.ts › login page shows its heading [web]
tests/search.e2e.ts › search page shows its heading [web] (left out by decision: 15% likely affected)
# this branch
judging 1 file changed since 42794d4
decision kept all 3 tests (e2e.ci.config.ts changed, so the decision filter left nothing out)
tests/cart.e2e.ts › cart page shows its heading [web]
tests/login.e2e.ts › login page shows its heading [web]
tests/search.e2e.ts › search page shows its heading [web]
- The integration test covers
configFilefor a discovered config, a custom--configname, a project nested in a monorepo, a symlinked config, and a config object (packages/e2e/tests/integration/test-filters.test.ts).