fix(cli): report a skipped test in one shard only
Merged
JA
9d ago+90+90+01234567890123456789-3-3-0123456789
What and why
--shardcuts its slices from the runnable tests only, andapplyShard(packages/e2e/src/collect/select.ts:776) left every skipped pair ({ skip },test.skip(...)at declaration, a missing capability) untouched. So each shard kept every skipped test, ande2e list --shard, thelistsummary,junit.xml,summary.md, andreport.jsonof every shard listed it again. A matrix of 3 shards over 3 tests and 1 skipped one reports 6 results instead of 4, and anything that adds the shards' JUnit files up counts the skip 3 times.- Now a skipped test is reported by the shard that runs the test before it in report order, the first shard when none comes before. The cut itself is unchanged: slices still differ by at most one runnable test, and a skipped test still never makes a shard non-empty.
- In the other shards the pair becomes
filteredand keeps its ownskip, the way aplatformsmiss already does, soNO_TESTSstill counts it as skipped withtest.skipand the "split across N shards" count does not grow. - Playwright also reports each skipped test in one shard, but counts skipped tests as items of the cut; here the cut stays runnable tests only, so the documented slice sizes hold.
// packages/e2e/src/collect/select.ts, applyShard (this branch)
let previousKept = shard.index === 1;
return pairs.map((pair) => {
if (pair.test.kind !== 'test') return pair;
if (pair.disposition === 'skip') return previousKept ? pair : { ...pair, disposition: 'filtered' as const };
if (pair.disposition !== 'run') return pair;
previousKept = kept.has(shardItem(pair));
...
Verified
Ran it locally: yes
- Scratch project, built CLI, one file with
one,two,three, andlater({ skip: 'not ready yet' }),e2e run --shard i/3 --reporter list,junit:
| shard | main | this branch |
|---|---|---|
| 1/3 | 1 passed | 1 skipped (2), junit tests="2" skipped="1" | 1 passed (1), junit tests="1" skipped="0" |
| 2/3 | 1 passed | 1 skipped (2), junit tests="2" skipped="1" | 1 passed (1), junit tests="1" skipped="0" |
| 3/3 | 1 passed | 1 skipped (2), junit tests="2" skipped="1" | 1 passed | 1 skipped (2), junit tests="2" skipped="1" |
- The shards'
report.jsonfiles validate againstreport-v1.schema.json, and theirsummary.selectednow add up to the unsharded run's. - New unit test in
packages/e2e/tests/unit/select.test.ts(a skipped test first, in the middle, and last, over 2 shards) fails on main and passes here.
Checklist
- Changeset added (
pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only. - Breaking change: title carries
!, the changeset body names what breaks and what replaces it, and the deprecation warning landed a window earlier (stability policy in CONTRIBUTING.md). - Docs updated in the same PR: the
docs/**/*.mdxpage for the behavior, andskills/e2e/if the skill describes it. - Contract change: emitted
.d.tsreviewed,tests/types/sdk-types.tsupdated; wire change edits the schema and both fixtures. - Engine contract change (
e2e/engine): changesets fore2e,@e2e-dev/web, and@e2e-dev/mobile.