webviews: a ratchet refuses new fixed-time waits in tests

Merged
TE
15d ago
+587-2

Summary

New webview tests can no longer wait a fixed number of milliseconds. check:fixed-waits, now part of bun run check (CI's web lint (fast)), names each fixed wait in a test file or test/support helper:

  • page.waitForTimeout(…)
  • Bun.sleep(…)
  • a setTimeout that resolves a new Promise after a delay other than 0 (a zero delay only yields to the next macrotask and stays allowed)
  • each use of a sleep helper built on one of those (a call, or the helper passed or returned as a value, applied outward through helpers built on helpers), counted at each use rather than at the definition. A helper whose delay defaults to 0 waits only when a use passes a delay; an exported helper also counts at its definition.

The scan covers every source under test/ (harnesses, support and latency helpers; fixtures skipped) and the tests under src/.

A fixed wait passes on an idle machine and fails on a loaded runner that wakes it late, or it hides the bug it waits out. #23323 is an example: a 200 ms wait woke after the card's 250 ms delay. Instead, a test waits for a condition (an event, a DOM state, a painted frame), or uses fake timers when the timing itself is the behavior. A wait that must stay says why in a // fixed-wait: <reason> comment anywhere in its statement, or on a comment line above it. A marker on a helper covers its uses.

The 989 existing waits in 132 files (each wait site counts, and so does each use of a helper that waits, inside other helpers too, so a new wait anywhere raises the count) have per-file counts in scripts/fixed-waits-baseline.tsv. It works like the type-scale ratchet:

  • a file fails with more unmarked waits than its row;
  • a file fails with fewer until the row is lowered with bun run check:fixed-waits --update-baseline;
  • --update-baseline refuses to raise a row.

The sweep PRs that follow replace the waits in batches of about 20 files and lower the baseline. This PR changes one wait: pageSwitch's 16 ms fallback for a page with no requestAnimationFrame gets a marker with its reason.

How the detector was checked: the AST detector finds every non-zero hit of a broad regex (waitForTimeout, setTimeout(resolve, N), sleep(N), wait(N)). Every regex hit it skips is a setTimeout(resolve, 0) yield.

Test first: the first commit adds test/check-fixed-waits.test.ts, which fails with the module missing. The second commit adds the script; the test then shows 5 pass. After review, the third commit adds 6 cases that fail on the first detector (helper uses, block-body timers, exported helpers, statement-wide markers), and the fourth commit makes all 11 pass.

Observed / Not run

  • Observed: bun test ./test/check-fixed-waits.test.ts red, then green; bun run check passes in webviews/; scripts/web-lint.sh and prepush-gate.py pass.
  • Not run: nothing else. No test behavior changes.

Changelog

none

Agent session: huge-blue

🤖 Generated with Claude Code