ci: record what a hung test run is blocked on
Merged
SA
1mo ago+22+22+01234567890123456789-1-1-0123456789
About 5% of Test jobs hang. #8270 caps the cost at 10 minutes and #8286 streams the logs, which caught two hangs:
- One froze in
useDashboardSidebarShortcuts.test.ts, whose only test callsBun.spawnSync. bun printedkilled 1 dangling processat the 5 s test timeout and then never moved. This is the shape of oven-sh/bun#43697 (1.4.2, Linux, KVM guest:spawnSyncspins and the test timeout cannot end it). - One froze a bun task and a node task within 120 ms of each other, in a file that spawns nothing.
The logs do not show what the stuck process waits on. This change records it:
- At 345 s a background shell writes every thread's state, CPU, wait channel, plus each bun, node and turbo process's current syscall and open files, to a file.
- At 360 s
timeoutkills the whole process group. The step fails with exit code 124 instead of running to the job limit. - The next step prints the file when the Test step failed.
A normal Test step takes about 70 s (p90 about 90 s), so 360 s does not touch a healthy run.
Verified on this PR with the timers set to 15 s and 40 s: the step ended at 40 s with exit code 124, no orphan processes, and the dump listed each process (bun test --isolate in do_epoll_wait, turbo in futex_wait_queue, bun x in do_wait).
This is diagnostic. Remove it when the cause is fixed.
Summary by CodeRabbit
- Chores
- CI test runs now have a time limit and provide additional diagnostic information when a run fails or is canceled. This helps make test-run issues easier to investigate; there are no user-facing feature changes in this update.