node:test: report a verdict for each suite registered with bun:test under run()
RO
3mo ago+593+593+012345678901234567890123456789-84-84-01234567890123456789
Problem
- Under
node:test'srun(), adescribe()/suite()block declared at the top level of a file (or nested in another one) never gets atest:pass/test:failevent of its own. Node v26.3.0 reports one after the suite's children. Only suites declared inside a running test (inline suites) were reported. Same on 1.4.0 and main; never worked. - Consequences for
run()consumers: thesuites Ndiagnostic andcounts.suitesundercount (the file in the test below reportssuites 3on main, node sayssuites 7), and a failure of the suite itself is invisible: a failing suite-levelafter()hook, or adescribe()callback that throws, only surfaced as a synthesizedtest:failfor the whole file carrying the child's stderr, where node reports the suite as failed (failed running after hook,hookFailed) and nothing for the file. - Cause, in
src/js/node/test.ts:addSuite()registers such a suite as a bun:testdescribeblock and nothing reports it when the block is done;reportNodeToRunParent()only handles tests, and the suite event was emitted fromscheduleSuiteSubtest()alone, the inline path. On the parent side,runOneFile()derived the per-filesuccessand the "did the file itself die" check fromcounts.failed, which counts tests only, so a failed suite could not make a file unsuccessful and would not have stopped the file-level verdict either.
Fix
- Child side, only when the file runs as a
run()child (NODE_TEST_CONTEXTset): the describe callback wrapper (buildReportedSuite()) registers a bun:testbeforeAll/afterAllpair in the block before running the callback. bun:test runs a block'sbeforeAllhooks before and itsafterAllhooks after everything declared in the block, nested blocks included, so thebeforeAllstarts the suite's clock and theafterAll(settleReportedSuite()) runs the suite'safter()hooks, stopping at the first failure like node'srunHook(), and reports the verdict. Nested suites therefore report before their parent, and children before their suite, which is node's order. - The verdict is node's
Suite.run()+Test.postRun(), inconcludeSuite(): a failure recorded against the suite itself wins (abefore()/after()hook failure, recorded asfailed running <kind> hook/hookFailedwith the hook's error as cause, or a throwing / rejecting describe callback), otherwise children that finished without passing and are not todo fail it withN subtests failed/subtestsFailed.rollUpIntoEnclosingSuite()is the roll-up for tests and suites registered with bun:test; inline subtests keep theirs. Underrun(),after()hooks of such a suite are collected on the node for the closing hook to run (their failure still reaches bun:test through the hook'sdone(err), exactly whatafter()handed it before, so the child's exit status is unchanged);before()hooks keep registering with bun:test and additionally record their failure on the suite. - A todo suite's own failures are advisory in node (
countCompletedTestleavessuccessalone, the process exits 0): they are recorded so the suite reportstest:failwith thetododirective, and not handed to bun:test. The same path now reports todo suites in general, which were previously reported as passed at declaration time, before their children; skipped suites are unchanged (reportSkippedNode(), formerlyreportDirectiveOnlyNode(), is now only ever called for skips). - A describe callback that throws or rejects is reported at declaration time and rethrown: bun:test drops the whole block in that case, hooks included, so this is the one verdict that cannot be emitted in execution order. The inline suite path (
scheduleSuiteSubtest()) now goes through the sameconcludeSuite()/reportSuiteToRunParent(), so both kinds of suite produce the same event shape; its only visible changes are a realduration_msinstead of 0 and the first failure attached as the cause ofsubtestsFailed, whichexecuteTestNode()already did for tests (makeSubtestsFailedError()is the shared helper). - Parent side:
runFiles()/runOneFile()carry node'sharness.successnext to the counters (RunTally).republishChildEvent()clears it for a failed or cancelled test and for a failed non-todo suite, as node'scountCompletedTest()does; the per-file and run-leveltest:summaryreport it, and the file-level verdict is now synthesized only for a nonzero exit that no reported failure accounts for (previously: no failed test), so a suite-only failure yieldssuccess: false,fail 0, and no file-level event, as in node. - Why this is correct: it matches node. For every fixture in the new tests, node v26.3.0 and this branch agree on the sequence of verdicts (names,
details.type, directives, error messages and failure types), on thesuitesdiagnostic and on everytest:summaryfield the change touches; the comparison is in the details block. The known remaining differences are listed there too; none of them changes a verdict or a count. - Both new hooks declare and call
done, like every other callback this module hands to bun:test: bun:test derives "waits for done()" from the callback's arity, and cannot read it through the async-context wrapper a callback registered under an activeAsyncLocalStoragecontext gets, which is the case for every nested describe callback (it runs inside its parent'srunWithNode()). A zero-arity hook there waits 5s for adone()nobody calls and, for abeforeAll, skips the block's tests. #38910 fixes that in bun:test; the code here is right either way. - Plain
bun teston a node:test file is untouched: every new code path is behindrunChildReporterEnabled, androllUpIntoEnclosingSuite()returns immediately outside arun()child. - Verified:
test/js/node/test_runner/node-test.test.ts, newnode:test run()block (5 cases: the suite sequence above, suite-only after-hook failure, todo suites' own failures, before-hook failure, throwing and rejecting describe callbacks). All 5 fail on the unfixed build (no suite verdicts,suites 3instead of 7, file-level verdicts instead of suite ones) and pass with it; the 45 existing cases in the file still pass. The portedtest/js/node/test/parallel/test-runner-*files behave as on main (test-runner-todo-skip-tests.js, which pins a skipped suite's single event, included;test-runner-mock-timers-scheduler.jsfails its 100ms wall-clock bound on the unmodified debug build too).test/regression/issue/{19111,19412,24147,24338,24339,24374,26915,28431}.test.tspass.
Background
run()is node:test's programmatic runner. With the default process isolation it runs each file in a child process; the child reports its tests as events, and the parent republishes them on the streamrun()returned and appends the run-level plan, diagnostics andtest:summary. Bun'srun()spawnsbun teston each file withNODE_TEST_CONTEXTset; in that modenode:testprints one JSON event per verdict, which the parent rebuilds.- In node every suite is itself reported: after its children, as a
test:passortest:failwhosedetails.typeis"suite".countCompletedTest()(node'stest_runner/utils.js) counts such an event only insuites, never intests/passed/failed, and clears the harness'ssuccessflag when the suite failed and is not todo. That flag is whattest:summaryreports assuccess; it is the only place a suite failure shows up in the totals. - Node's
FileTestsynthesizes a verdict for the file itself only when the child exited nonzero without any of its own top-level verdicts explaining it (a file that failed to load, a root-level hook failure); a file whose tests or suites failed is represented by those verdicts. - Bun's
node:testregisters top-level tests and suites with bun:test (Bun.jest), which then drives execution: tests through a runner callback, suites asdescribeblocks. Suites declared while a test is running are executed by the module itself ("inline" suites). bun:test orders a block's hooks around its contents, so abeforeAll/afterAllpair registered from the describe callback observes the start and the end of everything in the block.
node v26.3.0 vs. this branch on the test fixtures, and the remaining differences
Verdicts as name (details.type, directive), in stream order, for suites.js from the first test. The left column is node v26.3.0 and is also, line for line, what this branch emits:
node v26.3.0 = this branch bun main
skipped suite (suite, skip) skipped suite (suite, skip)
ok todo suite (suite, todo)
bad FAIL ok
inner (suite) FAIL "1 subtest failed" bad FAIL
ok2 ok2
outer (suite) FAIL "1 subtest failed" fails inside a todo suite FAIL (todo)
empty suite (suite) todo child FAIL (todo)
fails inside a todo suite FAIL (todo) skipped child (skip)
todo suite (suite, todo) inline child
todo child FAIL (todo) inline suite (suite)
skipped child (skip) parent test
suite with directives (suite)
inline child
inline suite (suite)
parent test
suites 7 suites 3
after-hook.js (one suite whose after() throws, one passing child):
node: ok; suite (suite) FAIL "failed running after hook" hookFailed, cause "after boom"; summary success false, fail 0, suites 1
main: ok; after-hook.js FAIL (file-level, error = the child's stderr); summary success false, fail 1, suites 0
this: same as node
todo-suites.js (todo suites whose before(), after(), and callback fail): node and this branch both report the three suites as test:fail with the todo directive and the hook / callback error, success: true, todo 3, suites 3, no file-level verdict. Main reported the suites as passed at declaration time plus a file-level failure.
Remaining differences, all pre-existing or noted in the code:
- Children of a suite whose
before()hook or callback failed: node cancels them (cancelledByParent); here bun:test decides what happens to them. After a failedbefore()they currently run and pass (bun:test only prints an error passed to a hook'sdone(); once it fails the hook instead, #39112, it will skip them), so the before-hook test only pins the suite's verdict and the summary fields that hold either way. After a throwing callback bun:test drops them (no events), except in a todo suite, where the error is not rethrown and they run; there the counts agree with node, which counts the cancelled children as todo as well. - A suite whose callback threw is reported when it is declared (bun:test drops the block), so it appears earlier in the stream than in node. Skipped suites were already reported at declaration time; unchanged.
- Shape: the child does not label test code failures
testCodeFailure(pre-existing for tests, same for describe callbacks here), andsubtestsFailederrors carry the first failure ascausewhere node repeats the message.nesting/testNumber/topLevelare #39284's. - Inline suites still fold their own hook or callback failure into
N subtests failed(pre-existing); the sharedconcludeSuite()would take a recorded error there too.