feat(runner): host e2e sessions in another test runner
Merged
BH
14d ago+2375+2375+0123456789012345678901234567890123456789-288-288-012345678901234567890123456789
Projects on Playwright Test, Vitest, Jest, or another harness can use e2e engines without moving their suites to e2e run.
openHost<Fixtures>()one2e/runnerowns the worker lifetime: prepare, app processes, engine init, once per worker.host.openSession({ name, timeout, signal, agent, onStep })opens one test-lifetime attempt: fresh fixtures, deadline, steps, artifacts, and atrace.mdat close.- A hosted attempt is published, so
expect.softcollects for the session andexpect.pollhonors its budget and cancellation. One hosted attempt may be opening, live, or closing per process. - The session
timeoutis enforced for every operation: when it passes, running steps fail withTEST_TIMEOUT, the session is timed out, andclose()reports it. An engine that ignores its operation timeout, or a model call, cannot outlive it. - A launch that fails or is cancelled leaves the host usable, as
e2e runkeeps its worker for the retry. Only failed engine cleanup retires a host. - Artifacts land under
results/host/<safe segment>/; the session name never becomes a raw path component. Opening errors, notices, titles, and traces are redacted. - Both lifetimes have idempotent
close()and async disposal. Separate hosts and config loads cannot bind secrets or engine ownership to one another. - The MCP one-shot path stays internal and shares the worker (
packages/e2e/src/run/standalone.ts), the attempt (standalone-attempt.ts), and target selection (select-target.ts). No new dependencies, engine contract, or CI changes.
const test = base.extend<{ e2e: E2ESession<Fixtures> }, { e2eHost: E2EHost<Fixtures> }>({
e2eHost: [async ({}, use) => {
const host = await openHost<Fixtures>({ target: 'android' });
try { await use(host); } finally { await host.close(); }
}, { scope: 'worker' }],
e2e: async ({ e2eHost }, use, testInfo) => {
const session = await e2eHost.openSession({ name: testInfo.titlePath.join(' › '), timeout: testInfo.timeout || undefined });
try { await use(session); } finally {
const failures = await session.close();
await testInfo.attach('e2e trace', { path: session.tracePath, contentType: 'text/markdown' });
if (failures.length > 0) throw new AggregateError(failures, 'e2e session close failed');
}
},
});
flowchart LR
W[Worker fixture] -->|openHost: prepare, processes, init once| H[Prepared host]
H -->|openSession with test name| S[Reserve, publish, launch attempt]
S --> A[Fixtures, soft expectations, bounded polling, deadline timer]
A -->|session.close| C[End attempt, write trace, release publication]
C -->|next test| H
H -->|host.close| D[Dispose engine, finish, stop processes]
Verified
Ran it locally: yes
- A scratch Playwright Test project (
@playwright/test1.64.0) hosting the testbed web target in real Chromium through the builtdist, four tests on one worker: profile form filled; fresh form state on the reused host (samerunId, one host boot of ~500 ms, sessions open in under 10 ms);expect.softfailure returned byclose()and failing the Playwright test; atap()on a missing element with the sessiontimeoutat 1500 ms settled at 1463 ms withTEST_TIMEOUT, the trace page readingtimed out.
[host] opened in 492ms runId=01a13158-795c-7936-8579-70ca501493b7
[session] "fills a profile form" opened in 9ms ... closed, failures=[]
[session] "starts with fresh form state on the reused host" opened in 3ms ... closed, failures=[]
[session] "collects a soft failure and fails the test at close" closed, failures=["ASSERTION_FAILED: 1 soft assertion failed ..."]
[step] locator.tap getByLabel("No such field"): failed TEST_TIMEOUT
[deadline] tap settled after 1463ms code=TEST_TIMEOUT message=session timed out after 1500 ms
[host] closed, failures=[]
4 passed (3.5s)
- Fake-engine coverage in
packages/e2e/tests/integration/open-host-*.test.ts: lifecycle and reservation races, deadline (polls, contributed ops, a hung core op), redaction, engine ownership, trace pages. - Your runner owns retries and test-body failures; e2e writes per-session traces, not a run report. The session timeout starts after launch and does not bound provisioning or disposal.