feat(testmuai): @e2e-dev/testmuai, TestMu AI hosted browsers for the web engine

Closed
SA
22d ago
+882-6

What and why

  • New package @e2e-dev/testmuai: web({ browser: testmuai() }) runs a web target in TestMu AI (formerly LambdaTest) hosted Chrome or Edge on Windows and macOS. One session per worker slot, or one per attempt with scope: 'attempt', like kernel().
  • A TestMu AI session starts when its CDP websocket opens and ends when it closes, so the provider only builds the URL (wss://cdp.lambdatest.com/<route>?capabilities=…). There's no SDK and no API call, release is a no-op, and @e2e-dev/web is the only peer, like @e2e-dev/eas.
  • LT_USERNAME/LT_ACCESS_KEY are read from request.env and travel only in the CDP URL. The reporter line is TestMu AI session "<name>" in build "<build>", never the URL.
  • Options: scope, browserName, browserVersion, platform, build, route, hub, and capabilities for further LT:Options (video, network, tunnel, idle timeout). Unknown keys, a bad route/hub/browserName, or capabilities that would override what the provider sets fail with INVALID_CONFIG at config load. idleTimeout defaults to 600 s.
  • route defaults to /puppeteer, which serves raw CDP on every account today. /playwright-cdp labels sessions as Playwright on the TestMu AI dashboard; the backend change that makes it serve raw CDP is rolling out, and the default can flip once it reaches production.
  • No record, downloads or sweep: TestMu AI records its own video, downloads land on its machines, and a session a dead worker held ends at TestMu AI's idle timeout. The integration page documents all three.
  • The hub host, LT:Options and the LT_* variable names keep the old brand because they are the service's live contract.
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { testmuai } from '@e2e-dev/testmuai';

export default {
  targets: [
    {
      name: 'testmuai',
      engine: web({ browser: testmuai({ platform: 'Windows 11', capabilities: { video: true } }), viewport: null }),
      app: { url: 'https://staging.example.com' },
    },
  ],
} satisfies E2EConfig;
sequenceDiagram
  participant W as web engine
  participant P as testmuai()
  participant H as cdp.lambdatest.com
  W->>P: acquire(request)
  P-->>W: { id, cdpEndpoint } (caps + LT_* from request.env)
  W->>H: connectOverCDP(cdpEndpoint)
  Note over H: socket open = session starts
  W->>H: close
  Note over H: socket closed = session ends
  W->>P: release() (no-op)

Verified

Ran it locally: yes

  • Packed the package (pnpm pack) and installed the tarball into a scratch project on e2e 0.15.1, so exports/files were exercised: the tarball holds only dist/, LICENSE, NOTICE, README.md and package.json.
  • Real runs on TestMu AI production, /puppeteer, Windows 11 + Chrome, video on, 3 tests on the-internet.herokuapp.com (taps, assertions, evaluate):
ScopeResultSessions
worker2/3 passed1 session for the worker
attempt2/3 passed1 session per test
  • The failing test in both is add and remove an element: the-internet.herokuapp.com stalls its jQuery for about 30 s on a fresh browser's first load, so the tap lands before the page's script binds. Session, connect and the other assertions are fine; the same test passes when the page is warm.
  • Missing LT_ACCESS_KEY fails before any test with browser provider "testmuai" could not lease a browser: LT_USERNAME and LT_ACCESS_KEY must be set, and no session is created.
  • No report, log or artifact file from any run contains the access key.
  • Known limits, documented under Limits on the integration page: in worker scope about 25 workers per target fit the engine's 16 KB lease hand-off (packages/web/src/provider.ts, MAX_HANDOFF_ENV_BYTES; a unit test pins it), and hosts that take more than 60 s to provision (Edge on macOS, for one) fail with connectOverCDP: Timeout 60000ms exceeded. BROWSER_LAUNCH_TIMEOUT_MS is fixed at 60 s in packages/web/src/surface.ts and launchTimeout doesn't reach the CDP connect. Happy to follow up with a separate PR that lets it, if you want that.

Checklist

  • Changeset added (pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only.
  • Breaking change: n/a, new package.
  • Docs updated in the same PR: docs/integrations/testmuai.mdx, nav, integrations card, environment variables, security (outbound connections), README packages table, AGENTS.md, CONTRIBUTING.md, and the skills/e2e setup row.
  • Contract change: n/a, no change to e2e or @e2e-dev/web contracts.
  • Engine contract change (e2e/engine): n/a.

[!IMPORTANT]
Before merge (maintainers): @e2e-dev/testmuai is a new public package. Per CONTRIBUTING.md, publish its first version by hand and connect it to release.yml as a trusted publisher; otherwise the next release's publish step fails. If you'd rather land the code first, I can drop the changeset from this PR and add it in a follow-up once the package exists on npm.

The changeset takes the package from 0.0.0 to a minor, like EAS. The integrations card uses the generic cloud icon until there's a brand SVG to add.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KoNxaN1YAAP9PwmHtMm1WX