test: Playwright tests run on their own accounts in fresh contexts

Merged
HA
8d ago
+785-1460

The Playwright suite shared one seeded user across workers and one browser context across a worker's tests, so settings, saved scripts, the editor draft and the theme leaked between tests. Several tests could not fail, five were skipped, and the execution helper polled the API in a sleep loop.

Each worker now registers its own account through the API and signs it and the seeded admin in once. Every test gets a fresh context built from those cookies: page for a guest, userPage and adminPage for the two roles. An api fixture is a client signed in as the worker's user, and execution starts an execution through it. That makes the notification tests real (read, delete, mark all) and gives the admin executions and sagas tables a row to find.

Specs are flat test() calls without describe wrappers or helpers defined inside functions. They wait for specific toasts and visible state with retrying expect, and hold the login and execute requests with page.route to check the loading states. Labels come from fieldTitle and option lists from the generated enums and Zod schemas, so a new execution status or saga state shows up in the tests by itself. The admin heading and sidebar checks are one data-driven spec over ADMIN_ROUTES, and access control has its own.

svelte-check and Prettier now cover e2e/, and the type check caught two wrong assertions on the way. CI keeps one retry for a second trace and fails a test that passes only on the retry. The webServer entry is gone, since npm run dev proxies to a hostname that only resolves inside compose.

Ran locally against the stack with k3s in Docker: 86 passed in about 42 seconds, three runs in a row on the same database, then 258 of 258 with 6 workers and --repeat-each=3. npm run check, npm run lint and npm run format:check exit 0.

Summary by cubic

Reworks the Playwright suite so every worker registers its own account through the API and each test gets a fresh browser context built from those cookies. State like user settings, saved scripts, editor drafts, and theme no longer leaks between tests, and the five previously skipped notification tests now exercise read, delete, and mark-all for real.

Test structure

  • page is a guest, userPage and adminPage are signed in; api is a client for the worker's user, and execution starts an execution through it so the admin executions and sagas tables have a row to find.
  • Specs are flat test() calls without describe wrappers or nested helpers; they wait for specific toasts and visible state with retrying expect and hold login and execute requests with page.route to check loading states.
  • Labels come from fieldTitle and option lists from generated enums and Zod schemas, so a new status or state shows up in the tests on its own. Admin heading and sidebar checks are one data-driven spec over ADMIN_ROUTES, and access control has its own spec.

Tooling

  • svelte-check and Prettier now cover e2e/, and CI fails a test that passes only on retry.
  • The webServer entry is gone because npm run dev proxies to a hostname that only resolves inside the compose network.