ci: run k3s in a container so executor pods don't change the runner's network

Merged
HA
20h ago
+24-19

Summary

Frontend E2E went red twice on main (38484880660, 38486315543). Each time a page.goto failed with net::ERR_NETWORK_CHANGED about 2.5 s after a test started an executor pod, and failOnFlakyTests failed the job on the passing retry. k3s ran on the runner itself, so every pod added network interfaces next to the browser.

Changes

k3s now runs as a privileged rancher/k3s container, pinned by digest in K3S_IMAGE, that publishes 6443 and mounts /etc/rancher/k3s. The cert-generator finds the kubeconfig where it always did, and the backend and worker still reach host.docker.internal:6443. The crictl pre-pull goes through docker exec k3s. When Kueue's controller misses its 120 s wait, the step prints the kueue-system pods, descriptions and events; that wait failed once in 38486315543 with no reason shown.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • CI/CD or tooling changes

Test Plan

  • Backend unit tests pass (cd backend && uv run pytest tests/unit)
  • Frontend unit tests pass (cd frontend && npm run test)
  • Backend E2E tests pass (docker compose exec -T backend sh -c 'uv sync --group test --no-dev --frozen --no-install-project --no-build && pytest tests/e2e')
  • Frontend E2E tests pass (cd frontend && npm run test:e2e)
  • API spec and client regenerated if routes or schemas changed (./deploy.sh openapi, then ./deploy.sh types), never edited by hand
  • Manual testing performed

Validation: locally with the same image and flags, the kubeconfig appeared in the mounted directory, a container reached the API via host.docker.internal with TLS verified, and a running pod's veths stayed inside the k3s container (host interfaces 6 before and after). The flake only ever hit main, so this PR's green E2E jobs show the setup works, not that the flake is gone.

Checklist

  • My code follows the project's style guidelines
  • I have run pre-commit run --all-files and all checks pass
  • I have added tests that prove my fix/feature works
  • I have updated documentation if needed
  • My changes don't introduce new warnings

Summary by cubic

Runs k3s in a privileged Docker container instead of on the runner itself, so executor pods no longer add network interfaces to the runner and Chromium stops aborting in-flight navigations with net::ERR_NETWORK_CHANGED.

  • Replaces the host install with a digest-pinned rancher/k3s container that publishes 6443 and mounts /etc/rancher/k3s, keeping the kubeconfig location and host.docker.internal:6443 unchanged.
  • Pre-pulls executor images via docker exec k3s crictl after the container is ready.
  • e2e-ready copies the kubeconfig from the mounted directory and retries the node-ready check; if Kueue's controller isn't available within 120 s, it prints the kueue-system pods, descriptions, and events before failing.