fix: cert-generator waits up to 300 s for Kueue's controller

HA
2h ago
+2-1

Summary

Follow-up to #402, which raised e2e-ready's wait for Kueue's controller from 120 s to 300 s but missed cert-generator, the compose service that installs Kueue in every stack. In #407's frontend shard, cert-generator's own 120 s wait timed out at 00:55:02. set -e then stopped setup-k8s.sh before it created the ResourceFlavor, ClusterQueue and LocalQueue. The stack came up anyway, since the entrypoint runs setup-certs.sh after a ; (deliberate since #184). From then on every executor pod was refused: "failed calling webhook mpod.kb.io ... no endpoints available for service kueue-webhook-service".

Changes

setup-k8s.sh waits up to 300 s for deployment/kueue-controller-manager, with the same one-line reason as #402: Kueue's cert bootstrap answers /readyz with 404 until the kubelet mounts its new secret.

[!WARNING]
In production Kueue is usually installed already, so this wait returns at once. On an empty cluster (the yearly k3s certificate reset), a slow Kueue start can now outlast the deploy's docker compose up --wait --wait-timeout 180 and fail the deploy loudly, where today it comes up without the queues.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • CI/CD or tooling changes

Test Plan

  • Backend unit tests pass (cd backend && uv run pytest tests/unit)
  • Frontend unit tests pass (cd frontend && npm run test)
  • Backend E2E tests pass (docker compose exec -T backend sh -c 'uv sync --group test --no-dev --frozen --no-install-project --no-build && pytest tests/e2e')
  • Frontend E2E tests pass (cd frontend && npm run test:e2e)
  • API spec and client regenerated if routes or schemas changed (./deploy.sh openapi, then ./deploy.sh types), never edited by hand
  • Manual testing performed

Validation: a one-value change; both E2E jobs on this PR run cert-generator with it.

Checklist

  • My code follows the project's style guidelines
  • I have run pre-commit run --all-files and all checks pass
  • I have added tests that prove my fix/feature works
  • I have updated documentation if needed
  • My changes don't introduce new warnings

Summary by cubic

Fixes cert-generator's wait for Kueue's controller-manager so its ResourceFlavor, ClusterQueue, and LocalQueue get created on slow starts. Follow-up to #402, which raised the same wait in e2e-ready but missed cert-generator.

setup-k8s.sh now waits up to 300s instead of 120s. The shorter timeout could expire while Kueue's cert bootstrap answers /readyz with 404 until the kubelet mounts its new secret; set -e then stopped the script before creating the queues, leaving every executor pod refused with no endpoints for kueue-webhook-service. On clusters where Kueue is already installed the wait returns immediately; on a slow fresh start it may outlast the deploy's docker compose up --wait --wait-timeout 180 and fail loudly instead of coming up without the queues.