Let Slack TaskSpawners create TaskPipelines

KN
2d ago
+16930-547

What type of PR is this?

/kind api

What this PR does / why we need it:

Stacked on #1853. Only the top commit is new; review that one. It lets a
Slack TaskSpawner start a TaskPipeline. Combined with #1853's conditional
stages, one Slack front door can route each request to the right agent without
a new CRD, as discussed on #1761.

kind: TaskSpawner
spec:
  when:
    slack:
      channels: ["C0123456789"]
  pipelineTemplate:
    stages:
      - name: route
        taskTemplate:
          workerPoolRef: {name: router-pool}
          prompt: "Reply docs or none: {{.Params.Body}}"
      - name: docs
        conditions:
          - input: '{{index .Stages "route" 0 "Results" "response" | b64dec}}'
            operator: In
            values: [docs]
        taskTemplate: {...}
  • TaskSpawner.spec.pipelineTemplate.stages creates a TaskPipeline for each
    matching message instead of a Task.
    • The pipeline gets the same name and dedupe behavior a Task would.
    • It carries the spawner label, a controller owner reference, and the Slack
      reporting annotations.
    • It counts toward status.activeTasks and maxConcurrency. The
      TaskSpawner controller watches TaskPipelines so the count updates.
  • TaskPipeline.spec.params is an immutable map[string]string, exposed
    as .Params to stage prompt, branch, and condition templates. The spawner
    fills it from the work item's string template variables (for Slack: ID,
    Title, Body, URL, Kind). This sidesteps a template clash: the
    spawner can't fill in request fields at creation time while leaving
    {{.Stages...}} for the pipeline to fill in later.
  • taskTemplate becomes optional. CEL requires exactly one of
    taskTemplate or pipelineTemplate, so every existing manifest still
    applies unchanged.

This is the second of three stacked PRs:

  1. #1853: conditional stages.
  2. This PR: TaskSpawner pipelineTemplate.
  3. Propagate the initiator's Slack reporting onto stage Tasks, so results return to the thread.

Which issue(s) this PR is related to:

Part of #1760. Depends on #1853.

Special notes for your reviewer:

  • Scope is Slack only, by CEL. pipelineTemplate is rejected with any
    other source and with spec.credentials. Each stage sets its own
    worker/workerPoolRef.
    • The polling spawner, webhook handlers, and manual runs each have their own
      Task-creation and dedupe logic. Wiring them is follow-up work, and lifting
      the restriction later only loosens validation.
    • kelos run --from and Console manual runs return an error naming the
      spawner instead of building a broken Task.
  • How taskTemplate became optional. I used json:"taskTemplate,omitzero"
    with +optional, keeping the value type. Converting it to a pointer would
    touch about 98 references. With omitzero, a typed Go client omits a zero
    taskTemplate, which the integration specs exercise with the typed client.
    The three existing spec-level rules that read self.taskTemplate are now
    guarded with !has(self.taskTemplate) || .... One spec checks that the
    inline-credentials rule still fires for a taskTemplate spawner.
  • v1alpha1 round-trip. pipelineTemplate is kept in a
    kelos.dev/v1alpha2-pipeline-template annotation, following the existing
    preservation pattern. v1alpha1 always serializes taskTemplate, including
    its non-pointer credentials. On the way back up, that empty template is
    dropped so the exactly-one rule still holds.
  • CRD size. PipelineStage embeds the full WorkerSpec schema, so the
    TaskSpawner CRD grows from about 844 KB to about 1.09 MB of compact JSON.
    That is under etcd's 1.5 MiB object limit, but worth knowing.
    TestHelmReleaseSecretFitsKubernetesLimit passes.
  • Slack server RBAC gains create/get on taskpipelines.

Testing:

  • make verify, make test, and make test-integration are clean
    (179/179 specs, plus 12/12 install specs).
  • New unit tests:
    • Slack happy path: a TaskPipeline with params, stages, owner, and reporting
      annotations is created, and no Task is created;
    • duplicate delivery dedupe;
    • a name collision with an unowned pipeline;
    • BuildTaskPipeline, including a deep-copy check;
    • ownership checks against a recreated spawner;
    • manual-run rejection;
    • active-pipeline counting and watch mapping;
    • .Params in prompts and conditions;
    • conversion round-trip, malformed annotation, and no-annotation cases.
  • New integration specs:
    • exactly-one-template in both directions;
    • the Slack-only source rule;
    • the credentials rule;
    • duplicate stage names;
    • the existing rule still enforced;
    • params immutability.

Does this PR introduce a user-facing change?

TaskSpawner accepts `spec.pipelineTemplate` for the Slack source, creating a
TaskPipeline for each matching message instead of a Task. TaskPipeline accepts
`spec.params`, exposed to stage templates as `.Params`; pipeline spawners fill
it from the message. `spec.taskTemplate` is now optional, with exactly one of
`taskTemplate` or `pipelineTemplate` required.

🤖 Generated with Claude Code


Summary by cubic

Lets Slack TaskSpawners create TaskPipelines instead of Tasks, so a single Slack front door can route each message to the right agent using conditional stages.

  • spec.pipelineTemplate.stages defines the pipeline for each matching message; the pipeline keeps Task naming, dedupe, spawner label, owner reference, and Slack reporting annotations, and counts toward maxConcurrency and status.activeTasks.
  • TaskPipeline gains spec.params, an immutable string map exposed to stage templates as .Params; the spawner fills it from the message's string template variables.
  • spec.taskTemplate is now optional — exactly one of taskTemplate or pipelineTemplate is required via CEL, so existing manifests still apply unchanged.
  • pipelineTemplate is Slack-only (rejected with other sources and with spec.credentials); manual runs (kelos run --from, Console) return an error naming the spawner.
  • Stages can declare conditions that render an earlier result and compare with In/NotIn; templates gain b64dec to decode base64 results.

Notes

  • pipelineTemplate survives a v1alpha1 round-trip via a preservation annotation; v1alpha1 always serializes a taskTemplate, which is dropped on conversion back so the exactly-one rule holds.
  • The TaskSpawner CRD grows to about 1.09 MB of compact JSON, still within etcd's 1.5 MiB object limit; Slack server RBAC gains create/get on taskpipelines.