Feat: store sensitive workflow data in Secrets instead of ConfigMaps

PA
3mo ago
+816-34

Description

Implement dual-store mechanism to store sensitive workflow data in Kubernetes Secrets instead of ConfigMaps.

Fixes: https://github.com/kubevela/kubevela/issues/7153
Depends on: https://github.com/kubevela/pkg/pull/133

Problem

KubeVela's workflow engine stores inter-step runtime state in ConfigMaps. When workflow steps pass sensitive data (tokens, credentials, API keys), that data is persisted in plaintext in the ConfigMap, creating a security vulnerability.

Solution

This PR implements a dual-store mechanism:

  • ConfigMap - stores non-sensitive workflow variables (existing behavior)
  • Secret - stores sensitive workflow variables (new)

Two paths are covered:

  1. workflow.steps[].outputs[] - via pkg/hooks/data_passing.go

    • When sensitive: true is set on an output, it routes to the Secret store

  2. op.ws.#DoVar - via pkg/providers/builtin/workspace.go

    • Added Sensitive bool field to VarVars struct
    • Put with sensitive: true stores in Secret
    • Get tries sensitive store first, falls back to regular store

Changes

  • pkg/context/context.go - Added secretStore, sensitiveVars, GetSensitiveVar(), SetSensitiveVar(), syncSecret(), LoadFromSecret()
  • pkg/context/interface.go - Added interface methods for sensitive vars
  • pkg/context/storage.go - Added in-memory Secret storage for testing
  • pkg/context/context_test.go - Added 8 tests for sensitive vars functionality
  • pkg/hooks/data_passing.go - Routes outputs with sensitive: true to Secret store
  • pkg/hooks/data_passing_test.go - Added tests for output routing
  • pkg/providers/builtin/workspace.go - Added Sensitive field to VarVars, updated DoVar()
  • pkg/providers/builtin/workspace_test.go - Added 2 tests for sensitive DoVar
  • pkg/providers/legacy/workspace/workspace.go - Same changes for legacy provider

Usage

Via outputs:

workflow:
  steps:
    - name: read-secret
      type: read-object
      outputs:
        - name: secretData
          valueFrom: output.value.data
          sensitive: true  # Store in Secret instead of ConfigMap

Via DoVar in CUE templates:

#DoVar: op.ws.#DoVar & {
  method: "Put"
  path: "myToken"
  value: secretData
  sensitive: true  # Store in Secret instead of ConfigMap
}

Backward Compatibility

  • Existing workflows without sensitive flag continue to work unchanged
  • Secret store is created lazily only when sensitive data is stored
  • Get operations automatically fall back to ConfigMap if data not in Secret

Testing

  • 14 tests pass in pkg/context/... (73% coverage)
  • 3 DoVar tests pass in pkg/providers/builtin/... (80% coverage)
  • pkg/hooks/... tests will pass after kubevela/pkg#133 is merged

Note

⚠️ CI will fail until https://github.com/kubevela/pkg/pull/133 is merged and go.mod is updated. This is expected - raising as draft per maintainer request.

Summary by cubic

Store sensitive workflow variables in Kubernetes Secrets instead of ConfigMaps, with lazy Secret creation, clear read/write precedence, and safety fixes to prevent leaking secrets and runtime panics.

  • New Features

    • Dual store: non-sensitive in ConfigMap; sensitive in Secret workflow-<name>-context-secret (created lazily with owner refs).
    • Read precedence: Inputs read ConfigMap first, then Secret; op.ws.#DoVar Get reads Secret first, then ConfigMap. Writes route by outputs[].sensitive: true and op.ws.#DoVar Sensitive: true.
    • Context API: GetSensitiveVar/SetSensitiveVar, GetSecretStore, secret sync/load, and in-memory secret support; requires kubevela/pkg #133.

  • Bug Fixes

    • Prevent in-memory mode panic by returning early in syncSecret.
    • Consistent secret naming via stored workflowName to avoid owner-ref conflicts.
    • LoadContext uses caller ctx and surfaces non-NotFound errors when loading Secrets.
    • Fix races with RW locks, normalize namespaces in in-memory getters, and make delete match across namespaces.