feat(redis): mutual TLS (client certificate) for Redis connections

Merged
JU
4mo ago
+187-0

Description

Onyx supports one-way Redis TLS (REDIS_SSL, REDIS_SSL_CA_CERTS, REDIS_SSL_CERT_REQS) but has no way to present a client certificate, so deployments whose managed Redis requires mutual TLS can't connect.

Adds REDIS_SSL_CERTFILE / REDIS_SSL_KEYFILE, threaded into every Redis connection path:

  • Sync pool (redis_pool.py RedisPool.create_pool) — ssl_certfile/ssl_keyfile kwargs on BlockingConnectionPool.
  • Async client (_build_async_redis_connection) — SSLContext.load_cert_chain(...).
  • Celery broker + result backend (celery/configs/base.py) — appended to the rediss:// URL query so the workers get mTLS too, not just the API/app (easy to miss otherwise).

Validated loudly at startup (same style as the Postgres/OpenSearch TLS work): cert and key must be set together, require REDIS_SSL=true, and the files must exist. IAM auth keeps precedence.

Certs are referenced by file path (consistent with the existing REDIS_SSL_CA_CERTS); a managed Redis that hands out base64-encoded certs decodes them into a mounted secret and points these vars at the paths — matching the redis.Redis(ssl_certfile=..., ssl_keyfile=...) shape.

How Has This Been Tested?

  • test_redis_mtls.py: the sync pool forwards ssl_certfile/ssl_keyfile; the async client calls load_cert_chain; the Celery broker and result-backend URLs include the cert params; config validation rejects cert-without-key, client-cert-without-REDIS_SSL, and a nonexistent cert path.
  • backend/tests/unit/onyx/redis + db + utils → 494 passed; ty + ruff + pre-commit clean.

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.
  • [Optional] Override Linear Check

Summary by cubic

Adds mutual TLS support for Redis so the app and Celery workers can present a client certificate. This enables connecting to managed Redis that require mTLS.

  • New Features

    • Adds REDIS_SSL_CERTFILE and REDIS_SSL_KEYFILE (must be set together, require REDIS_SSL=true; paths are validated at startup).
    • Threads client cert/key through all Redis paths: sync via redis.BlockingConnectionPool, async via ssl.SSLContext.load_cert_chain, and Celery broker/result URLs via rediss:// query with percent-encoded paths so special characters don’t break the URL.
    • IAM auth continues to take precedence when enabled.

  • Migration

    • Provide file paths for the client cert and key via REDIS_SSL_CERTFILE and REDIS_SSL_KEYFILE and set REDIS_SSL=true.
    • If your provider gives base64 certs, decode to files (e.g., mounted secrets) and point the env vars to those paths.
    • Startup will fail if only one is set or the files do not exist.