feat(api): priority-aware server-side API backpressure
Why
When an Infrahub instance runs heavy background work (generators, artifacts, diffs, repo syncs, computed attributes) while a human uses the frontend, both compete for the same finite uvicorn worker pool and Neo4j connection pool. Today the API is origin-blind — frontend and background requests are indistinguishable — so under heavy background load the API can no longer serve the frontend and the app appears to hang.
Goal: keep the frontend responsive under background overload by shedding load by priority, with no per-customer tuning.
Non-goals (out of scope, tracked separately): the client/SDK backoff that consumes Retry-After, the frontend setting X-Priority: high, and enforcement that a high claim is legitimate. This PR is server-side only.
Implements Jira IFC-2886 (which implements JPD idea INFP-636).
What changed
Behavioral changes (what operators/callers observe):
- Requests may declare priority via a new
X-Priorityheader (high/normal/low); missing or invalid →normal, so the layer is safe to deploy before any caller is updated. - Under overload the server sheds low-priority requests first and protects high-priority (interactive) requests, returning a fast
429 Too Many Requests+Retry-Afterwith no handler work performed. Shedding is adaptive (CoDel on measured slot-wait / sojourn time), not a fixed threshold, and self-terminates within a bounded window after overload ends. - Eight new per-priority metric families on the existing
/metricsendpoint (infrahub_admission_*): offered, admitted, rejected-by-reason (codel/backstop), in-flight, waiters, sojourn histogram, derivedmax_concurrency, and a no/invalid-priority counter for adoption tracking. - New config knobs (
INFRAHUB_API_BACKPRESSURE_*) including a kill-switch (backpressure_enabled, default on) and a newINFRAHUB_DB_MAX_CONNECTION_POOL_SIZE(default100, now passed explicitly to the Neo4j driver).
Implementation notes:
- New package
backend/infrahub/api/admission/: a pure-ASGIAdmissionMiddleware(registered outermost), a cancellation-safePrioritySlotPool(per-class FIFO, priority hand-off, modelled onasyncio.Semaphore), a pureCoDelControllerwith an injected clock, anAdmissionControllercomposing them, capacity derivation, and the metrics module. - Per-worker and coordination-free by design (no shared/global limiter, no Redis).
max_concurrencyis derived from the process's own Neo4j pool size — no hard-coded constant.
What stayed the same:
- No database schema or migration change; no GraphQL change; no persistence.
- No new hard dependency (
prometheus-clientwas already present transitively and imported in-tree; it is now declared explicitly). - Default behaviour is unchanged under normal load: with no caller sending
X-Priority, everything isnormaland nothing is shed until load exceeds the derived cap.
Suggested review order
contracts/x-priority-header.mdandcontracts/metrics.mdin the spec dir — the external contract.slot_pool.pyandcodel.py— the concurrency core (cancellation safety, CoDel state machine).controller.py→middleware.py→server.py— how they wire together.- Tests under
backend/tests/unit/api/admission/andbackend/tests/component/api/test_admission_middleware.py.
Related Context
Full spec, plan, research, data model, contracts, critique, and implementation report live in dev/specs/ifc-2886-priority-api-backpressure/. Key success criteria: high shed rate ≈ 0% under sustained overload while low sheds first (SC-002); sub-interval bursts shed zero (SC-003); every shed is a fast 429 + Retry-After with no handler work (SC-004); shedding self-terminates after overload (SC-005).
The headline latency bound (SC-001) is intentionally discovery-measured against a live overload scenario (see Test Plan) rather than pre-committed.
Documentation Updates
- Added changelog fragment
changelog/+ifc-2886.added.md. - Regenerated
docs/docs/reference/configuration.mdxfor the new settings.
How to review / test
# Deterministic unit tests (fake-clock CoDel, slot-pool cancellation, capacity, parser)
uv run pytest backend/tests/unit/api/admission/ -v
# Component tests (middleware end-to-end: classification, 429+Retry-After, gradient, metrics, kill-switch, excluded paths)
uv run pytest backend/tests/component/api/test_admission_middleware.py -v
Both suites pass locally with no external infrastructure (42 tests). CI to watch: backend unit/component, ruff/mypy lint, and validate-generated-documentation (the regenerated config reference is committed).
Live overload / SC-001 discovery (manual, needs a running server):
export INFRAHUB_DB_MAX_CONNECTION_POOL_SIZE=4
export INFRAHUB_API_BACKPRESSURE_CODEL_TARGET_SECONDS=0.005
export INFRAHUB_API_BACKPRESSURE_CODEL_INTERVAL_SECONDS=0.1
# drive a saturating `X-Priority: low` stream + a steady `X-Priority: high` stream, then:
curl -s localhost:8000/metrics | grep infrahub_admission_
# expect rejected_total{priority="low"} climbing while {priority="high"} stays ≈ 0
Impact & rollout
- Backward compatibility: fully compatible. No
X-Priorityheader →normal, no behaviour change under normal load. No request/response body change for non-shed requests. - Performance: protects contended API/Neo4j capacity under load; hot-path cost is a header read + slot attempt + CoDel eval (O(1), per-worker).
- Config/env changes: new
INFRAHUB_API_BACKPRESSURE_*settings andINFRAHUB_DB_MAX_CONNECTION_POOL_SIZE(default 100 preserves current driver behaviour). - Deployment notes: safe to deploy. Ships inert under normal load; kill-switch
INFRAHUB_API_BACKPRESSURE_ENABLED=falseis the instant rollback. Interactive protection is fully realized once the frontend sendsX-Priority: high(separate ticket); until then interactive traffic isnormal. - Governance heads-up: new
X-Priorityrequest header +429behaviour (public API surface); the middleware sits in the admission path and reads a client-controlled header (borderline auth, no auth logic changed);X-Priorityis untrusted and cooperative in v1 (enforcement deferred).
Checklist
- Tests added/updated (42 unit + component tests)
- Changelog entry added (
changelog/+ifc-2886.added.md) - External docs updated (regenerated
configuration.mdx) - Internal .md docs updated (spec/plan/contracts/report under
dev/specs/ifc-2886-priority-api-backpressure/) - I have reviewed AI generated content
🤖 Generated with Claude Code
Summary by cubic
Adds priority-aware, server-side API backpressure with CoDel- and database‑stress–driven shedding and adaptive Retry-After to keep interactive requests responsive under load. The per-worker cap derives from the DB pool with a default concurrency factor of 0.5 (pool × 0.5), enabled by default behind INFRAHUB_API_BACKPRESSURE_ENABLED.
-
New Features
- Priority admission (
high/medium/low), exact excluded-path matching, CORS preflight bypass, and CORS allow‑list forx-priority; per‑priorityinfrahub_admission_*metrics;prometheus-clientdeclared. - Database stress-aware shedding from a rolling median of the reference permission query with tiered, graduated fractions (
reason="stress"), alongside CoDel; newINFRAHUB_API_BACKPRESSURE_*knobs; DB pool default 100. - Adaptive Retry-After scaled by shed intensity and sustained-load duration (new sustained-load metric), capped under the SDK per‑retry ceiling.
- Client integration: the frontend sends
X-Priority(high by default; low opt‑in; preserved on retries/uploads; only for the Infrahub API origin). Background flows set SDKRequestContext.priority; artifact generation no longer forces HIGH. Thepython_sdktracksinfrahub-developwith a 429 retry budget of 10.
- Priority admission (
-
Bug Fixes & Refactors
- Slot‑pool observer failures can’t leak slots or strand waiters; non‑finite concurrency factor is rejected at config load.
- Admission controller is built at app startup and read from
app.state; excluded paths and CORS preflights bypass admission before reading startup state; exclusions match exact path or descendant and include/api/config; middleware decision branching is exhaustive. - CI: disable backpressure only for the legacy Playwright E2E job via
INFRAHUB_API_BACKPRESSURE_ENABLED=false; default remains enabled elsewhere. - Internal cleanup:
RetryAfterPolicynow pushes sustained‑load duration to a registered observer (metrics‑free, sink pattern), wired in the controller; renamedcodelmap tocodel_priority_mapfor clarity. - Docs: extracted IFC‑2886/IFC‑2890 into ADRs and backend/frontend knowledge; archived specs as “extracted” and fixed index links and an archived‑spec knowledge link depth; no behavior change.
Update since original description
- Tier rename
normal→medium. Header values are nowhigh/medium/low; a missing or invalid value resolves tomedium. Read everynormalabove asmedium. - Database-stress signal added. The admission decision now also consumes a per-worker signal that times the global permission (reference) query: past per-class stress-ratio triggers it sheds a graduated fraction of a class, reported as
reason="stress"alongsidecodel/backstop. This augments the CoDel sojourn signal described above. - Config defaults retuned. DB connection pool
100 → 50, admission concurrency factor1.0 → 0.5, plus newINFRAHUB_API_BACKPRESSURE_*stress knobs. - New backend knowledge doc
dev/knowledge/backend/api-backpressure.mddocuments the admission layer and the stress signal (the IFC-2886 spec predates the stress signal and the rename).