feat(ratelimit): enforce limits across replicas sharing a database
Draft
SA
2d ago+1176+1176+0123456789012345678901234567890123456789-338-338-012345678901234567890123456789
Description
- Rate limits (requests, tokens, concurrency) now hold across all replicas sharing a database. Before, each replica admitted up to the full limit, so N replicas allowed about N times the limit.
- Each writer's snapshot rows now hold only what that writer counted itself, never counts it loaded from others. The cluster count for a key is the sum of its rows with a matching epoch, each rolled forward to now. This replaces the max-merge restore. A restarted instance's earlier rows count as one more writer's, so each generation is counted once however many restarts there are. The legacy empty-writer row is one total, so it is summed like the others.
- The limiter keeps a remote view per key: the other writers' summed counters, plus their live in-flight gauges. Admission, status,
x-ratelimit-*headers,/v1/usage, and the routing probe all check local + remote. The remote view is refreshed on the flush loop (save, then load) and seeded from the load at Start. The request path never calls the database, and admission adds no allocations (perf guard passes). - The guarantee is approximate. A replica's counts reach its peers within about two flush intervals (one to save, one for peers to load), so the cluster can overshoot by about that much of the other replicas' traffic, plus requests admitted on several replicas at the same instant. The docs say so and point to sticky routing or a single replica for strict caps such as
concurrent=1. A strict per-request mode is left for a follow-up. - Load cost: a flush writes only the counters that changed since the last flush (rows that didn't change keep their counts and still prune correctly after two periods). It reads only rows with
updated_atat or after the previous load minus a 10s overlap, which covers slow commits and clock skew. Rows are merged into a cache keyed by row identity, with a full reload once a minute as a backstop. A newupdated_atindex serves this on SQLite, PostgreSQL, and MongoDB. Rows from dead writers stay cached until their windows roll out. A reset or epoch change clears local and remote counts for the key on every replica, and cached rows of the old epoch are skipped. - Concurrency: each writer publishes its in-flight count per key on every flush (new
in_flightcolumn/field), and publishes 0 once when the count drops. Peers count a row only while it is fresh: saved within max(3 × flush interval, 5s) by the reader's clock, so replica clocks should be NTP-synced. A crashed replica's slots therefore free within that bound. Stores keep in-flight rows for a day, so pruning never races the heartbeat. In-flight gauges ignore the epoch, matching how a local reset leaves them alone. - No new setting.
RATE_LIMITS_FLUSH_INTERVALsets the sharing cadence, and0means counts load at start only. A single instance behaves as before, except that after a crash (not a clean shutdown), a restart within ~5s sees the crashed run's last published in-flight slots until they go stale. - The schema change only adds a column to
rate_limit_window_snapshots, the unreleased table from #1296, plus an index. v0.2.0 never reads that table, so rolling upgrades are unaffected.
Tests: two-replica shared limit (requests, tokens, headers, routing probe), restart across four generations without inflation, reset and mode change clearing peer counts on every replica, peer in-flight counting and expiry on an injected clock, incremental load / full reload / roll-out eviction, failed-save retry, and the shared store suite (since filter, save stamps, in-flight round trip and retention) on SQLite, PostgreSQL and MongoDB. Mutation-checked: 15 targeted mutations, all caught. make test-race, lint, go fix, the hot-path perf guard and mint validate pass.