Maintain connection reservations incrementally in the pool
Merged
KL
2mo ago+400+400+012345678901234567890123456789-28-28-01234567890123456789
Summary
Stacked on #1075. Implements the follow-up suggested by @11kkw there: the last O(in-flight) cost per assignment pass was rebuilding request_connections from the full request list, which is quadratic in aggregate across a large burst.
Changes
- Replace the per-pass
{r.connection for r in self._requests}rebuild with a refcounteddict[connection, int]on the pool. - Update it at the mutation points: assignment,
ConnectionNotAvailablerequeue (now under the thread lock), cancellation, and response close. - Sync pool generated via unasync as usual.
Benchmarks
Unbounded concurrent requests against a local uvicorn server, best of 3:
| n | #1075 | this PR |
|---|---|---|
| 1,000 | 0.33s (3072 rps) | 0.34s (2972 rps) |
| 5,000 | 1.75s (2853 rps) | 1.50s (3325 rps) |
| 10,000 | 3.85s (2595 rps) | 3.07s (3257 rps) |
Throughput on #1075 decays as burst size grows (the residual quadratic term); with this change it stays flat, making total pool work effectively linear.
Verification
- Full suite passes with 100% coverage; mypy/ruff/unasync clean.
- HTTP/2 against hypercorn over TLS: 500-request cold burst and warm-idle burst multiplex on a single connection, unchanged.
- Real services with
http2=Trueunchanged; streaming, cancellation storms, keepalive expiry,PoolTimeout, and threaded syncClientall behave as before.
AI Disclaimer
This PR was developed with the assistance of either Claude or Codex. I've reviewed and verified the changes.