Track connection pool state incrementally instead of scanning on every request
Summary
AsyncConnectionPool._assign_requests_to_connections ran on every enqueue and every release and walked every pooled connection each time (several delegated state queries per connection, plus a poll() on every idle socket via has_expired()), so per-request cost grew with the number of connections. With an unbounded pool at 512 concurrent requests, ~85% of client CPU was pool bookkeeping and throughput fell to a third of the c16 figure.
The pool now keeps its state incrementally and a pass is O(1) in the common case:
- Connections and in-flight requests live in insertion-ordered dicts; requests waiting for a connection are a FIFO
deque. - Idle HTTP/1.1 connections are indexed by origin, oldest first, which serves reuse,
max_keepalive_connectionseviction (longest idle first, as before) and the "close an idle connection to another origin when the pool is full" case. - Sharable connections (HTTP/2, or not yet negotiated with
http2=True) are kept per origin and checked withis_available()on lookup, so a burst against a warm HTTP/2 connection is still multiplexed within one pass. - Reservations are refcounted per connection (the idea from #1076), and a pass re-examines only the connections released since the previous pass.
- A server-closed idle socket is detected when the connection is about to be reused (one probe per reuse) instead of probing every idle socket on every pass; keepalive expiry is swept only once the oldest idle connection may have expired.
Origingets a__hash__consistent with its__eq__, andAsyncPoolRequestcreates its wakeup event only when it actually has to wait (the common path assigns before waiting, so noanyio.Eventand nosniffiocall per request).
_assign_requests_to_connections keeps its name and cadence (once per enqueue, once per release), so the existing pass-counting tests hold. #1076 becomes redundant with this change.
Benchmarks
scripts/benchmark --lib httpx2 --python main=... --python pool=..., CPython 3.14 + zuvloop, 2 vCPU, interleaved rounds (medians):
| scenario | main | this PR |
|---|---|---|
| c1/1k | 798 rps · 391 us/req | 817 rps · 371 us/req |
| c16/1k | 2,619 rps · 299 us/req | 3,104 rps · 251 us/req |
| c128/1k | 2,037 rps · p99 90 ms · 426 us/req | 3,494 rps · p99 59 ms · 248 us/req |
| c512/1k | 787 rps · p99 1152 ms · 1058 us/req | 3,267 rps · p99 218 ms · 263 us/req |
| c64/100k | 2,023 rps · 412 us/req | 2,695 rps · 317 us/req |
| c64/1k POST | 2,266 rps · 379 us/req | 2,974 rps · 287 us/req |
| c8/5m, c8/1m POST | unchanged (body-bound) | unchanged |
Retention (rps at c512 over rps at c16) goes from 30% to 105%; per-request CPU is now flat across concurrency. Measured on the final state of the branch, after the review changes.
Tests
New tests cover: an idle connection closed by the server is discarded on reuse; staggered keepalive expiry keeps younger idle connections (HTTP/1.1 and HTTP/2); an externally closed HTTP/2 connection is discarded; and the AsyncEvent/Event primitives directly, since the pool no longer exercises every branch on each request. The async tests use anyio.sleep, mapped to a threaded concurrency.sleep by unasync.
Checklist
- I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
- I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- I've updated the documentation accordingly.
🤖 Generated with Claude Code