perf: Rewrite `_assign_requests_to_connections` as a single-pass loop

Merged
MB
5mo ago
+182-82

The previous implementation re-scanned self._connections per queued request to rebuild available_connections and idle_connections lists, giving O(N*M) behaviour when many requests are queued against a populated pool. The keepalive-eviction step in pass 1 was also quadratic, since sum(... is_idle() ...) was recomputed for every connection.

This rewrite:

  • Walks self._connections once to drop closed connections and schedule expired ones for close.
  • Counts idle connections once, then evicts surplus idle connections in a single pass to enforce max_keepalive_connections.
  • Snapshots available_connections once before the queued-request loop, and consults it (rather than re-filtering self._connections) for each request.

Behaviour is preserved exactly:

  • The same connection may still be assigned to multiple HTTP/1.1 requests in a single call. The pool's existing ConnectionNotAvailable retry loop in handle_async_request handles that case as before.
  • HTTP/2 multiplexing onto a single connection is preserved for the same reason — connections are not removed from available_connections on assignment.

Inspired by encode/httpcore#1035 by @VictorPrins.

Citing some issues related to this, including in downstream libraries relying on httpcore: