[python] re-resolve the environment proxy when the host is assigned

Merged
YU
2mo ago
+218-31

Configuration.__init__ picks the environment proxy by the scheme of self.host:

scheme = urlparse(self.host).scheme
proxy = proxies.get(scheme) or proxies.get("all")

host is a property, not the constructor argument, and server_index falls back to 0 when no host is given. For a spec with no server URL that makes host an empty string, so urlparse("").scheme is "", proxies.get("") misses, and only the proxies.get("all") fallback can match. Clients that construct Configuration() and assign the host afterwards never see HTTP_PROXY or HTTPS_PROXY, and requests go direct with no warning. ALL_PROXY is the only variable that works, which is a confusing thing to have to discover.

$ HTTPS_PROXY=http://127.0.0.1:8888 python -c "
from petstore_api import Configuration
c = Configuration(host='')
c.host = 'https://api.example'
print(repr(c.proxy))"
None

The host is simply not known at construction time in that pattern, so no reordering inside __init__ can fix it. This re-resolves the proxy in the host setter, which is the point at which the scheme first becomes known.

Construction is unchanged — same code, same order, same result — so nothing that works today behaves differently. The only addition is that assigning a host refreshes a proxy that came from the environment. An explicitly provided proxy is never touched: the property setter clears the "came from the environment" flag, so config.proxy = X survives a later config.host = Y, and config.proxy = None still means no proxy.

I first tried resolving lazily on every read of proxy. That also fixes the bug but makes the value re-read the environment on each access, and it broke test_proxy_settings_default_from_environment, which constructs inside patch(getproxies) and asserts outside it. Re-resolving on host assignment keeps that test passing unmodified, which seemed like the better signal.

Scoped to the urllib3 client with {{^async}}, matching the existing guards on the getproxies/urlparse imports and on no_proxy, and consistent with #24082 leaving the async backends to their own proxy handling. Five urllib3 samples change; the httpx and aiohttp samples never had the block.

Tests: the three existing proxy tests pass unmodified, and five are added covering a host assigned after construction (both schemes), a host unknown at construction, an assigned proxy surviving a later host change, an assigned None, and the ALL_PROXY fallback. tests/test_configuration.py is 18 passed; the rest of the petstore suite is unchanged apart from the usual failures that need a live server.

Reported downstream at kubernetes-client/python#2681, where load_kube_config() builds the configuration with no host and sets the server afterwards. Verified there by patching the generated file: with HTTPS_PROXY set and a kubeconfig pointing at a name that does not resolve, the client goes from proxy=None and a DNS failure to a ProxyManager and the proxy logging CONNECT.

Follows #24082. Related to #20226.

PR checklist

  • Read the contribution guidelines.
  • Run the following to build the project and update samples: 
    ./mvnw clean package || exit
./bin/generate-samples.sh ./bin/configs/*.yaml || exit
./bin/utils/export_docs_generators.sh || exit
    Commit all changed files.
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

@cbornet @tomplus @krjakbrjak @fa0311

Summary by cubic

Re-resolves environment proxies when Configuration.host is assigned so scheme-specific proxies apply. Previously, assigning host after construction skipped HTTP_PROXY/HTTPS_PROXY and sent requests direct unless ALL_PROXY was set; now host assignment refreshes env-derived proxies while explicit proxies remain unchanged.

  • Scope: python generator for urllib3 only; httpx and aiohttp unchanged.
  • Implementation: track env-origin proxies with _proxy_from_env; store _proxy behind a proxy property whose setter disables re-resolution; re-resolve in the host setter using extracted _env_proxy(proxies, host) with ALL_PROXY fallback; no_proxy handling unchanged.
  • Tests: add coverage for host assigned post-construction (both schemes), host unknown at construction, explicit proxy and None surviving host changes, and ALL_PROXY fallback.
  • Migration: none. Clients setting config.host after construction now pick up HTTP_PROXY/HTTPS_PROXY; explicit config.proxy still overrides and None disables proxies.