Let a cancelled response close be retried

Closed
DS
3d ago
+115-2

This pull request was posted by OpenCode on behalf of David.

Summary

Closes #1326.

Response.aclose() set is_closed before awaiting the stream's close. Since 2.12.0, aiter_raw() runs that close in a finally, so it happens inside whichever cancel scope cancelled the read: a deadline cut the close short while the response was already flagged closed. Every later aclose() returned early, so the interrupted close could never be finished.

aclose() now clears is_closed when the stream close raises. The flag is still set before the await, so a concurrent aclose() still returns early instead of closing twice, but a failed or interrupted close no longer claims to be done.

Scope. This affects transport streams whose aclose() awaits without shielding: a custom transport, or a MockTransport handler that returns its own stream, as in the issue's reproduction. The default transport shields its close in httpcore2 (PoolByteStream.aclose, and the HTTP/1.1 and HTTP/2 stream classes), so it releases the connection either way; under cancellation that path is unchanged.

What it does not do. It does not make a close cancellation-safe. With an unshielded transport close, async with client.stream(...) cancelled inside a deadline exits with the stream still open; the caller can finish it with await response.aclose() outside the scope, which is what the client-level test does. client.get() never returns the response, so nothing can close it then. Cancellation safety belongs to the transport that owns the I/O, and the default transport already has it.

Ordinary close failures. The same rollback applies when the stream close raises for another reason: the response stays open and the next aclose() retries it. That is a behavior change from main, and test_failed_close_is_retried_by_a_later_aclose pins it.

Alternatives. Marking the response closed only after a successful close would let a second concurrent aclose() call stream.aclose() while the first is still in flight, and custom streams need not tolerate that. Shielding the close inside aclose() would make a transport close that hangs uncancellable, on top of the default transport already shielding at the layer that owns the I/O; anyio is also not a dependency of httpx2 core, since pyproject excludes it on Emscripten (sys_platform != 'emscripten') and Response imports on every platform.

Deliberate asymmetry. Sync Response.close() keeps the current behavior. Sync code has no cancel scopes, so there is no interruption to recover from, and changing its ordinary-error semantics is not part of #1326.

Out of scope, tracked in #1328. PoolByteStream.aclose() in httpcore2 latches _closed before awaiting, so after an ordinary inner-close failure the retry this PR enables reports success without closing. Pre-existing, and reachable only through that failure path.

Test

test_cancelled_close_is_retried_by_a_later_aclose streams a body that stalls after one chunk under an anyio.move_on_after deadline. Its transport counts close attempts and completed closes, and awaits like a real close so the cancellation cuts it short. It asserts the attempt was made and did not complete, that the response stayed open, and that a later aclose() closed it.

test_failed_close_is_retried_by_a_later_aclose does the same for a stream whose first close raises OSError: the error reaches the caller, the response stays open, and the retry closes it.

test_cancelled_stream_close_is_retried_by_the_client_context is the issue's reproduction through the public client: a MockTransport stream read inside a deadline, then closed from a shielded AsyncExitStack exit. It covers AsyncClient.stream's finally, which the bare-Response tests skip.

All three fail on main and pass with the fix, on the asyncio and trio backends.

  • uv run pytest tests/httpx2/models/test_responses.py tests/httpx2/client/test_async_client.py — pass
  • scripts/check (lock, ruff format, ruff check, mypy, unasync) — pass

Checklist

  • I understand that this PR may be closed in case there was no previous discussion.
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly. No docs change is needed: aclose() is still idempotent, it just no longer reports a stream it failed to close as closed.