Let a cancelled or failed `Response.aclose()` be retried
Summary
Response.aclose() sets is_closed before awaiting stream.aclose(). Since 2.12.0, aiter_raw() closes the response in a finally, so a cancelled read runs that close inside the cancelled scope. If the stream's aclose() awaits, it is cut short, but the response was already marked closed, so every later aclose() returned early and the stream stayed open.
aclose() now resets is_closed when the stream close is cancelled or raises, so the next call retries it.
with anyio.fail_after(1):
async for chunk in response.aiter_bytes(): # cancelled here
...
...
with anyio.CancelScope(shield=True):
await response.aclose() # before: no-op, now: closes the stream
The flag is still set before the await. A response has two closers, the iterator and its owner, and asyncio finalizes an abandoned iterator in its own task. Setting the flag first is what keeps those two from both calling stream.aclose().
This is the same approach as #1327.
Scope
- The default transport is not affected.
httpcore2shields its close, so the connection is released either way. This matters for transports and streams whoseaclose()awaits without a shield. - This does not make a close survive cancellation. It lets a caller finish it by calling
aclose()again from a shielded scope. - The sync
close()is unchanged. It has no cancellation to recover from. - On the default transport, a retry after a close that failed with an exception does not release the connection.
PoolByteStreaminhttpcore2keeps its own flag, so the second call returns early there. That is #1328, and it behaves the same onmain.
Behaviour change
When closing the stream raises, is_closed stays False and the next aclose() calls stream.aclose() again. Before, the response was marked closed and the stream's close was never called again.
Closes #1326.
Checklist
- I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
- I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- I've updated the documentation accordingly.
Session: pi --session 01a1e3ea-ab1f-71fe-9b2c-85e666e73d75