Offload large GZip compression

Merged
KL
3mo ago
+92-9

Summary

  • offload GZip body chunks of at least 128 KiB to an AnyIO worker thread
  • add thread_minimum_size as an optional GZipMiddleware setting
  • use a dedicated, private, per-event-loop capacity limiter (40 tokens, matching AnyIO's default worker-thread concurrency) so GZip work does not compete with the shared thread pool
  • update the ASGI benchmark runner to support middleware calls that suspend (landed separately in #3412)

Rationale

Compressing a multi-megabyte response inline blocks the event loop until zlib finishes — ~150–310 ms for a 10 MiB level-9 response, freezing every other request on the loop. zlib releases the GIL, so moving sufficiently large chunks to a worker thread lets unrelated requests continue while preserving the inexpensive inline path for smaller chunks.

Built on the zlib.compressobj refactor (#3411): compress() performs all deflate work inside the call, so the offloaded path is simply compressor.compress(body) running in the worker with no flush workarounds.

The dedicated limiter keeps GZip work from starving AnyIO's shared worker-thread capacity (used by run_in_threadpool, FileResponse, etc.).

Threshold choice (128 KiB)

The worker-thread dispatch costs a fixed ~70 µs per chunk. Below 128 KiB that overhead is a meaningful fraction of the compression work (offloading 32 KiB streamed chunks measured up to +18% total time on realistic data), while compressing a chunk just under 128 KiB inline blocks the loop for at most ~1.7 ms (random data) to ~7 ms (level-9 worst case) — within normal scheduling jitter. At 128 KiB and above, dispatch overhead is ≤4% of the compression work.

Compatibility

thread_minimum_size defaults to 128 KiB. Existing middleware configuration remains valid. Responses below minimum_size, responses with an existing Content-Encoding, server-sent events, and http.response.pathsend continue through their existing paths.

Benchmark

The responsiveness benchmark (baseline in #3409) schedules a tiny bypass response behind a 10 MiB level-9 response: inline compression blocks the tiny response for the full ~190 ms; with offload it completes in ~0.25 ms (~750× improvement), while the large response's total completion time is unchanged.

Note: CodSpeed's instruction-counting simulation can only observe the dispatch cost of offloading, not the event-loop time it frees — the remaining ≥128 KiB single-chunk deltas are expected.

Validation

  • Full test suite and focused middleware/benchmark suites pass on Python 3.10 and 3.14
  • scripts/check