fix(qwen35): trim legacy device pools between requests
Merged
BO
2mo ago+148+148+012345678901234567890123456789-15-15-01234567890123456789
Summary
- add an explicit trim hook for legacy CUDA/HIP device pools
- release per-evaluation Q8 memo allocations before trimming cached blocks
- trim target and distinct draft backends when Qwen3.5 request scratch is released
- extend the existing CUDA pool-shutdown regression test through explicit trimming
Root cause
On non-VMM ROCm devices, long prefills returned operator temporaries to the legacy ggml pool, but the pool retained those blocks indefinitely. A later request with a different allocation shape could require another large contiguous allocation and fatally OOM even though the previous request had released its scratch tensors.
Validation
- HIP gfx1201 release build completed for dflash_server and test_server_unit
- two independent back-to-back requests completed on an AMD Radeon AI PRO R9700: 40,013 and 39,234 prompt tokens
- request cleanup returned 2,405.3 MiB and 2,243.6 MiB from device pools respectively
- unit suite: 421/423 passed in the restricted workspace; the two socket-permission failures passed when rerun with normal socket permissions
- git diff --check clean