fix(webfetch): drain fallback response bodies

Merged
CO
2mo ago
+173-33

Summary

  • drain redirect and oversized response bodies when the runtime's undici body does not expose dump()
  • guard discard-time stream errors before quiet destruction
  • preserve the bounded native Undici dump({ limit: 1024 }) path
  • extract response-body cleanup into a focused module so the vendored fetcher stays below the repository's 250 pure-LOC ceiling

Upstream credit

This adapts the lifecycle hardening proposed by @Indosaram in code-yeongyu/pi-webfetch#7. The upstream PR identified that immediate destruction leaves the no-dump() body undrained and that discard-time stream errors need an attached listener before destruction.

Verification

  • RED: the existing Senpi fallback failed new assertions that the no-dump() body was drained and that an error listener was attached before destruction
  • GREEN: npm test -- test/suite/regressions/1089-webfetch-bun-body-discard.test.ts test/extensions-webfetch.test.ts — 2 files, 6 tests passed
  • npm run check — passed
  • changed-file TypeScript diagnostics — no diagnostics
  • real Bun reproduction — 200 https://httpbingo.org/get, exit 0, no uncaught exception
  • pure LOC — fetcher.ts 236, response-body.ts 30, regression test 101

Follow-up to #1091.


Summary by cubic

Drains Bun-compatible webfetch response bodies, guards discard-time stream errors, and makes cleanup abort-aware. Previously, bodies without dump() were destroyed immediately and could remain undrained or emit unhandled errors; now we attach an error listener before discard begins, drain up to the 5MB limit or until aborted, use dump({ limit: 1024, signal }) when available, and always destroy afterward.

  • Extracts cleanup into response-body.ts with discardBody(body, maxBytes, signal) and toUint8Array; fetcher.ts imports these and passes MAX_RESPONSE_SIZE_BYTES for redirects and oversized content-length.
  • When dump() exists, call dump({ limit: 1024, signal }) then destroy; otherwise, bounded, abort-aware async drain then destroy; drain failures are swallowed.
  • Attaches an error listener before discard starts so stream errors during best-effort cleanup do not surface as uncaught process errors.
  • Tests cover draining, aborting a stalled iterator, stopping after the size bound, listener ordering, and destroy order; no external API changes.
  • Updates webfetch/changes.md to credit upstream pi-webfetch #7 and note why extension hooks cannot replace this vendored cleanup.