fix(conversion): queue office conversions and retry on 429

PO
2mo ago
+118-30

Problem

  • An office document upload fails outright when a second conversion lands on a busy container — a server-side capacity failure, not a bad file.
  • The Gotenberg container caps the LibreOffice queue at 1 and jobs pick an instance at random from a pool of 2, so a job that lands on a busy instance is rejected immediately with 429 instead of waiting its turn.
  • The shared helper treats any non-ok response as fatal — no retry, no backoff, and it ignores the "retry shortly" hint — so the rejection surfaces as a failed upload.
  • Low volume today, but the risk grows with concurrency: two people uploading at once is enough.

Changes

  • Queue instead of bounce: raise the LibreOffice queue size so a concurrent job waits its turn.
- LIBREOFFICE_MAX_QUEUE_SIZE: "1",
+ LIBREOFFICE_MAX_QUEUE_SIZE: "10",
  • Retry the transient rejection: convertFileStreamWithContainer now retries a 429 with bounded exponential backoff (3 retries, 250 ms base, 5 s cap) and honors a Retry-After hint. Other non-ok responses still fail immediately, unchanged.
  • The request body is buffered once (the upload size is already bounded) so each attempt rebuilds a fresh request — the R2 stream cannot be replayed.
  • The retry lives in the shared container helper, so every container conversion benefits, not just office documents.

Testing

  • Added tests: a queue-full response that retries and succeeds, and one that gives up after exhausting the retry budget.
  • pnpm test (conversion + upload suites) and pnpm check (format, lint, types) pass.

Agent context

  • Considered re-fetching the R2 object per attempt instead of buffering; buffering keeps the fix inside the conversion helpers and avoids R2 stream-lifecycle handling, and the upload size is already capped upstream.
  • Retry-After parsing accepts numeric seconds only, matching the existing retry convention in src/integrations/firecrawl/client.ts.

Created with PostHog Desktop from this inbox report.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.