fix(vtex): time out reviews extension calls so one stall can't hold the render
Summary
vtex/loaders/product/extend.ts → reviewsExt fires 2 calls per product to {account}.myvtex.com/reviews-and-ratings/... with no timeout, and swallows every error (.catch(() => ({}))). When one of those calls stalls, the page render waits for it, and then still returns 200 with no error logged. Since Oct 20 these stalls last ~55.5s and hit several VTEX storefronts.
This PR bounds each reviews/rating call with AbortSignal.timeout(3_000). On timeout the product gets no rating, which is exactly what happens today on any other failure, just without holding the render.
Evidence
1. The stall is in this function
Uncached home render on www.teciplast.com.br (apps 0.153.0), hitting the pod directly (bypassing Cloudflare and ingress) with deco debug timings (?__d=true):
load-data;dur=55522
pages-home…@sections.variants.1.value.6.products.data;desc=hit;dur=85
pages-home…@sections.variants.1.value.6.products.extensions.0;desc=bypass;dur=560 ← normal
pages-home…@sections.variants.1.value.16.products.data;desc=hit;dur=83
pages-home…@sections.variants.1.value.16.products.extensions.0;desc=bypass;dur=55518 ← stalled
Product data is a cache hit (~80ms). The whole 55.5s is the vtex/loaders/product/extensions/list.ts extension. One of the stalled shelves has only reviews: true (no brands), so the call is reviewsExt.
It also reproduces by invoking the loader in-process: 16 parallel POST /live/invoke/vtex/loaders/product/extend.ts with {reviews: true} (17 products each, the same fan-out as one home render). Per round, 1–3 invocations took 55.8–56.2s while the rest finished in 1–2s.
2. The hang is invisible
Because errors are swallowed, the stalled page returns 200 after ~55s with nothing in the logs. Request latency distribution on teciplast (Cloudflare OriginResponseDurationMs, 24h) has a sharp floor:
| bucket (ms) | requests |
|---|---|
| 54000–55250 | 3 |
| 55250–55500 | 76 |
| 55500–56000 | 2135 |
| 56000–56500 | 128 |
3. It's upstream (VTEX reviews-and-ratings) and started ~Oct 20
Daily count of origin responses in the 55–56.5s band, per host:
| host | Oct 14–18 | Oct 19–23 | Oct 23 only |
|---|---|---|---|
| www.teciplast.com.br | 201 | 3357 | 2361 |
| www.todolivro.com.br | 28 | 512 | 394 |
| www.alphabeto.com | 32 | 411 | 300 |
| www.taiff.com.br | 10 | 185 | 141 |
| www.happybooks.com.br | 3 | 163 | 132 |
| www.lojainterdental.com.br | 30 | 145 | 114 |
| www.palacio.com.br | 2 | 121 | 101 |
The stalls are synchronized across these unrelated sites, some of which run on different clusters. They appear in the same hours: Oct 20 14–15h, Oct 21 15–16h, Oct 22 13–14h, then continuously from Oct 23 06h UTC. On most of them the stalls are on PDPs, where reviews are loaded. Teciplast is hit hardest because its home has 16 shelves with reviews: true (~500 reviews calls per uncached render).
What I ruled out
DNS (p99 246ms over 300 fresh connects from the pod), TCP (no stuck sockets / retransmits during a stall), the ingress and Cloudflare (reproduced against the pod directly), and VTEX search (products.data was a cache hit).
What I could not pin down
- I couldn't capture what VTEX actually returns on the stalled call (status/body), because the error is swallowed.
- A standalone Deno process in the same pod, firing up to 700 concurrent reviews/rating calls (including after idle gaps up to 90s), never stalled. Inside the running app it does.
- So there may be an additional connection-reuse factor in the long-lived process.
None of that changes the fix: whatever the cause, an unbounded call to a decorative endpoint shouldn't be able to hold a page render.
Why 3s
Measured from the pod, healthy reviews-and-ratings calls are p95 ≈ 1.2s at 700 concurrent and < 0.7s at normal concurrency. 3s leaves headroom for slow-but-healthy responses while cutting the worst case from ~55s to ~3s. It's a constant, not a prop; it can become configurable if a store needs it.
Test
vtex/loaders/product/extend_test.ts stubs both endpoints with promises that only settle when their signal aborts:
- With this change: passes in ~3s, products come back with an empty rating.
- Without it (stashed
extend.ts): the test never settles (Promise resolution is still pending but the event loop has already resolved).
deno fmt --check and deno lint pass on both files.
Validation status
- Unit test (above), and confirmed it fails on current code
- Real-environment validation: sites pick this up only after bumping
apps. Teciplast (apps 0.153.0) is being mitigated separately by removing the unused reviews extension: deco-sites/teciplast#244 - Not changed:
inventoryExthas the same no-timeout + swallow pattern (onvcs). Left out to keep this PR to what the evidence covers.
Summary by cubic
Bounds the VTEX reviews extension calls with a 3s timeout so a single stalled reviews-and-ratings call can't hold a page render for ~55s.
Bug Fixes
- Old code fired both reviews-and-ratings calls per product with no timeout and swallowed their errors, so one stalled call held the render ~55s and still returned 200.
- Each call now passes
AbortSignal.timeout(3_000); on timeout the product gets no rating, the same degradation as any other failure today. - Adds a unit test that stubs both endpoints to hang until aborted, verifying products resolve with an empty rating within the timeout.