fix(harness): clamp full-page screenshots below Anthropic's 8000px limit

Merged
PE
3mo ago
+156-17

Summary

A full-page take_screenshot of a long page comes back well over 8000px tall. Anthropic rejects any image over 8000px on either side — and because the screenshot is inlined as base64 into the message history, every subsequent turn replays it and gets the same 400. The thread is bricked permanently: "try again" can't work, and neither can the board's stall-nudge.

Real incident: thrd_uY3x6c0d6aMC1cimbF12E (Super Agent task on decocms.com, org deco-studio) took a full-page shot of the home page, then failed three turns in a row with

[Anthropic] messages.4.content.1.image.source.base64.data:
At least one of the image dimensions exceed max allowed size: 8000 pixels

…at messages.4, then messages.6 as the index shifted. Thread ended failed, no PR. packages/harness/src/decopilot/built-in-tools/portable-media-tools.ts passed fullPage straight through to Browserless with no dimension bound.

Change

  • Read the height off the returned JPEG's SOF marker (jpegHeight).
  • Only when a full-page capture exceeds 7000px, re-shoot clipped to the top 7000px. Puppeteer treats clip and fullPage as mutually exclusive, so the clamped call swaps one for the other.
  • If the clipped retry still isn't within the limit, fail the tool call instead of emitting the image. A failed tool call is recoverable; a poisoned history is not.

Screenshots that already fit are untouched — same single request, same bytes. No unconditional clipping, so short pages don't get a white tail (which would waste most of the model's ~1568px downsample budget).

Testing

  • bun test packages/harness/src/decopilot/built-in-tools/ — 87 pass, incl. 7 new covering the option builder (clip replaces fullPage, only on the clamped retry) and the JPEG parser (normal height, >8000 height, non-JPEG, truncated).
  • bun run --cwd=packages/harness check clean, bun run lint clean, bunx knip clean, bun run fmt applied.

⚠️ Not verified against live Browserless — no BROWSERLESS_TOKEN locally. The retry path assumes chrome.browserless.io (v1) honors options.clip beyond the viewport. If it doesn't, the second height check catches it and the tool errors out cleanly instead of poisoning the thread — so the failure mode is bounded either way, but worth one manual full-page screenshot of a long page after deploy.

Not in scope

That thread's first failure was a separate bug — Error: producer produced no output before timeout at 18:09, with the sandbox coming up at 18:13. Untouched here.

Summary by cubic

Clamp full-page screenshots in the harness to 7000px to stay under Anthropic’s 8000px image limit and prevent threads from being bricked by oversized images. Oversized captures are retried with a clipped top region; if still too tall, the tool call fails instead of emitting the image.

  • Bug Fixes
    • Detects tall full-page screenshots and re-shoots with clip to 7000px; otherwise returns a recoverable tool error.
    • Adds jpegHeight to read height from JPEG SOF and buildScreenshotOptions to swap fullPage for clip on retry.
    • Leaves normal screenshots unchanged; uses jpeg at quality 80 with the default viewport. Tests added for option building and JPEG parsing in packages/harness/src/decopilot/built-in-tools/portable-media-tools.test.ts.