feat(apify): optional `posts` depth on social scrape endpoints
Summary
Implements recoupable/chat#1836. Contract: recoupable/docs#258 — merge order: docs → api (this PR).
Adds an optional posts parameter (integer, 1–100) to both scrape endpoints and threads it into the platform Apify actor inputs:
| Layer | Change |
|---|---|
POST /api/socials/{id}/scrape | ?posts=N query param, validated in validatePostSocialScrapeRequest (400 on 0/101/non-integer, parsed before auth) |
POST /api/artist/socials/scrape | optional posts body field on artistSocialsScrapeBodySchema |
scrapeProfileUrl / scrapeProfileUrlBatch | new optional posts arg forwarded to every platform ScrapeRunner |
Twitter (apidojo/twitter-scraper-lite) | maxItems: posts ?? 1 — last N tweets with engagement metrics incl. viewCount (impressions) |
YouTube (streamers/youtube-scraper) | maxResults: posts ?? 1, maxResultsShorts: posts ?? 0 — a requested depth includes Shorts, which the legacy snapshot excluded |
| Other platforms | accept and ignore the param (Instagram profile scrapes already return latestPosts) |
Backwards compatible / cost-neutral by default: omitting posts produces the exact same actor inputs as before (unit-tested), so existing profile-snapshot callers and their Apify billing are unchanged.
Tests (TDD, red → green)
All new behavior was written test-first and confirmed failing (20 failures across 8 files) before implementation:
startTwitterProfileScraping: default input unchanged;posts=20→maxItems: 20startYoutubeProfileScraping: default input unchanged (Shorts excluded);posts=20→maxResults/maxResultsShorts: 20scrapeProfileUrl+scrapeProfileUrlBatch:postsforwarded (andundefinedwhen omitted)validatePostSocialScrapeRequest:posts=20parses;0/101/abc/1.5→ 400 before authvalidateArtistSocialsScrapeBody+postArtistSocialsScrapeHandler: new suites (missing id → 400, posts pass-through, no-socials →[])
Full suite: 3745/3745 pass. Affected domains (lib/apify, lib/socials, lib/artist): 234/234. tsc --noEmit introduces no new errors (200 with this diff vs 203 pre-existing on base). ESLint clean on touched dirs.
Preview verification against the live deployment to follow as a PR comment.
🤖 Generated with Claude Code
Summary by cubic
Add optional posts depth to social scrape endpoints so callers can request up to N posts while keeping legacy single-item behavior by default (cost-neutral).
-
New Features
POST /api/socials/{id}/scrape: accepts?posts=1..100; invalid values return 400 before auth.POST /api/artist/socials/scrape: accepts optionalpostsbody field (1–100).- Threads
poststhroughscrapeProfileUrlandscrapeProfileUrlBatchinto platform scrapers. - Twitter (
apidojo/twitter-scraper-lite):maxItems = posts ?? 1. - YouTube (
streamers/youtube-scraper):maxResults = posts ?? 1,maxResultsShorts = posts ?? 0; when provided, Shorts are included. - Other platforms accept and ignore
posts(no behavior change).
-
Migration
- No changes required for existing callers.
- To request depth: use
?posts=Non/api/socials/{id}/scrapeor{ posts: N }on/api/artist/socials/scrape(1–100).