feat(apify): persist TikTok/X post rows like Instagram (chat#1840)

Merged
SW
4mo ago
+246-10

Item 2 of recoupable/chat#1840. Independent of api#752 (item 1) — no ordering constraint between the two. Contract note: docs#262 (merge docs first). No database dependency.

Why

api#740 made TikTok/X scrapes persist follower counts, but post rows still only persisted for Instagram — handleInstagramProfileScraperResults was the sole caller of upsertPosts/upsertSocialPosts. Succeeded TikTok/X runs left GET /api/artists/{id}/posts empty (issue evidence: runs 9AYX8xyaHWyHtnGtC / bx3asRqfbNnkKgogG SUCCEEDED with real items, zero post rows landed). This is the half of item 2 that #740 didn't cover.

What

  • lib/apify/persistPostsForSocial.ts — the posts half of the Instagram flow (upsertPosts → getPosts → link via upsertSocialPosts), extracted as a shared helper (SRP: one export). No-ops on empty rows; skips linking when the social row isn't found. Called after upsertSocials so first-seen profiles link correctly.
  • TikTok handler: post_url ← webVideoUrl, updated_at ← createTimeISO — fields verified on real run 9AYX8xyaHWyHtnGtC.
  • X handler: post_url ← tweet url (display casing kept — posts key on exact URL), updated_at ← createdAt converted from Twitter's legacy format (Thu Jul 02 17:21:21 +0000 2026) to ISO; invalid dates fall back to the column default. Verified on real run bx3asRqfbNnkKgogG.
  • Instagram handler untouched (already persists; refactoring it onto the helper is a separate cleanup).

Tests (TDD, RED→GREEN)

  • New helper suite: persist+link happy path (exact upsertSocialPosts rows), empty-rows no-op, missing-social skip.
  • Per-platform handler tests extended with the real-run post fields; empty-dataset no-op now also asserts no persistence call.
  • Full lib/apify + lib/supabase suites: 238 passed; tsc --noEmit no new errors vs main; eslint clean.

Done-when (issue) — preview verification to follow as a comment

  • POST /api/socials/{id}/scrape on a TikTok social and an X social → within one webhook cycle, GET /api/artists/{id}/posts returns the scraped posts — verified live 2026-07-03 (49 → 69 posts: +10 tiktok, +10 x), results table in comments

🤖 Generated with Claude Code

Summary by cubic

Persist TikTok and X post rows and link them to socials, matching the Instagram flow, so artist posts now populate for these platforms. Timestamps are normalized to ISO and bad dates no longer break upserts. Implements item 2 of chat#1840.

  • New Features

    • Added persistPostsForSocial to upsert posts and link to the matching social; no-op on empty, skip link if social missing.
    • TikTok: webVideoUrl -> post_url; createTimeISO -> updated_at via toIsoDate (unparseable dates are omitted so the column default applies).
    • X: url -> post_url; legacy createdAt -> ISO updated_at via toIsoDate.
    • GET /api/artists/{id}/posts now returns TikTok/X posts after a scrape.

  • Dependencies

    • Docs contract in docs#262 (merge first). Independent of api#752.

Summary by CodeRabbit

  • New Features

    • Saved scraped social posts alongside profile data for TikTok and Twitter.
    • Added support for linking posts to their matching social profile record.
    • Captured post URLs and timestamps during import, including date normalization for older Twitter data.

  • Bug Fixes

    • Improved handling when no posts are available by returning early with empty results.
    • Skips post-to-profile linking when no matching social profile is found.