feat(content-scraper): optional johnvc/linkedin-posts-api source for LinkedIn profiles

JO
20d ago
+236-21

Summary

Adds an optional linkedinActor setting to content-scraper. The default
stays harvestapi~linkedin-profile-posts. Setting it to johnvc fetches
the same weekly profile posts from the johnvc/linkedin-posts-api Apify
Actor and maps them into the existing LinkedInRawPost shape, so scoring,
dedup and storage are unchanged.

Disclosure: I build and maintain the johnvc Actor (Alpha OSINT).

Why this Actor

content-scraper already pulls a small number of recent posts per tracked
profile each week. This Actor returns public profile posts with no LinkedIn
login, mapped into the shape scoring and storage already use. A signed-out
person profile shows about 3 to 8 recent posts, which covers the current
maxPosts = 5.

Changes

  • server/types/env.ts: optional linkedinActor ("harvestapi" | "johnvc"),
    default "harvestapi".
  • server/lib/linkedin-scraper.ts: new fetch path calling the Actor with
    profileUrls: [profileUrl] and maxPostsPerProfile: maxPosts, plus a
    mapper to LinkedInRawPost (id, linkedinUrl, content, author name, URL and
    headline, likes, comments, postedAt.date).
  • server/tools/scraping.ts and cli.ts: pass the setting through so it
    takes effect.
  • README config note. CLI: LINKEDIN_ACTOR=johnvc.

Differences to know

  • Share counts are not available, so num_reposts is stored as 0.
  • No author avatar; author_profile_image is null.
  • A person profile returns the recent posts LinkedIn shows a signed-out
    visitor (usually 3 to 8). That covers the current maxPosts = 5.
  • Reposts on person profiles come back as posts with postType: "repost",
    not with a separate repostedBy, so they are attributed to the profile
    owner.

Testing

  • Offline mapper fixture test (bun test content-scraper/server/lib/linkedin-scraper.test.ts)
  • Pre-existing tsc errors in @decocms/runtime node_modules still present on a clean install; no new errors introduced in content-scraper sources
  • Live two-profile run not claimed in this PR

Made with Cursor

Summary by cubic

Adds an optional linkedinActor setting to content-scraper that selects which Apify Actor fetches LinkedIn profile posts. The default stays harvestapi~linkedin-profile-posts; setting it to johnvc uses the johnvc/linkedin-posts-api Actor and maps results into the existing LinkedInRawPost shape, leaving scoring, dedup, and storage unchanged. Disclosure: I build and maintain the johnvc Actor.

Notes

  • Configure via LINKEDIN_ACTOR=johnvc in the CLI or linkedinActor: "johnvc" in MCP state.
  • Share counts aren't available, so num_reposts is stored as 0.
  • Author avatars aren't returned, so author_profile_image is null.
  • Signed-out person profiles typically return 3 to 8 recent posts, which covers maxPosts = 5.
  • Reposts come back as postType: "repost" attributed to the profile owner rather than a separate repostedBy.