Replace publisher URLs with doi.org equivalents (313)

Merged
LU
6mo ago
+237-237

Summary

Many references in `content/` pointed at publisher pages (SAGE, ScienceDirect, Wiley, Springer, Royal Society, OUP, JSTOR, psycnet, etc.) instead of `https://doi.org/{DOI}\`. Publishers bot-block automated checks, so those links were producing unhelpful "broken" reports in the link checker even though the underlying papers are fine.

This PR converts 313 publisher URLs to their `doi.org` form, in two passes. Every replacement DOI was verified against the DOI Handle API before being applied — nothing was changed without confirming the new link resolves.

Pass 1 — DOI visible in URL (207)

Regex-extract from the URL itself, then verify.

countsource pattern
161`sciencedirect.com/science/article/[abs/]?pii/` (PII → DOI via Crossref `alternative-id` filter)
33`psycnet.apa.org/doiLanding?doi=...` (DOI lifted from query string)
6`royalsocietypublishing.org/doi/...`
3`link.springer.com/article|chapter|...`
2`journals.sagepub.com/doi/...`
1`onlinelibrary.wiley.com/doi/...`
1`tandfonline.com/doi/...`

Pass 2 — DOI looked up from URL ID (106)

Extract the publisher's record/article/stable ID, look it up via Crossref's `alternative-id` filter, fall back to OpenAlex bibliographic search for JSTOR/OUP. Verify via Handle API.

countsource pattern
57`psycnet.apa.org/{record,buy}/` (Crossref alt-id)
22`sciencedirect.com/.../pii/` — prior rate-limit misses retried
14`academic.oup.com//article-abstract/.../` (mostly OpenAlex)
10`jstor.org/stable/` (OpenAlex)
3`psycnet.apa.org/fulltext/.html`

Not addressed (~72 URLs left)

  • ~27 psycnet record/fulltext that neither Crossref nor OpenAlex indexes (likely book chapters, dissertations, or non-journal records).
  • ~23 ScienceDirect PIIs that neither service recognises (conference proceedings, supplements).
  • 7 JSTOR stable IDs that don't map to `10.2307/` (older content predating DOI deposit).
  • 7 `journals.lww.com/...` (lww IDs aren't indexed in Crossref or OpenAlex).
  • A handful of others.

These need per-URL human verification — likely "the original article never had a DOI deposited" rather than a fixable typo.

Test plan

  • Spot-check a couple of converted DOIs in the rendered site (do they go to the right paper?)
  • Re-run the Link Checker workflow after merging both this PR and #757; the "Publisher URLs that should use doi.org" section should be ~313 entries shorter.

🤖 Generated with Claude Code