fix(find-tools): validate urls at runtime instead of publishing a schema pattern

NI
3d ago
+51-14

Why

URL-scoped discovery (firecrawl_find_tools with urls) doesn't work from ChatGPT. The urls items publish pattern: "^https?:\/\/" in the tool's input schema. ChatGPT's connector validates arguments against that pattern before calling the server, and it rejects ordinary https:// URLs. Our own zod check accepts the same URLs, so the requests never reach the server.

Evidence from Alexandria feedback (2026-09-22 to 10-09, aggregated): 18 teams, 31 reports. 27 of the reports come from mcp-ua-openai-mcp (3.25.4 through 3.26.0) and the rest from Codex/OpenCode. The agents describe it the same way, for example "URLs selector rejected valid HTTPS addresses" or "rejected by connector validation despite matching its documented pattern". Most then said "coverage for this website could not be assessed" and scraped the site themselves.

This matters beyond those 18 teams. Over the same window, 44 ChatGPT teams rated a site partial or bad after semantic-only discovery found nothing, on hosts where URL discovery does return a provider (x.com, LinkedIn, Booking, eBay, Amazon, Spotify, ...). For those agents, URL-scoped discovery is the fallback, and this bug blocked it.

Change

  • urls items are now { type: "string" } with a description. They no longer carry pattern or format. A zod refine (not serialised to JSON Schema) still rejects anything that new URL() doesn't parse as http:/https:, so bare domains, ftp: and javascript: never reach the API. An uppercase scheme (HTTPS://...) used to fail the pattern and now passes.
  • urls gets a description telling the agent to pass a page URL rather than only the domain. Bare hosts such as https://podcasts.apple.com match nothing, while a show URL matches Particle.
  • Regenerated contract snapshots. The only change is the firecrawl_find_tools urls node, and its definition hash in every profile.
  • New test: the published schema has no pattern/format on urls, HTTPS:// is forwarded, and invalid URLs are rejected without calling the API.

This is narrower than the stale #396, which targets src/index.ts, removed in the server split, and doesn't touch firecrawl_find_tools.

pnpm exec tsc --noEmit, pnpm lint and pnpm test (192 tests) pass locally.

Open in Capy

Summary by cubic

Fixes firecrawl_find_tools URL-scoped discovery, which ChatGPT rejected because the urls items published a pattern: "^https?:\/\/" in the input schema. The server's zod check accepted the same URLs, so requests never reached the API.

  • URL validation now happens at runtime, so the published schema no longer carries pattern or format on urls items. The runtime check requires an explicit http(s):// prefix, rejects bare domains and malformed schemes, and now accepts uppercase schemes like HTTPS://.
  • Adds descriptions telling agents to pass a page URL instead of a bare domain.
  • Adds coverage asserting the published schema has no pattern/format, uppercase schemes are forwarded, and invalid URLs are rejected without calling the API.
  • Regenerates contract snapshots after merging main; only the firecrawl_find_tools urls node and its definition hash change.