feat(cli): let wrappers set the integration tag with FIRECRAWL_INTEGRATION

Merged
ER
15h ago
+263-13

Summary

Every command hardcodes integration: 'cli', so tools that wrap the CLI always show up as cli. The CLI now reads FIRECRAWL_INTEGRATION, the same variable firecrawl-mcp reads, and sends it as the X-Integration request header. The body (and query) integration stays cli.

  • New src/utils/integration.ts: integrationHeaders() returns { 'X-Integration': <trimmed value> } when the variable is set. There is no underscore rule and no list of names in the CLI: the API attributes a recognized name and silently ignores anything else, so the tag can never make a request fail. The only values dropped client side are ones that cannot be sent as a header at all (control or non-ASCII characters), which would otherwise make fetch throw.
  • The header goes on every API request the CLI makes:
    • SDK client requests (scrape, map, crawl, agent, alexandria, browser, and the research, developer and gov GETs that go through the client's http), via withIntegrationHeader() in getClient();
    • the keyless path (keylessRequest, keylessGet);
    • the raw fetch calls: feedback, search-feedback, interact, parse, monitor, credit-usage, status, doctor, list and terms.
  • The SDK (4.50.0) has no option for custom default headers, so withIntegrationHeader() adds it to the SDK client's axios defaults. The CLI already reaches into the same http member for research and developer search. The smallest proper fix is a headers?: Record<string, string> option on FirecrawlClientOptions, merged into the axios defaults in HttpClient; once an SDK release has it, the CLI can pass the header through that option instead.
  • origin and the keyless X-Origin header are unchanged.
  • README documents the variable next to the telemetry setting.
  • vitest.config.mjs sets FIRECRAWL_INTEGRATION to empty for test runs, so a value exported in the shell doesn't change request assertions.
  • Version bump to 1.26.15.

The API side: a valid X-Integration header must beat a body or query value of cli on every endpoint the CLI calls. That rule lands in https://github.com/firecrawl/firecrawl/pull/5260; until it ships, requests keep being attributed to cli, and nothing fails either way.

Testing

  • pnpm type-check, pnpm format:check, pnpm build pass; pnpm test: 768 passed, with and without FIRECRAWL_INTEGRATION=hermes exported.
  • New tests in src/__tests__/utils/integration.test.ts run against a local HTTP server: rtk, hermes and _rtk go out as X-Integration on SDK client POST and GET requests while the body keeps integration: cli; a junk value (not a valid tag!) is sent and the request succeeds; a value that cannot be a header is dropped and the request still succeeds. feedback.test.ts checks the header on feedback and search feedback.
  • The built CLI against the production API: firecrawl scrape https://example.com exits 0 with FIRECRAWL_INTEGRATION set to hermes, _rtk and not a valid tag!.

Merge order

Merge after https://github.com/firecrawl/cli/pull/337 (merged, 1.26.14) and once https://github.com/firecrawl/firecrawl/pull/5260 is deployed, so the README matches the API. Merging publishes 1.26.15.