feat(cli): let wrappers set the integration tag with FIRECRAWL_INTEGRATION
Merged
ER
15h ago+263+263+012345678901234567890123456789-13-13-01234567890123456789
Summary
Every command hardcodes integration: 'cli', so tools that wrap the CLI always show up as cli. The CLI now reads FIRECRAWL_INTEGRATION, the same variable firecrawl-mcp reads, and sends it as the X-Integration request header. The body (and query) integration stays cli.
- New
src/utils/integration.ts:integrationHeaders()returns{ 'X-Integration': <trimmed value> }when the variable is set. There is no underscore rule and no list of names in the CLI: the API attributes a recognized name and silently ignores anything else, so the tag can never make a request fail. The only values dropped client side are ones that cannot be sent as a header at all (control or non-ASCII characters), which would otherwise makefetchthrow. - The header goes on every API request the CLI makes:
- SDK client requests (scrape, map, crawl, agent, alexandria, browser, and the research, developer and gov GETs that go through the client's
http), viawithIntegrationHeader()ingetClient(); - the keyless path (
keylessRequest,keylessGet); - the raw
fetchcalls: feedback, search-feedback, interact, parse, monitor, credit-usage, status, doctor, list and terms.
- SDK client requests (scrape, map, crawl, agent, alexandria, browser, and the research, developer and gov GETs that go through the client's
- The SDK (4.50.0) has no option for custom default headers, so
withIntegrationHeader()adds it to the SDK client's axios defaults. The CLI already reaches into the samehttpmember for research and developer search. The smallest proper fix is aheaders?: Record<string, string>option onFirecrawlClientOptions, merged into the axios defaults inHttpClient; once an SDK release has it, the CLI can pass the header through that option instead. originand the keylessX-Originheader are unchanged.- README documents the variable next to the telemetry setting.
vitest.config.mjssetsFIRECRAWL_INTEGRATIONto empty for test runs, so a value exported in the shell doesn't change request assertions.- Version bump to 1.26.15.
The API side: a valid X-Integration header must beat a body or query value of cli on every endpoint the CLI calls. That rule lands in https://github.com/firecrawl/firecrawl/pull/5260; until it ships, requests keep being attributed to cli, and nothing fails either way.
Testing
pnpm type-check,pnpm format:check,pnpm buildpass;pnpm test: 768 passed, with and withoutFIRECRAWL_INTEGRATION=hermesexported.- New tests in
src/__tests__/utils/integration.test.tsrun against a local HTTP server:rtk,hermesand_rtkgo out asX-Integrationon SDK client POST and GET requests while the body keepsintegration: cli; a junk value (not a valid tag!) is sent and the request succeeds; a value that cannot be a header is dropped and the request still succeeds.feedback.test.tschecks the header on feedback and search feedback. - The built CLI against the production API:
firecrawl scrape https://example.comexits 0 withFIRECRAWL_INTEGRATIONset tohermes,_rtkandnot a valid tag!.
Merge order
Merge after https://github.com/firecrawl/cli/pull/337 (merged, 1.26.14) and once https://github.com/firecrawl/firecrawl/pull/5260 is deployed, so the README matches the API. Merging publishes 1.26.15.