feat(plugins): apply plugin upstream headers to provider calls

FE
2d ago
+922-44

What

Exchange.Headers.Upstream is documented as adding headers to the provider
call, but the host only logged the field. Prompt-phase plugins therefore
cannot set provider-bound headers such as an edge proxy token, or the
per-tenant metadata an observability proxy in front of the provider reads.

This change keeps the headers a plugin writes to Exchange.Headers.Upstream
in the request state, carries them on the request context, and merges them
into every outgoing provider request in llmclient (all Do* paths share
buildRequest). Header names are canonicalized when the state merges them, so
x-foo and X-Foo no longer race on map iteration order.

Where the headers land

  • Every provider attempt of the request, failover to another provider
    included: each attempt is built from the same request context.
  • Token counting and compaction (/v1/responses/input_tokens,
    /v1/responses/compact): both run the prompt chain, so a prompt-phase
    plugin reaches them too. The handler now attaches the captured headers and
    re-reads the context, which the provider call was reading before they were
    set.
  • Not on inference the gateway issues for its own work. A guardrail's LLM
    judge calls the provider through the plugin Host on the request context,
    so without a gate it would carry the client's headers — a
    Helicone-User-Id would bill the judge call to that user. The merge returns
    the request headers untouched unless core.GetRequestOrigin(ctx) is
    RequestOriginExternal, the same signal Idempotency-Key forwarding
    already uses.
func (p *tenantHint) OnPrompt(_ context.Context, x *pluginapi.Exchange) (pluginapi.Decision, error) {
	x.Headers.Upstream = http.Header{
		"Helicone-Property-Tenant":  {x.Meta.UserPath},
		"Helicone-Property-Request": {x.Meta.RequestID},
	}
	return pluginapi.Allow(), nil
}

What a plugin cannot set

Three deny lists, so credentials, framing and provider feature negotiation stay
out of plugin control:

  • Credentials: core.IsCredentialHeader, the one shared list
    (Authorization, Proxy-Authorization, Cookie, Set-Cookie, x-api-key,
    api-key for Azure, x-goog-api-key, x-anthropic-api-key, xi-api-key,
    x-auth-token, x-access-token, x-gomodel-key).
  • Transport and framing: Host, Content-Length, Transfer-Encoding,
    Connection, Te, Trailer, Upgrade, Expect.
  • Body encoding: Content-Type, Accept-Encoding — setting the latter
    turns off Go's automatic gzip decompression, and the gateway would get bytes
    it cannot parse.
  • Provider feature negotiation: anthropic-beta is refused on its own. The
    Anthropic header hook adds oauth-2025-04-20 for subscription credentials,
    and the merge runs after that hook, so a plugin value would drop the flag and
    Anthropic would reject the request. The provider sets its own headers first
    and the deny lists — not the order of operations — are what keep its
    credentials: for every other name the plugin value wins, deliberately, since
    that is what the field is for.

Behaviour changes for existing installations

  • Upgrading starts sending upstream_set headers. Releases documented it
    as "recorded but not forwarded", so a deployment that already filled it in
    had those values stored and dropped. They now reach the provider, or the
    proxy in front of it, for the first time. Check the upstream accepts them
    before rolling the upgrade out.
  • Plugins now see more credentials redacted inbound. The request state
    redacted with its own short list (Authorization, X-Api-Key,
    Proxy-Authorization, Cookie), so api-key, x-goog-api-key,
    x-anthropic-api-key, xi-api-key, x-auth-token, x-access-token and
    x-gomodel-key reached a plugin in clear. It now calls the shared
    core.IsCredentialHeader and they arrive as [redacted]. This also fixes
    audit redaction and tag-source rejection for those names, and it is visible
    to any plugin that reads Headers.Request.
  • header_edit validation rejects three more names. It shares the same
    list, so a config that sets or removes xi-api-key, x-access-token or
    x-gomodel-key now fails at load with "carries credentials" instead of
    being accepted.
  • Both changes are in docs/advanced/plugins.mdx and
    docs/advanced/guardrails.mdx.

Scope

This PR is the upstream-header merge only. It was split at
@SantiagoDePolonia's request: the vLLM discovery commit moved to
https://github.com/ENTERPILOT/GoModel/pull/1245, and the Gemini
EDGE_PROXY_AUTH_TOKEN commit was dropped — a fixed proxy secret is provider
config, and providers.<name>.headers with secret references is a separate
change (https://github.com/ENTERPILOT/GoModel/issues/1290).

The branch sits on 7431bd8c and is 45 commits behind upstream/main; a
rebase is separate work, not part of this PR.

Verification

  • go build ./..., go vet ./... and golangci-lint run on the touched
    packages: green.
  • go test -race ./internal/core/ ./internal/plugins/... ./internal/llmclient/ ./internal/providers/anthropic/ and go test ./internal/server/: green.
    CI runs the full matrix with provider keys unset.
  • New tests: header application on GET/POST/DELETE, the credential and
    transport deny lists including an Azure api-key set by the header setter
    and a plugin Host asserted on the header map, plugin-over-request
    precedence, an end-to-end Do, canonicalization across exchanges, the
    origin gate (a plugin-origin call carries no plugin header), and a handler
    test that drives /v1/responses/input_tokens and /v1/responses/compact.
  • Each fix's test fails without it (verified by reverting the call in place).

This change was AI-assisted.