PR #1179 - AltimateAI/altimate-code

Issue for this PR

Type of change

Bug fix
New feature
Refactor / code improvement
Documentation

What does this PR do?

What was broken. The ChatGPT-subscription (OAuth) model filter in plugin/codex.ts was assembled from the models.dev catalog instead of from what the Codex backend actually serves, so it was wrong in both directions at once.

It offered three models the backend refuses. gpt-5.2 and gpt-5.6 were both in OAUTH_ALLOWED_MODELS, and gpt-5.3-codex slipped in via the modelId.includes("codex") auto-allow. Picking any of them died mid-request with an opaque 400 {"detail":"The '<id>' model is not supported when using Codex with a ChatGPT account."}.

It also hid three models the backend does serve: gpt-5.6-sol, gpt-5.6-luna, and gpt-5.6-terra. Those are the current flagship subscription models. They had been excluded deliberately, on the untested assumption that the sol/luna/terra variants were API-tier-only, which meant subscribers could not select models they already pay for. This supersedes the narrower framing of #1132, which added plain gpt-5.6 on catalog presence alone — the backend rejects that id, and only its variants are served.

What I changed. I probed every gpt-5.x id in the catalog against POST https://chatgpt.com/backend-api/codex/responses on a ChatGPT Pro credential and rebuilt the allowlist from the responses:

Accepted (HTTP 200)	Rejected (HTTP 400)
`gpt-5.3-codex-spark`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.5`, `gpt-5.6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`	`gpt-5`, `gpt-5.1`, `gpt-5.2`, `gpt-5.2-pro`, `gpt-5.3-chat-latest`, `gpt-5.3-codex`, `gpt-5.4-nano`, `gpt-5.4-pro`, `gpt-5.5-pro`, `gpt-5.6`

I also removed the includes("codex") substring auto-allow rather than patching it. It cannot express the real policy — the tier accepts gpt-5.3-codex-spark but rejects gpt-5.3-codex, and accepts gpt-5.6-sol but rejects gpt-5.6. There is no derivable rule in that table, so membership is exact-match only, and the comments say so along with how to add an id later (probe it first).

On client identity. We identify as ourselves throughout. The wired plugin already sends originator: "altimate" and a User-Agent naming altimate-code with its Installation.VERSION via the chat.headers hook, and all verification above was done under that identity. Nothing here impersonates a first-party OpenAI client.

Why this list is static, and not discovered. The obvious objection to any hand-maintained allowlist is that it drifts the moment OpenAI ships a model or changes entitlements. So I went looking for a way to derive the list instead of asserting it, and there is an authoritative per-account endpoint:

GET https://chatgpt.com/backend-api/codex/models?client_version=<v>

It works, and it accepts our own identity — originator: altimate plus our own User-Agent, no impersonation required. It returns rich per-model metadata (slug, visibility, minimal_client_version, supported_in_api, context_window, …), and its visibility: "list" slugs are exactly the seven ids in the corrected OAUTH_ALLOWED_MODELS:

gpt-5.6-sol          vis=list  minver=0.144.0
gpt-5.6-terra        vis=list  minver=0.144.0
gpt-5.6-luna         vis=list  minver=0.144.0
gpt-reserve          vis=hide  minver=0.144.0
gpt-5.5              vis=list  minver=0.124.0
gpt-5.4              vis=list  minver=0.98.0
gpt-5.4-mini         vis=list  minver=0.98.0
gpt-5.3-codex-spark  vis=list  minver=0.100.0
codex-auto-review    vis=hide  minver=0.98.0

That is a useful result on its own: an independent source, reached a different way, confirms the table this PR builds by request-level probing. The two hidden entries (gpt-reserve, codex-auto-review) are internal and are not in the models.dev catalog anyway.

But it cannot be used as a runtime source of truth, so I did not wire it up. client_version is mandatory, and the backend gates each model behind its minimal_client_version — today 0.98.0 through 0.144.0. Those are Codex CLI release numbers, a numbering line we are not on, and our own versions sit below all of them. Worse, the gate fails silently:

`client_version` sent	Result
(omitted)	`400 {"detail":"Invalid client_version format"}`
`local` (our dev builds)	`400 {"detail":"Invalid client_version format"}`
`0.9.7` (our published version)	`200 {"models":[]}`
`0.144.0`	`200` — all 9 models

So discovery returns nothing usable unless we send a Codex CLI version we are not. That is impersonating the first-party client in the one field the backend actually gates on, and a silent empty list is a worse failure mode than a stale list: it would hide every model with no error to debug. I also checked whether the entitlement data is reachable another way — /backend-api/codex/{user_info,account,entitlements,config} all 404, and /backend-api/models returns the ChatGPT web app namespace (gpt-5-6, gpt-5.6-sol-wm), not Codex API slugs.

Conclusion: derive-don't-assert is the right instinct here, and it is blocked on a version handshake we cannot honestly make. Rather than lose the investigation, I committed it as a comment on OAUTH_ALLOWED_MODELS recording the endpoint, the corroboration, the blocker, and the drop-in shape (fetch → keep visibility === "list" → fall back to the static set when empty or failed) should a legitimate client_version ever exist. That is the only code change in the second commit; there is no behavior change.

The honest alternative available today, if the maintenance burden of this list becomes real, is lazy validation: keep the picker permissive and, on the specific 400 … is not supported when using Codex with a ChatGPT account, surface an error naming the models that do work. That trades a fast-fail at selection for never hiding a working model. I have not implemented it here — it is a different design with its own tradeoff, and this PR is already a behavior change to the picker.

What this is not. The failure initially looked like a transport problem, since the real Codex CLI negotiates a websocket (openai-beta: responses_websockets=...) while we POST over plain HTTP. That turned out to be a dead end. I probed the endpoint with escalating header sets — originator + User-Agent, then openai-beta, version, x-codex-beta-features, session-id, x-client-request-id — and every combination returned byte-identical results. Plain HTTP reaches every working model. The websocket path in the unwired plugin/openai/ refactor is a streaming optimization gated behind OPENCODE_EXPERIMENTAL_WEBSOCKETS, off by default even there, and it is not required for model access. So this PR changes no transport code and leaves the OAuth flow and token refresh untouched.

Two caveats worth stating plainly. This table is what one Pro account was served today; OpenAI can change the served set at any time, and I did not verify against a Plus (non-Pro) credential, so it is possible Plus is served a narrower set. And because the discovery endpoint is per-account, its agreement with the probed table confirms the list is right for this account — it is corroboration, not proof that every subscription tier sees the same seven.

How did you verify your code works?

Live, end to end, through the real CLI on a ChatGPT Pro credential.

A newly-allowlisted model returns real output:

$ bun run --conditions=browser packages/opencode/src/index.ts run \
    --format json --yolo --max-turns 1 --model openai/gpt-5.6-sol "Reply with exactly OK"
{"type":"text",...,"text":"OK",...}
{"type":"step_finish",...,"reason":"stop","tokens":{"total":55675,...}}

And a second, previously-working model still does:

$ ... --model openai/gpt-5.4 "Reply with exactly OK"
{"type":"text",...,"text":"OK",...}

A removed model now fails fast at selection with a useful pointer, instead of a 400 halfway through a request:

$ ... --model openai/gpt-5.6 "hi"
Error: Model not found: openai/gpt-5.6
Did you mean: gpt-5.6-sol, gpt-5.6-luna, gpt-5.3-codex-spark, gpt-5.4-mini, gpt-5.5

Before the change, that same gpt-5.6 command reached the network and returned
400 {"detail":"The 'gpt-5.6' model is not supported when using Codex with a ChatGPT account."}.

Gates:

bun test test/plugin/codex-allowlist.test.ts test/plugin/codex.test.ts test/plugin/openai-ws.test.ts test/plugin/openai-rollout.test.ts → 53 pass, 1 skip, 0 fail
bun run typecheck in packages/opencode → clean
bun run script/upstream/analyze.ts --markers --base main --strict → All custom code in upstream-shared files is properly marked

The allowlist was also checked against the discovery endpoint described above, which is an independent source reached a different way. Its visibility: "list" slugs and the CLI's picker agree exactly:

$ bun run --conditions=browser packages/opencode/src/index.ts models | grep '^openai/'
openai/gpt-5.3-codex-spark
openai/gpt-5.4
openai/gpt-5.4-mini
openai/gpt-5.5
openai/gpt-5.6-luna
openai/gpt-5.6-sol
openai/gpt-5.6-terra

All gates were re-run after the second (comment-only) commit and are still green.

The rewritten codex-allowlist.test.ts encodes the truth table as two lists and asserts both directions, so a future edit that either hides a working model or admits a rejected one fails the suite. It also pins the allowlist to exactly the verified set, so speculative additions trip a test rather than reaching users. No test added here touches the network.

Screenshots / recordings

Not a UI change.

Checklist

I have tested my changes locally
I have not included unrelated changes in this PR

[!NOTE]
Medium Risk
Changes which OpenAI models OAuth subscribers see and can select; wrong entries affect UX but do not alter OAuth tokens or request auth paths.

Overview
Replaces the ChatGPT-subscription (OAuth) model filter in plugin/codex.ts with a probe-verified exact allowlist instead of catalog guesses and a "codex" substring auto-allow.

Picker behavior: Adds working ids (gpt-5.3-codex-spark, gpt-5.6-sol/luna/terra) and removes ones the backend rejects (gpt-5.2, plain gpt-5.6, and ids that only slipped in via includes("codex") like gpt-5.3-codex). Unsupported models should fail at selection rather than mid-request with HTTP 400.

Alias fix: New disallowedOAuthModelKeys judges each entry by upstream model.api.id (fallback to map key), so user config aliases of supported models are no longer dropped when the map key differs from the API id.

Tests: codex-allowlist.test.ts pins the allowlist to verified accepted/rejected sets and covers alias and cold-cache snapshot codex-id behavior.

^{Reviewed by Cursor Bugbot for commit a8508e2efab9110371e43db7fc8c3aea4e22bb4a. Bugbot is set up for automated code reviews on this repo. Configure here.}

Summary by cubic

Fixes the ChatGPT-subscription model allowlist in plugin/codex.ts so it matches what the Codex backend actually serves, and makes the filter match on the upstream api.id so config aliases of supported models survive. Users can now pick working flagship models like gpt-5.6-sol, unsupported ids like gpt-5.6 fail fast at selection instead of dying mid-request with a 400, and user-defined aliases of working models no longer vanish from the picker.

Notes

Every gpt-5.x id was probed against the live endpoint on a ChatGPT Pro credential; the verified accepted and rejected sets are documented in the code comments.
Removed the includes("codex") substring auto-allow; membership is exact-match only. That now drops stale codex ids from the bundled catalog snapshot (gpt-5.1-codex, etc.) that the old rule offered, and on a cold cache the new gpt-5.6 models stay hidden too — the snapshot has no such variants — until the catalog refreshes from models.dev.
The filter now matches model.api.id with a fallback to the map key, so aliases like fast-spark → gpt-5.3-codex-spark survive while aliases of rejected ids are still deleted.
Rewrote codex-allowlist.test.ts to pin the allowlist to the verified set and cover both filter behaviors, so speculative additions or either direction of breakage trips a test.
Supersedes #1132; the backend rejects plain gpt-5.6.
The list is a snapshot from one Pro account; Plus may be served a narrower set, and OpenAI can change it at any time.
Closes #1178.

Summary by CodeRabbit

Bug Fixes
- Improved ChatGPT subscription model access with an exact, verified allowlist.
- Preserved supported model aliases while filtering rejected or unverified variants.
- Added approved GPT-5.6 variants and GPT-5.3 Codex Spark.
- Prevented unsupported legacy and arbitrary Codex identifiers from being accepted.
- Clarified Pro-tier and static-discovery limitations for available models.
Tests
- Expanded coverage for accepted, rejected, aliased, legacy, and unsupported model identifiers.
- Verified unavailable models are removed while supported catalog models remain available.