PR #4161 - ReviewStage/luke

P16 of "Bringing Codex or Claude Code Max plans into Luke". Gate A2 approved by Charles on the CI evidence, without the live run.

Live fixtures deferred: the live ChatGPT-plan run of pnpm probe:codex was skipped for now and moves to the end of the Codex track, before the cohort is turned on. Until then, tools/codex-probe/fixtures/live/report.json stays a placeholder and no fixtures/live/app-server.jsonl is committed. Every claim below marked as needing the live run (resume after a sandbox stop/resume, the real model waiting on CI, plan usage, the scans inside a real sandbox) is unverified until that run. P20 and P22 build on the hermetic recording in fixtures/hermetic/.

What this adds

tools/codex-probe is the probe for P16's two checks. Each step is a single command, run from the root of a vercel linked checkout:

pnpm probe:codex signs in through the browser, keeps the credentials in memory, opens its own Vercel Sandbox, runs every check, prints pass/fail per check, and writes a sanitized recording to tools/codex-probe/fixtures/live/. It refuses to write that recording if the leak guard finds anything secret-shaped in it.
pnpm probe:codex:sign-in signs in only. It writes the credentials to a 0600 file in a fresh 0700 temp directory and prints the path, never the values. LUKE_PROBE_CODEX_CREDENTIALS=<path> pnpm probe:codex then reads that file and refuses it if anyone but the owner can read it.
Optional settings: --repository owner/name clones a throwaway repo. The GitHub token for it comes from the developer's own gh auth token, is set at the sandbox firewall as today, and never enters the sandbox. --model picks the model and --ci-seconds sets how long the stand-in CI watch runs (default 420 s).

Sign-in. This is OpenAI's open-source flow, written against the docs:

loopback callback on 127.0.0.1:1455/auth/callback, falling back to any free port
PKCE S256, fresh state and nonce per attempt
client_id=dynamic_agent_client, agent_name_hint=Luke, a fresh ext_agent_host_id (urn:uuid:…)
the code is exchanged under the issued oaiapp_… client id
the chatgpt.tokens.use.direct scope is required
the ID token's RS256 signature, issuer, audience, nonce and expiry are checked against OpenAI's published keys
endpoints come from OIDC discovery

OpenAI's devkit (@siwc/local) is not used: it ships under a noncommercial licence and isn't on npm.

Sandbox. The sandbox is built from today's image: the probe calls the same prepareImage every agent snapshot uses. That function is now exported from agent-runtime/sandbox.ts; that export is the only change there, and PREPARE_SCRIPT and the image digest are untouched. On top of that image the probe:

installs @openai/codex@0.163.0 root-owned under /opt/luke-codex;
creates agent with useradd and a 0700 home. The SDK's createUser isn't used because its homes are readable by the default user's group;
launches codex app-server --listen stdio:// through the Sandbox API with sudo: true, dropping to agent with setpriv --reuid=agent --regid=agent --init-groups --inh-caps=-all --ambient-caps=-all --bounding-set=-all. ACCESS_TOKEN travels in the API's command environment, never in argv. asUser() isn't used because it puts env into argv as env K=V.

It uses @vercel/sandbox 3.8.0 (for stdin: true and writeStdin). The 3.3.0 that eve vendors has no stdin.

Codex pin and config (for P17 and P20)

codex app-server --listen stdio://
  -c model_provider="openai_chatgpt_plan"
  -c model_providers.openai_chatgpt_plan.{name="ChatGPT plan", base_url="https://api.openai.com/v1", env_key="ACCESS_TOKEN", wire_api="responses", requires_openai_auth=false, supports_websockets=false}
  -c shell_environment_policy.inherit="core"  -c shell_environment_policy.exclude=["ACCESS_TOKEN"]
  -c mcp_servers.luke_probe.{command="node", args=[…], default_tools_approval_mode="approve"}
  -c sandbox_workspace_write.network_access=true
thread/start { approvalPolicy: "never", sandbox: "workspace-write", cwd }
initialize { clientInfo: { name: "Luke", title: "Luke", version } }

default_tools_approval_mode="approve" is required. With approval_policy="never", Codex 0.163.0 otherwise fails every MCP call with "MCP tool call requires approval, but approval policy is never". P20's config needs this for mcp_servers.luke.
clientInfo.name is Luke, not the plan's luke. OpenAI's guide says it should match agent_name_hint. P20 should use whichever Gate A2 settles.
CODEX_HOME must exist before app-server starts, or Codex refuses to start. P17 should create it, owned by agent.
MCP tools are deferred behind Codex's tool_search. The model finds luke_probe tools through it, and calls then arrive as function_call with namespace: "mcp__luke_probe".

Where Codex keeps threads, and resume

Codex keeps each thread under CODEX_HOME as sessions/YYYY/MM/DD/rollout-<timestamp>-<threadId>.jsonl, plus state_5.sqlite (and logs_2, goals_1, memories_1, queue_1). After a fresh app-server process, thread/resume { threadId } reloads the thread with its turns, and the next turn continues it. That is shown hermetically (2 turns reloaded, the answer quotes turn 1). The live probe checks the same thing across sandbox.stop() and Sandbox.get({ resume: true }) with a refreshed token (plan.resumes-after-restart).

Waiting on CI natively

Codex 0.163.0's shell is unified exec: exec_command returns a session id while the command runs, and write_stdin with empty chars polls it, waiting up to 300 s per poll. In the hermetic run the watch's commandExecution item completes with its last line, and the turn's reply quotes it. The live check (plan.waits-on-ci) uses a 420 s stand-in for gh pr checks --watch, longer than one poll, and records how long it took and the item's source. If the live run passes, Luke's sleep doesn't need serving to Codex.

Token isolation: findings so far (local evidence, re-checked live per seat)

The plan's "no sudo" isn't enough. In a Vercel Sandbox, the default user's processes hold the full capability set without sudo (CapEff and CapAmb = 000001ffffffffff). As that user, with no sudo, I read /etc/shadow and read the environment of a process running as another unprivileged user. Running the shell under setpriv --inh-caps=-all --ambient-caps=-all --bounding-set=-all blocks both, and sudo too. The probe reports the default shell (token.not-from-terminal, expected to fail today) and the shell with capabilities dropped (token.not-from-terminal-without-capabilities). I passed this to the orchestrator for P00 and P03.
shell_environment_policy hides the token from a command's own env, but not from /proc. Codex doesn't make app-server non-dumpable, so a command running as the same uid can read /proc/<app-server>/environ. Under danger-full-access it did: the token showed up in 4 processes. Under workspace-write, Codex's bwrap sandbox gives commands their own PID namespace, and the token wasn't reachable. The catch: bwrap only runs once the launch drops capabilities. With them, it fails with "bwrap: Unexpected capabilities but not setuid". That is why the launch empties every capability set.
The MCP server doesn't get ACCESS_TOKEN (access_token_in_env=false).

The live probe also checks:

the disk, scanned as root for the token's SHA-256;
launch paths the default user could rewrite;
that the default user's sudo is gone after the sudoers lines are deleted;
whether the API's sudo: true still runs as root after that.

Fixtures

fixtures/hermetic/app-server.jsonl: 99 messages, both directions, from the real codex 0.163.0 app-server. It was driven through the probe's own turns (apply_patch edit, scanner command, tool_search + MCP echo, a CI watch waited out under a turn/steer, an interrupted turn, then a restart and thread/resume). The model was scripted by a local Responses server instead of the ChatGPT plan, and paths were moved to the sandbox's. It was sanitized by the probe's own sanitizer, and P20 and P22 can use it now.
fixtures/live/: a placeholder until the live run writes app-server.jsonl and report.json there.

Checks

pnpm exec vitest run in tools/codex-probe: 37 tests. They cover the RFC 7636 vector, the authorize URL, callback refusals, code exchange and refresh bodies, the ID token checks, the loopback against a real local listener, the credentials file modes, the launch spec (token never in argv), the sanitizer and leak guard, and judge over the hermetic recording plus failing seats.
pnpm check:affected: tests, lint, knip and discarded-effect pass. @luke/web's typecheck ran out of memory in my sandbox at the default heap (unrelated to this change); it passes alone with a larger heap, and CI runs it.
Repo plumbing: the root vitest.config.ts lists the new workspace, and scripts/lib/task-names.mjs allows planType (OpenAI's rate-limit field) in the probe's files.

Still to do (Gate A2)

The developer runs pnpm probe:codex once on their Mac with a ChatGPT Plus or Pro account, then commits the sanitized fixtures/live/ output here. plan.usage-on-chatgpt-plan is a manual check: the turns should show up on chatgpt.com/settings/usage.

Made with Luke