#4 feat(llm): loopback OpenAI endpoint for validation checks

Fourth of five stacked PRs for the worker validation gates DEP (ai-dynamo/dynamo#15897). Based on #3. The code doesn't depend on #3, but the stack is kept linear so #5's end-to-end test covers the card hold and this endpoint together.

Why

A gated worker is unreachable by design: its card is held, so no frontend routes to it. Its checks still need to send requests that go through the frontend's OpenAI preprocessing, tool-call and reasoning parsers.

What

• New lib/llm/src/entrypoint/input/local_openai.rs. When DYN_WORKER_VALIDATION_GATES is set, LocalModel::attach serves the worker's first base-model card that takes tokens and has a chat or completions surface. It listens on 127.0.0.1:${DYN_WORKER_VALIDATION_LOCAL_PORT}, where 0 (the default) means an ephemeral port.
  • It uses the same pipeline as the in-process token HTTP input, via serve_in_process_tokens, extracted from http.rs with no behaviour change.
  • On each request it looks up the engine in the LocalEndpointRegistry through RegistryEngine. This is a serde round trip, the inverse of JsonProbeAdapter, so the parsers see exactly what they'd see over the request plane. The endpoint returns 503 until the engine is registered.
  • No listener opens when gates are unset.
• Gate trigger. Gates now start only once this endpoint is serving and the engine is registered. They get OPENAI_BASE_URL=http://127.0.0.1:<port>/v1 and DYN_VALIDATION_MODEL. This replaces #2's start-on-first-local-engine trigger, which could also fire on load_lora or clear_kv_blocks registrations.
• Endpoint lifetime. The endpoint closes on release, and stays up after a failed gate so the held worker can be debugged.
• Local engine registration. The unified Rust worker (backend-common) and the dyn:// token input (the mocker and run_input) now register a local engine when gates are set, even without a canary payload. Python workers already register one for every served endpoint, so the bindings don't change.
• enable_rl. HttpServiceConfig::enable_rl is now Option<bool>, so the loopback service can turn RL discovery off. DYN_ENABLE_RL is also a worker env var.
• One endpoint per process. WorkerValidation::claim_local_endpoint() ensures there's only one, even with multiple cards or LoRA attaches.

Tests

• lib/llm/tests/local_openai_validation.rs:
  • loopback_matches_frontend_on_the_mocker runs with hold=health, so the frontend can see the gated worker to compare against. It starts a real mocker worker with scripted output against a real frontend (ModelWatcher + HttpService). Unary and streaming chat responses from both are identical, with reasoning (deepseek_r1) and tool calls (hermes) parsed. Gates get the right URL and model, and the endpoint closes on release. Stripping the tool parser on the loopback side makes the test fail.
  • ungated_worker_opens_no_listener.
  • endpoint_is_unavailable_until_the_engine_registers: 503 and gates pending until the engine registers; then the gates start.
• Unit tests in local_openai.rs cover card eligibility, port parsing, 503 before registration, and the JSON round trip.
• The runtime validation tests are updated for the moved trigger.
• cargo fmt, cargo clippy -p dynamo-llm -p dynamo-runtime -p dynamo-backend-common --all-targets -D warnings and cargo check --workspace are clean.

Constraints

• There's no in-pod signal for hostNetwork, so it isn't rejected here. Under hostNetwork the endpoint is reachable from the node while gates run; an operator-side check is needed.
• Prefill, encode, text-input and embeddings-only workers get no endpoint, so their gates never start (fail closed). Don't gate them for now.
• Decode workers are served, but their requests bypass the prefill router.
• Only the base model is served; LoRA adapters aren't.
• Parity is with a default frontend. Per-frontend flags and Python chat processors aren't mirrored.
• Error chunks are shaped as Annotated errors rather than LLMEngineOutput with finish_reason=Error. Both surface as errors, but engine-error bodies may differ slightly from the frontend's.

Stack

1. #1 Split /live from /health
2. #2 Runtime validation gates
3. #3 Hold the model card until release
4. This PR: loopback OpenAI endpoint
5. Reference check runner, docs and an example DGD

🤖 Generated with Claude Code