#715 feat(server): add in-process multi-model load routing

A single dflash_server can now serve multiple independently batched models behind one endpoint. --next-model starts a separate model option block; requests with model: "auto" select the least-occupied eligible model, while explicit names remain pinned. Two slots per model provide the initial Qwen/R9700 + DS4/Strix 2+2 configuration. Full capacity returns 503 without a global queue.

Each model retains its own tokenizer, chat template, model-card defaults, backend, sequence engine and scheduler. Startup parses every option block once before loading, rejects duplicate names and incompatible options early, identifies the offending block in errors, and prints the listener/model layout. Host/port/CORS are shared. The launch guide includes a copyable command, separate Bash arrays for model settings, and configuration inspection through /props.

Qwen KV overrides now travel through its backend configuration into budgeting and allocation without rewriting environment variables. DS4 uses a fixed cache layout and warns when the previously ineffective KV-type flags are supplied. Its automatic MMVQ width now belongs to its own GPU backend: the old process-wide default of 16 crashed Qwen's batched drafter. A backend crossover setting preserves the original DS4 policy; the existing thread override also forces MoE dispatch, so it cannot express that default faithfully. Explicit environment overrides remain process-wide.

The listener parses JSON and endpoint structure once, then the selected handler applies model-specific defaults and tokenization. Reservations last through engine retirement and output draining, including cancellation. Shutdown waits for workers and all borrowed model contexts to become unused; detached clients publish completion under the same mutex as the shutdown waiter.

Discovery lists the named models and auto in the existing OpenAI and Codex schemas. /props and /status/json expose per-model capacity, occupancy, configuration and worker status. This implementation covers local paged engines and occupancy routing. Context/capability-aware routing, global queueing, conversation affinity and quality collaboration remain outside its scope.

See server/docs/MODEL_ROUTING.md for startup, request selection, setting scopes and validation limits.

Dependencies and review scope

The branch includes #598 at a085cc222c513be0dd9d015fe8f7f8a61b6bcc8c and #707 at 828ecf14a951bcc6ca32dba17da1e958a6e2db8e, merged above main 3ce2b916. The main-based diff includes those prerequisites until they land.

The routing implementation starts at b007d96d. Follow-up commits separate request/lifetime fixes (d3889bf9), model launch configuration (b0afa4e1), and the backend-owned DS4 policy plus validation/docs (2176cbea).

Validation

• Built for both HIP gfx1151 and gfx1201, verified in all actual HIP compile commands. An earlier validation note incorrectly claimed both targets while CMake had overwritten the generic setting; the final build uses DFLASH27B_HIP_ARCHITECTURES.
• Model-free server suite: 456 passed, 0 failed. Includes real HTTP/scheduler integration for 2+2 admission, overload, pinning, independent progress, per-model tokenizers/templates/defaults, malformed input, cancellation/reuse, response protocols and shutdown.
• GPU dispatch regression passes: tested ROCmFP4 projections through 16 columns remain bit-identical to separate single-column computation, and a second backend retains valid Q8 dispatch at 16 columns.
• Real Qwen3.8-27B IQ4_XS + Q8_0 DFlash2 on R9700 and monolithic DeepSeek-V4-Flash ROCMFP2 on Strix: explicit and automatic C4 routing both reach 2+2, with correct response-model provenance and 503 overload.
• Both models' generated outputs match between their C2 and mixed C4 runs on two comparison prompts each. DS4 also matches the previous environment-based policy control on both prompts.
• Repository canonical benchmark: 12/12 requests passed, each returning 128 completion tokens, across three C4 waves of mixed code/prose prompts.
• Real TCP-reset cancellation and subsequent reuse pass for each model. Shutdown with one active request per model and an incomplete HTTP upload closes every client and exits with status 0; GPUs are released.
• Seven CLI validation cases pass before model loading. git diff --check passes.

Validation uses 4,096-token configured context limits and short prompts on this exact model/device pair. It does not establish long-context quality, other checkpoints/devices, or a performance improvement. Keep this PR in draft while the prerequisite changes and implementation are reviewed.