[OVEP] OpenVINO EP 1.26.0 Development Release Updates

Merged
AN
6mo ago
+817-212

Summary

Periodic upstream sync of Intel's OVEP branch (ovep_1_26_release) into ORT main. All changes are scoped to the OpenVINO EP and its tests.

OpenVINO 2026.0 / 2026.1 support

  • Add V_2026_0 / V_2026_1 version enums; capability.cc default bumped to V_2026_1.
  • Register FLOAT8E4M3FN / FLOAT8E5M2 initializer types on CPU / GPU / NPU.
  • Disable OVEP-level QDQ-stripping on OV ≥ 2026.1 (OV handles it internally).
  • Add ReduceSum to no-dimension-supported ops.

KV-cache / stateful CausalLM

  • Rename ReorderKVCache → SetReorderKVCacheStatus across backend interfaces.
  • Populate src_idx / dst_idx in PreProcessInferRequest with shape validation; clean state after inference and on RewindKVCache.
  • FuseCacheReorder: beam_idx and src_idx/dst_idx paths are now mutually exclusive; reject models that already carry reorder inputs.
  • Behavior change: RewindKVCache(index > 0) now throws when reorder is enabled (physical KV-cache eviction pass is a TODO).

NPU / provider options

  • Force disable_dynamic_shapes=true on NPU unless enable_causallm is set.
  • Preserve user-supplied NPU_COMPILATION_MODE_PARAMS; skip it when importing precompiled blobs.
  • Preserve factory-level device_type when session options don't override it (fixes NPU mis-selection from Python).
  • Behavior change: removed the ORT_OPENVINO_NPU_COMPILER_TYPE env override — OV's default NPU compiler is used now.

External initializers / weight sharing

  • Drop the 32 MB embed threshold — always externalize when multiple external initializers are in memory.
  • DumpOpenVINOEPModel rebuilds a self-contained proto when initializer data was stripped.
  • AddExternalWeight validates re-adds against existing offset/size/location (parity with ABI EP); fix race in device-tensor mapping.
  • ov_bin_manager: bounds-checked pointer view over mapped weights (fixes read-only blob import).
  • qdq_stripping: use std::from_chars so offsets/lengths > 4 GB parse correctly.

Perf-count dump

  • New ORT_OPENVINO_PERF_COUNT=<dir> env var writes per-subgraph CSV (Layer Name,Status,Layer Type,Real Time (us),Exec Type), replacing the old stdout-only debug dump. Requires ov::enable_profiling on the compiled model; logs a warning and no-ops otherwise.

Misc

  • API: IBackend::Infer is no longer const (needed for perf-dump bookkeeping).
  • Filter orphaned graph outputs from OVEP sub-graphs.
  • Better error message for "cannot export dynamically compiled model" (points to reshape_input).
  • Human-readable ovep_exception::type strings.
  • ov::shutdown() on DLL unload.

Tests

  • Add OVEP_ExtInit_DynamicEmbed_Tests and OVEP_ExtInit_EmptyRawData_Tests; refactor setup into SetUpTestSuite.
  • Narrow OVEP exclusions in embed_layer_norm, fused_matmul, matmul_4bits, quantize_linear (skip only unsupported sub-cases).
  • perftest: reset outputs per run to support data-dependent output shapes (e.g. NonZero).

Testing

Validated against the OpenVINO versions this release targets (2025.3 – 2026.1) on CPU / GPU / NPU:

  • New OVEP tests pass: OVEP_ExtInit_Tests, OVEP_ExtInit_DynamicEmbed_Tests, OVEP_ExtInit_EmptyRawData_Tests
  • Narrowed contrib-op exclusions verified against EmbedLayerNorm, FusedMatMul, MatMulNBits, QuantizeLinear
  • Stateful CausalLM flow exercised for KV-cache reorder + rewind
  • ORT_OPENVINO_PERF_COUNT=<dir> verified to produce per-subgraph CSVs
  • 2+ GB external-initializers-in-memory model loads on CPU / GPU / NPU