fix: WSL2 fallback for GPU device enumeration

TO
10d ago
+724-111

Problem

Fixes #13.

Under WSL2, GetSupportedDevices finds zero devices, making the EP unusable. The same wheel + GPU work on native Windows and bare-metal Linux.

Root cause

Two hardcoded hardware-identity checks in src/tensorrt_rtx_provider_factory.cc both fail under WSL2's virtual PCI topology:

  1. Vendor filter (GetSupportedDevicesImpl): requires HardwareDevice_VendorId(device) == 0x10DE. ORT's WSL2 PCI discovery reports the GPU as vendor 0x1414 (Microsoft virtual PCI).
  2. PCI bus mapping (IsOrtHardwareDeviceSupported, Linux path): cudaDeviceGetByPCIBusId("3dc3:00:00.0") fails. The CUDA driver knows the same card as 0000:0B:00.0 (cudaDeviceGetPCIBusId(0) succeeds).

Classic TRT/CUDA EPs are unaffected, because they address CUDA devices by index.

Fix

Two things are needed, and they are separable:

1. Accept the virtual vendor. IsEligibleGpuVendor accepts 0x1414 in addition to NVIDIA's 0x10DE on Linux. Windows still matches by LUID, so 0x1414 is not a candidate there.

2. Pair adapters with CUDA devices instead of trusting the PCI id. WSL2 can expose several identical 0x1414 adapters (the NVIDIA GPU plus an iGPU) for a single CUDA device, and none of them share a PCI id with the driver. So discovery now probes each hardware device, then assigns each supported CUDA device to at most one adapter:

  • ProbeOrtHardwareDevice resolves LUID (Windows) or pci_bus_id (Linux), and reports a failed PCI lookup separately instead of binding to device 0.
  • ClassifyHardwareDevice treats a failed lookup on a 0x1414 adapter as an unmapped virtual adapter. A real NVIDIA device whose bus id does not resolve stays rejected rather than being aliased.
  • SelectSupportedEpDevices (in src/tensorrt_rtx_device_selection.h, header-only so it is unit-testable without a GPU) lets mapped devices claim their index first, then pairs unmapped virtual adapters with the lowest-index remaining supported CUDA device. Extra adapters are left unselected, so each GPU is registered once.

Discovery publishes its decision as device_id EP metadata, and CreateEpImpl reuses it. The pairing rules need the whole hardware list, so re-resolving a single device at EP creation would always fail for a virtual adapter.

Compute-capability gate. Selection uses IsComputeCapabilitySupported, the same gate device init applies (8.6, 8.9, 12.0+), rather than a separate major >= 8 test. Otherwise selection could publish an SM 8.0 device that the EP then refuses to initialize. SelectionGateMatchesTheEpComputeCapabilityGate locks the two together.

Merge with 0.5.0

Rebased onto upstream main (Release 0.5.0). Upstream kept IsOrtHardwareDeviceSupported and added a cuda_device_id out-parameter plus the IsComputeCapabilitySupported gate; this branch had replaced that function with the probe/classify/select split above. Both are kept:

  • IsOrtHardwareDeviceSupported survives with its upstream signature and gate, now implemented on top of ProbeOrtHardwareDevice. It remains the strict single-device query and never applies the WSL2 pairing rules.
  • Upstream's target_architecture EP metadata is now also published by the selection loop.

Two conflicts merged textually but were broken behaviorally, so they are called out explicitly:

  • device_id moved from ep_options to ep_metadata. Upstream removed the manual device_id provider option. Leaving it in ep_options auto-forwards it into session config, where ProviderOptionsParser rejects the session with Unknown provider option.
  • CreateEpImpl no longer re-resolves strictly. Upstream's per-device lookup always fails for a WSL2 virtual adapter, which would have left the EP uncreatable and silently undone this fix.

Verification

Environment: WSL2 Ubuntu 26.04.1, RTX 5070 (sm_120), driver 617.42, TensorRT-RTX 1.7.1.116, ONNX Runtime 1.30.0, CUDA Toolkit 13.3.

  • EP library and unittests build clean against TensorRT-RTX 1.7.1.116 and ORT 1.30.0.
  • Full suite: 253 tests, 232 passed, 0 failed. 21 skipped (Vulkan interop and optional CUDA features).
  • DeviceSelection suite: 13/13 pass.
  • EP enumerates and runs inference on the GPU under WSL2, including FP8 and FP4 custom-op models.

Confirmed discovery output under WSL2:

vendor=0x1414  pci_bus_id = 3dc3:00:00.0
device_id = 0, target_architecture = sm120

Two notes for reviewers:

  • Upstream's SessionBindsToSelectedCudaDevice cross-check resolved the expected ordinal via cudaDeviceGetByPCIBusId, which rejects a WSL2 virtual bus id, so it failed on a host where the EP worked correctly. It now reproduces the 0x1414 pairing rule; real NVIDIA adapters keep the strict lookup, so it stays a genuine cross-check rather than a tautology.
  • Building this tree requires TensorRT-RTX 1.7 or newer (upstream 0.5.0 enforces it at configure time and in src/nv_includes.h). ORT 1.30.0 is required: the 1.31 SDK ships a stub onnxruntime_experimental_c_api.inc without the EpContextConfig symbols the EP needs.