Add FLOAT8E8M0 data type support in ONNX Runtime

Merged
CO
6mo ago
+672-6

Description

This PR adds support for the FLOAT8E8M0 data type in ONNX Runtime, following the same pattern used for existing Float8 types (Float8E4M3FN, Float8E4M3FNUZ, Float8E5M2, Float8E5M2FNUZ).

FLOAT8E8M0 Format

FLOAT8E8M0 is an 8-bit floating point format with:

  • 8 exponent bits
  • 0 mantissa bits
  • No sign bit (all values are positive)
  • Values are powers of two: 2^(val - 127)
  • 0xFF is reserved for NaN
  • No infinity, no negative values, no denormals

This format is useful as a scaling factor type in microscaling (MX) quantization formats.

Changes

Core Type Definition

  • include/onnxruntime/core/common/float8.h - Added Float8E8M0 struct with float conversion routines and std::numeric_limits specialization. The float→Float8E8M0 conversion correctly handles NaN (including negative NaN) and infinity by checking these special values before sign-dependent logic, consistent with how Float8E4M3FN handles them.

C API

  • include/onnxruntime/core/session/onnxruntime_c_api.h - Added ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E8M0 enum value at ordinal 24 to match the ONNX protobuf spec (FLOAT8E8M0 = 24). Note: UINT2 and INT2 shift to ordinals 25 and 26 respectively to align with the spec. Language bindings (Java, C#, JS) will need corresponding updates in a follow-up PR.

Framework Integration

  • onnxruntime/core/framework/element_type_lists.h - Added to AllFixedSizeIRv9, AllIRv9, AllFloat8 type lists
  • include/onnxruntime/core/framework/to_tensor_proto_element_type.h - Added template specialization
  • include/onnxruntime/core/framework/data_types_internal.h - Added dispatch cases
  • include/onnxruntime/core/framework/data_types.h - Added to IsTensorContainedType and IsSparseTensorContainedType
  • onnxruntime/core/framework/utils.h - Added GetONNXTensorElementDataType specialization
  • onnxruntime/core/framework/data_types.cc - Full type registration (tensor, sparse, seq, optional, prim)
  • onnxruntime/core/framework/tensorprotoutils.cc - UnpackTensor, size calculation, serialization
  • onnxruntime/core/framework/tensor_type_and_shape.cc - Type mapping
  • onnxruntime/core/framework/onnxruntime_map_type_info.cc - Type mapping

Shared Library Provider Bridge

  • onnxruntime/core/providers/shared_library/provider_interfaces.h
  • onnxruntime/core/providers/shared_library/provider_bridge_provider.cc
  • onnxruntime/core/providers/shared_library/provider_wrappedtypes.h
  • onnxruntime/core/session/provider_bridge_ort.cc

Tests

  • onnxruntime/test/framework/float8e8m0_test.cc - Unit tests for Float8E8M0 type including conversion, rounding, NaN (positive and negative), infinity (positive and negative), negative values, operators, and batch conversion
  • onnxruntime/test/onnx/tensorprotoutils.cc - Test infrastructure updates
  • onnxruntime/test/unittest_util/checkers.cc - Test checker updates

Motivation and Context

ONNX has added the FLOAT8E8M0 data type. This PR adds the corresponding support in ONNX Runtime so that models using this data type can be loaded and processed correctly.