[Python] Add native lazy import support for Python 3.15

BE
2d ago
+419-22

Description

This PR adds support for Python 3.15's native lazy imports (PEP 810) to the Python generator.

OpenAPI-generated clients can contain hundreds or thousands of Pydantic models, many of which may never be used by the application. Loading all these modules during initialization can increase memory consumption and startup time.

Native lazy imports allow Python to defer loading modules until they are actually needed, helping reduce this overhead.

Changes

  • Added the lazyNativeImport configuration option to enable native lazy imports.
  • Added support for Python 3.15's native lazy import syntax in generated clients.
  • Restricted this feature to Python 3.15 or later.
  • Preserved the existing lazy import implementation for backward compatibility with older Python versions.
  • Kept native lazy imports optional to avoid changing the default behavior of generated clients.

Benefits

  • Reduced initial memory consumption, especially for clients with many Pydantic models.
  • Improved startup performance by avoiding unnecessary module imports.
  • Native Python support without requiring additional lazy-loading mechanisms.

Backward Compatibility

OpenAPI Generator already provides a lazy import mechanism that works with older Python versions.

This PR introduces native lazy imports as an additional option rather than replacing the existing implementation.

Both approaches should coexist until Python 3.14 reaches End of Life (EOL). After that, the legacy implementation could be considered for removal.

Benchmark Results

I ran a benchmark to compare standard imports, the existing lazyImports=true implementation, and the new lazyNativeImport=true option.

Each condition was executed 12 times to evaluate memory consumption, startup performance, first-use latency, and the number of imported modules.

ConditionnUSS MiBQ1–Q3Startup msFirst msWarm msAPI modulesModel modules
Standard imports12166.88166.77–167.026,542.6110.076.821861,113
lazyImports=true1252.3652.35–52.36404.81115.847.0110162
lazyNativeImport=true1253.7853.76–53.78416.04113.737.1810162

Key findings:

  • Memory consumption: Native lazy imports reduced initial USS memory usage by approximately 67.8% compared to standard imports (166.88 MiB → 53.78 MiB).
  • Startup performance: Startup time decreased by approximately 93.6% compared to standard imports (6,542.61 ms → 416.04 ms).
  • Module loading: The number of imported API modules decreased from 186 to 10, while imported model modules decreased from 1,113 to 162.
  • Existing vs. native lazy imports: Both implementations showed very similar performance and memory usage.
  • First-use overhead: Both lazy import approaches introduce additional latency on the first use, while warm execution times remain close to standard imports.

These results show that Python 3.15's native lazy imports provide performance and memory benefits comparable to the existing implementation while taking advantage of Python's built-in functionality.

Related Issue

Closes #25318

References

PR checklist

  • Read the contribution guidelines.

  • Run the following commands to build the project and update samples:

    ./mvnw clean package || exit
    ./bin/generate-samples.sh ./bin/configs/*.yaml || exit
    ./bin/utils/export_docs_generators.sh || exit
    

    (For Windows users, please run the script in WSL)
    Commit all changed files.
    This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
    These must match the expectations made by your contribution.
    You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
    IMPORTANT: Do NOT purge/delete any folders/files (e.g. tests) when regenerating the samples as manually written tests may be removed.

  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.


Summary by cubic

Adds an optional lazyNativeImport config that generates Python 3.15+ clients using Python's native lazy import syntax (PEP 810), deferring module loads until first use to cut startup time and memory for large clients.

  • Emits native lazy from ... statements instead of relying on the existing lazyImports emulation; the older mechanism remains unchanged and available.
  • Enforcing the option sets generated packaging (setup.py, pyproject.toml, requirements) to require Python >= 3.15 and Pydantic >= 2.14, and narrows CI templates to Python 3.15 only.
  • Rejects configurations that set both lazyImports=true and lazyNativeImport=true.
  • Adds generator and runtime tests across all four Python library templates, plus a GitHub workflow that exercises native lazy imports in CI.