perf: import rarely used modules lazily on Python 3.15 via __lazy_modules__

Draft
BE
2d ago
+89-0

PEP 810 (Python 3.15) lets a module name other modules in __lazy_modules__. Module-level import x / from x import y statements for a listed module then bind a lazy proxy, and the module is imported the first time the name is used. Python 3.11–3.14 ignore the variable, so on those versions nothing changes.

This PR lists modules that are imported at module level but not used while the importing module executes, so on 3.15 they are only loaded when needed.

Listed modules

File__lazy_modules__Used only in
reflex/app.pyreflex.compiler, reflex.compiler.compilercompiling the app. Importing reflex.app, creating rx.App() and add_page don't need it. Deferring it skips 19 modules (compiler plugins, reflex_base.components.memo, reflex_base.plugins.compiler, the radix theme and document components).
reflex/utils/exec.pyclickparsing one env var in a function
reflex/utils/prerequisites.py, reflex/utils/js_runtimes.pypackagingversion checks in functions
reflex/utils/telemetry.pymultiprocessingcpu_count() when telemetry is sent
reflex_base/utils/console.pyrich.progress, rich.prompt, rich.tableprogress(), ask() and table output. rich.console stays eager because the module creates its Console at import time.
reflex_base/plugins/sitemap.pyxml.etree.ElementTreesitemap generation

The comment next to each declaration says to keep these names out of import-time code. A module-level use would silently load the module eagerly again.

How I picked them

  • On 3.15.2 I recompiled reflex's own modules with every module-level import marked lazy and ran each workload below. Then I listed which import proxies were never resolved and whether their target got loaded anyway. Internal modules can't all be made lazy at once (the reflex_base.vars import order breaks), so I probed those one at a time.
  • I excluded modules whose deferral would skip import-time side effects:
    • reflex.utils.telemetry_accounting imports reflex.istate.shared, which registers SharedStateBaseInternal as a substate of State. This also rules out listing reflex.utils in reflex/app.py, because that one statement also imports prerequisites and js_runtimes.
    • reflex.istate.manager.redis records its pid and start time at import for its debug-log prefix.
  • I checked that reflex compile of a small app writes identical .web output and log messages (apart from durations) on 3.14 (eager) and 3.15.2 (lazy).
  • tests/units/test_lazy_modules.py (skipped before 3.15) imports each module in a fresh interpreter and asserts the exact set of names still bound to lazy proxies. It fails if a declaration is dropped or a listed name gets used at import time.
  • ruff doesn't flag the assignment before the imports: E402 already allows __lazy_modules__, so no noqa is needed.

Timings (Python 3.15.2)

Median of 40 paired samples per row, two independent runs shown as run 1 / run 2. Each pair runs upstream main and this branch back to back (order alternating) in fresh interpreters, from two copies of the source tree that differ only in these 7 files, in the same uv venv on Python 3.15.2. Import rows time only the import statement in-process (perf_counter), excluding interpreter start-up and shutdown. The CLI row is full-process wall time. "Minimal app" imports a module with import reflex as rx, one rx.State subclass with an event handler, one page, rx.App() and add_page.

Workloadmain (ms)this PR (ms)Paired diff (ms)Modules loaded
import reflex2.8 / 2.82.8 / 2.80.0 / 0.052 → 52
import reflex.state116.1 / 111.3108.0 / 105.5−6.5 / −5.8479 → 455
import reflex.app221.0 / 217.4205.6 / 203.2−16.1 / −14.2911 → 865
import reflex.model150.5 / 149.1146.9 / 146.3−4.3 / −2.6534 → 510
Minimal app module222.1 / 217.4206.6 / 204.7−14.3 / −12.9916 → 870
reflex --help (wall)70.3 / 69.864.7 / 64.4−5.8 / −4.8315 → 289

That is about 6–7% off importing an app and about 7–8% off CLI start-up. import reflex alone was already cheap on 3.15, since lazy_loader.attach binds native lazy imports there (#6930).

Deliberately left eager

  • socketio (~70 ms, and it pulls in redis, aiohttp and opentelemetry): EventNamespace(AsyncNamespace) in reflex/app.py subclasses it at import time, and App() creates the AsyncServer.
  • sqlalchemy / sqlmodel in reflex.model: Model subclasses SQLModel at import time.
  • pydantic (serializer registration), rich.console, starlette (reflex_base.utils.streaming_response subclasses StreamingResponse, reached from reflex.app through reflex._upload), and the reflex_base vars/event/component modules: all are used while reflex.state / reflex.app execute.
  • reflex.utils.exec in reflex/state.py and reflex/app.py: App() loads it anyway, and the gain for a bare import reflex.state was within noise once click was lazy.
  • hashlib, json, subprocess: under 1 ms each and spread over several files.
  • reflex/utils/console.py re-exports reflex_base.utils.console with import *, which resolves every lazy name. So the rich entries only help where nothing imports that shim, i.e. the CLI and reflex.model. An __all__ in reflex_base.utils.console could avoid that, but that's a code change I left out of this PR.

🤖 Generated with Claude Code