module_info: fixed-width tagged wire format with one string table per executable
What does this PR do?
Follow-up to #40506. Shrinks the serialized ModuleInfo (the ES module record — imports, exports, requested modules — that --compile --bytecode --splitting embeds per chunk and that the runtime transpiler cache / --isolate source-provider cache store per file), and removes the decode step: JSC now reads the wire bytes in place.
Before: every import/export record was 1 kind byte + 4 × u32 slots, every string had a u32 length prefix, requested modules were three parallel u32/u32/u8 arrays, every chunk carried its own copy of the same chunk specifiers and export names, and loading a record memcpy'd the whole blob into an aligned buffer and built a fresh new Identifier[] for all of its strings.
After:
- A record is a string table + a body. Header bytes pick the integer width — u8, u16 or u32 — for string ids (from the table's count) and for string offsets (from the total string bytes). No varints.
- One tag byte per record:
kind | fetch-kind << 3 | same-name << 6. Slots the kind implies (*for a namespace import,ExportInfoLocal's padding, aNone/JS/Wasm/JSON fetch parameter) are not written;same-namedrops an import's local name when it equals the imported name.STAR_NAMESPACE/STAR_DEFAULTarecount/count+1. --compile: one string table for the whole executable (HAS_MODULE_INFO_STRING_TABLEregion in theStandaloneModuleGraph, in the startup run next to the bytecode string table); each chunk's body indexes it directly.- No decoded copy.
ModuleInfoDeserializedis a validated view over the body bytes and the table; for a compiled chunk both live in the mapped section, so building theJSModuleRecordallocates nothing on the Rust side.toJSModuleRecordwalks tag bytes + fixed-width ids and range-checks as it goes. - Per-VM identifier slots. Ids of a compiled chunk index one
Vector<Identifier>onJSVMClientData, filled on first use, so each name is UTF-8-decoded and atomized once per VM instead of once per chunk that mentions it. Self-contained records (transpiler cache) use a throwaway fastMalloc'd array (wasnew Identifier[]). - Runtime transpiler cache / isolated module cache: one blob = the module's own table + body, same reader. Cache version → 26.
Numbers
bun build --compile --splitting --bytecode --format=esm (this branch also has #40506's chunk folding, hence fewer chunks). Every configuration produces the same program output as the unbundled source; debug builds assert every embedded record decodes and diff it against JSC's own ModuleAnalyzer result.
| fixture | chunks | module_info | JS | bytecode | |
|---|---|---|---|---|---|
1 entry + 40 import() routes, 300 shared modules | main | 218 | 212,887 B | 150,177 | 205,424 |
| this PR | 150 | 19,491 B (10,896 bodies + 8,595 table) | 86,877 | 162,576 | |
main --minify | 218 | 217,303 B | 131,654 | 202,192 | |
this PR --minify | 150 | 20,908 B | 71,658 | 160,128 | |
| 1 entry + 100 routes × 40 imports, 600 shared modules (17.7k import/export names) | main | 515 | 1,603,741 B | 1,032,270 | 1,252,992 |
| this PR | 358 | 129,266 B (101,627 + 27,639) | 576,121 | 1,111,664 | |
main --minify | 515 | 1,662,600 B | 919,031 | 1,242,784 | |
this PR --minify | 358 | 154,078 B | 483,028 | 1,103,488 |
−90…91%; ~91 → ~8.7 bytes per import/export name on the larger fixture. Format-only on an identical chunk layout (before rebasing over #40506) was 219,522 → 30,393 B (−86%): after fixed-width packing, 88% of what remained was string bytes repeated across chunks, which the shared table removes.
How did you verify your code works?
test/cli/test/isolation.test.ts(the wide-module test now uses long names so the table's offsets take the u32 path),test/regression/issue/30887.test.ts(transpiler-cache round-trip with TLA),test/cli/run/transpiler-cache.test.ts(corrupt-record test rewritten for the new layout; out-of-range ids are still rejected withparseFromSourceCode failed),bundler_compile_splitting,bundler_compile,type-export,debugger-buntranspiledmodule,bundler_barrel— green withbun bd test.- Compiled both fixtures (plain and
--minify) plus a zero-import single-file entry with the debug build and ran them, including a driver that loads every lazy route; a 70k-export module round-trips through--isolate(u32 ids) locally.
Decode cost (added after merge, corrected)
CI release binaries, main at a3745c2 (already includes #40506, so the chunk layout is identical: 359 files) vs this PR, 1 entry + 100 lazy routes + 600 shared modules, --compile --splitting --bytecode --minify:
| main | this PR | |
|---|---|---|
| startup (entry + its one shared chunk, 2 module records), wall | 6.3 ms ± 0.9 | 7.0 ms ± 0.6 (noise) |
| startup, userspace instructions | 10.62 M | 10.71 M (flat) |
| load every chunk (358 module records), wall | 71.1 ms ± 4.3 | 65.8 ms ± 3.4 (1.08×) |
| load every chunk, userspace instructions | 261.1 M | 236.6 M (−9.4%) |
| executable size | 83.5 MB | 82.8 MB |
An earlier version of this note compared against a canary that predated #40506 and so overstated the startup win; those numbers were #40506's, not this PR's. With only two module records on the startup path there is nothing to see there; the gain shows when many chunks load.