Bytecode cache: smaller payloads that page in only what is decoded (alias mapped bytes, region layout, 32-bit offsets, sparse metadata table)

Merged
JA
3mo ago
+1287-436

The bytecode-cache parts of the startup-snapshot branch (#397) that help every bun build --compile --bytecode executable, plus the format work that came out of measuring them, plus the UnlinkedMetadataTable change (was #496). Supersedes #187; #495/#496 folded in here. Bun side: oven-sh/bun#40201.

Test corpus for all numbers: a 12 MB minified bundle (typescript, babel, prettier, zod, …), 48k functions, encoded as one module the way Bun's --bytecode does; jsc shell, x64 Linux; "resident" = mincore on the cache file mapped cold.

mainthis PR
cache size75.2 MB (6.4× source)38.9 MB (3.3×) incl. checksums
resident after decoding the top-level block (module load)32.7 MB, 143 ms1.3 MB, 8–10 ms
… + bodies of its 5,823 direct children50.4 MB, 280 ms9.7 MB, 62 ms
… everything (48k blocks)71.7 MB, 785 ms36.2 MB, 283 ms
peak RSS force-decoding all 48k (none linked)217 MB175 MB (metadata table change alone)
anonymous memory of a Bun --compile --bytecode binary after running 4,000 functions43.7 MB32.0 MB (aliasing on vs off, same binary)

Commits

  1. Decoded InstructionStream / ExpressionInfo alias a persistent payload instead of copying it. CachePayload::setIsPersistent() is an explicit embedder promise (Bun: bytecode embedded in the running executable's section). InstructionStream::Ref reads through a span the stream owns (writer refreshes it after every mutation) so owned/borrowed share one branch-free path; UnlinkedCodeBlock reports only owned bytes to the GC. useBorrowedBytecodeFromCache (default on); shell testing switch diskCachePayloadIsPersistentForTesting.
  2. Region layout. A block's arrays and all its children's executable records are written contiguously; bodies are queued and emitted breadth-first; ExpressionInfo goes to a cold tail. Encoder-only (offsets are relative). This is the 32.7 MB → 1.9 MB row by itself.
  3. 32-bit relative offsets, alignof(T) alignment, sparse metadata offset table. CachedPtr 8→4 B, CachedVector 16→8 B; no more rounding every allocation to 16; CachedMetadataTable stores only the entries where the cumulative offset changes instead of unsigned[51] (212 B) in every code block. 75.2 → 47.8 MB.
  4. Bounds-check the sparse table's indices on decode.
  5. Varint tails for the two per-function records; inline string characters; drop dead offset fields. Counts/registers/positions/flags of CachedCodeBlock and CachedFunctionExecutable are a LEB128 tail after the fixed record (positions as deltas from the function start), read once into the constructed object; the fields CachedBytecode::commitUpdates patches stay fixed. CachedUniquedStringImpl is a 4-byte header followed by the characters. CachedString/CachedIdentifier/CachedHashMap/CachedInlineMap no longer carry an unused VariableLengthObject offset (CachedIdentifier was 12 bytes for a 4-byte pointer). 47.8 → 40.5 MB.
  6. Store byte-identical arrays once (instruction streams, expression info, jump tables, metadata steps): content-hashed in the encoder, later copies point at the first. 40.5 → 37.9 MB.
  7. CRC-32C per code block / executable record / ExpressionInfo. Computed by the encoder after all deferred slots are final; checked with hardware crc32c before a block is decoded (only over bytes that decode reads anyway). A damaged block decodes as absent → generated from source; damaged ExpressionInfo → empty. Pointers from slots the shell's commitUpdates rewrites are bounds-checked, and commitUpdates re-seals what it patches. Cost: +0.96 MB (2.5%), decode time and residency unchanged within noise; 244 single-byte corruptions swept across a cache file with no crashes. verifyBytecodeCacheChecksums (default on).
  8. UnlinkedMetadataTable holds only the offset table while no CodeBlock is linked; the value-profile/metadata buffer is allocated at link() and released when the owning table is unlinked (was: allocated at finalize/decode and kept for the UnlinkedCodeBlock's life).

Testing

JSTests/stress through the disk cache (write run + JSC_forceDiskCache read run, all 5,259 tests without special mode requirements), with and without the persistent-payload switch, after each commit: identical pass/fail to main (the v8-*finalizationregistry* and a couple of stack-depth regexp tests are flaky in cache mode on main too). Bun built against the preview: bun-build-compile, bundler_compile -t bytecode, bundler_compile_splitting test files pass, plus the new RSS test in oven-sh/bun#40201.