perf(decofile): decompress .bin with node:zlib instead of npm:brotli
Why
engine/decofile/fs.ts imports { decompress } from "npm:brotli@1.3.3" to read the production decofile (/app/decofile/decofile.bin). That package's index.js also does require('./compress'), which loads build/encode.js. That's an Emscripten build that allocates TOTAL_MEMORY = 318767104 bytes (304 MiB) when the module loads.
fs.ts is imported statically (engine/manifest/manifest.ts), so every pod of every site pays this, even though only decompress is ever called.
Measured on Linux (denoland/deno:2.5.6), in a script that only imports the package:
| VmRSS | Deno.memoryUsage().external | |
|---|---|---|
npm:brotli | 89 MiB | 312 MiB |
node:zlib | 56 MiB | 3 MiB |
So it costs ~33 MiB of real memory per pod, plus a 312 MiB "external" reading that makes memory dashboards and V8's external-memory accounting misleading. In a heap snapshot of a production-mode farmrio instance, that single Int16Array was 324 of 579 MiB.
What
Use brotliDecompressSync from node:zlib, Deno's native Rust brotli. No npm package, no WASM/asm.js heap.
Tests
- New
engine/decofile/fs.test.ts: writes a brotli + base64decofile.bin(the production format) and reads it back throughnewFsProviderFromPath. - In a farmrio production-mode replay (800 real requests),
externalwent from 309 MiB to 4 MiB with no functional change.
Summary by cubic
Swaps the decofile decompressor from npm:brotli to node:zlib, cutting per-pod memory usage. The old package loaded its Emscripten encoder at import time, reserving a 304 MiB heap in every pod even though only decompress was used; measured RSS dropped from 89 MiB to 56 MiB and external memory from 312 MiB to 3 MiB.
- Replaces
decompressfromnpm:brotli@1.3.3withbrotliDecompressSyncfromnode:zlibinengine/decofile/fs.ts. - Adds a test in
engine/decofile/fs.test.tscovering the production brotli + base64.binformat. - Adds
disposeon the fs provider to close itsDeno.watchFswatcher and clear pending debounced reads, fixing a handle leak. - Starts the watcher in its own async task so a
watchFsfailure can't reject the decofile promise, and skips state updates afterdispose.
Summary by CodeRabbit
- Bug Fixes
- Improved compatibility when reading Brotli-compressed decofiles, including Base64-encoded files, while preserving support for regular JSON files.
- Improved cleanup when a filesystem provider is disposed, helping prevent lingering file updates after it is no longer in use.
- Tests
- Added coverage verifying that Brotli-compressed, Base64-encoded decofiles load and return the expected decoded content.