Never alias borrowed string bytes on the C++ side of BunString

DY
1mo ago
+2454-2522

What does this PR do?

BunString::toWTFString(ZeroCopy), transferToWTFString(), Zig::toString(EncodedSlice) and the property-key helpers (getIfPropertyExistsImpl, getOwn, toIdentifier) wrapped a Latin-1/UTF-16-tagged borrowed EncodedSlice with StringImpl::createWithoutCopying. When the resulting WTF::String was retained — as a JSString, an Identifier (which atomizes that impl in place), a SourceProvider URL, a Blob file name — it outlived the Rust buffer. Whether a given call was safe depended on which Rust constructor had tagged the string (borrow_utf8 → copied, from_bytes(ascii) → aliased, from_static → atomized), which neither type system shows.

The boundary now has one rule per use:

  • toWTFString() / transferToWTFString() (result may be retained): WTF-backed shares the impl, static is atomized, a borrowed EncodedSlice is copied, a globally-allocated one is adopted (Zig::adoptExternal).
  • view() / Zig::toStringView(EncodedSlice) (call-scoped): a StringViewWithUnderlyingString over the bytes, no allocation — used for plugin filter matches, Yarr regexes, JSONParse, header lookup, URLSearchParams, setTimeZone, StringBuilder::append. This is cheaper than the old zero-copy path, which allocated a StringImpl header per call.
  • Bun::toIdentifier / Zig::toIdentifier: Identifier::fromString(vm, span) — an atom-table lookup that allocates only on first insert — for every property get/put/delete key, module registry key and SQL column name, instead of copy-then-atomize.

On the Rust side the same three shapes get three types, all the 24 bytes of BunString and modelled on String / &str:

  • bun_core::String owns — a WTF ref, 'static bytes via from_static, or Empty/Dead; clone() never allocates and its borrowing constructors (from_bytes, borrow_utf8, borrow_utf16, ascii) are gone.
  • &Str is the borrow and carries the whole read API (len, latin1_slice/utf16_slice/byte_slice, to_utf8() -> Utf8Bytes<'_>, eql, substring, …); String, StringView and the JSStringView GC guard all Deref to it, so read-only parameters are &Str in Rust and across the FFI (cppbind maps const BunString* → &Str, BunString* → &mut String) and callers pass &s. Str: ToOwned<Owned = String> makes to_owned() the single, visible allocation point (and Cow<'_, Str> work).
  • StringView<'a> is only the by-value carrier for borrowed caller bytes (StringView::utf8/utf16/latin1/from_bytes) or a borrow C++ returns; it is the one type that can hold the EncodedSlice tag with non-'static bytes, so the extern "C" signature is the only place a lifetime is erased.

The compiler then pointed at the places that had put a borrowed String where an owner belonged — the auto-install sourceURL above, dev-server error-report stack frames borrowing the request body, source-map lookup names borrowing the cached map — which now own their copy; a handful of call-scoped sites (process.argv, os.homedir() fallback, UDP hostname, inspector route list) simply moved their one copy from C++ to Rust. Exception source lines stay zero-copy: C++ hands Rust an ExternalStringImpl over the source provider's bytes that holds a provider ref (released with the string), and remapped lines are substring-sharing impls of the fetched original source, so the old referenced_source_provider / JSC__SourceProvider__deref pairing is gone.

ZeroCopyTag/NonNullTag are removed; EncodedSlices built in C++ name their encoding (Zig::latin1Slice/utf8Slice/utf16Slice). On the Rust side, embedded module sources that relied on zero-copy (wasi-runner.js, the sqlite module sources, the runtime) use String::create_static_external_latin1(&'static ..) (neither copied nor atomized), String::create_if_different never returns a bitwise clone of a borrowed string, and Bun::toJS/transferToJS/BunString__toErrorInstance/BunString__toJSON/FormData from a URL-encoded body/the JSON module loader throw or return a STRING_TOO_LONG error instead of yielding an empty value when a non-empty string cannot be materialized.

Fixes:

  • String::create_atom / try_create_atom (Bun.dns cache keys, Bun.file(fd) paths) and the property-get helpers (getIfPropertyExists, bake's module-namespace get) atomized a createWithoutCopying header in place, leaving the thread's atom table pointing into a Rust temporary after it was freed. They now atomize by content.
  • An auto-installed module's sourceURL (stack traces, inspector) pointed into a buffer dropped when the install finished.
  • FormData.append(name, bunFile) stored the entry's file name as a view into the Blob's stored name.
  • onLoad returning contents as a Uint8Array/Buffer decoded bytes ≥ 0x80 as Latin-1 ("héllo" → "héllo"); the string form was already UTF-8 — test added.
  • A plugin's onLoad received args.path as a JSString over the loader's specifier bytes rather than its own copy. On main those bytes are kept alive by the module-key string for ASCII specifiers and already copied for non-ASCII ones, so this was latent rather than observable; it is now an owning string like onResolve's.

How did you verify your code works?

test/js/bun/plugin/plugins.test.ts gains a case with a non-ASCII resolved path and TypedArray contents (fails on 1.4.0 with mojibake, passes here) and an ASCII-path case that reads args.path after an async boundary and GC. Debug+ASAN build driven through plugin onResolve/onLoad/virtual modules, FormData with File/Bun.file, JSON.parse, Headers, URLSearchParams, Intl time zone, bun:sqlite, Symbol.for, Object.fromEntries, with BUN_JSC_validateExceptionChecks=1; plugins, bundler_plugin, FormData, import-meta, node module, vm, headers, hot, serve, sqlite, wasi and bake ecosystem test files pass.