bun_ptr: add OwnedRef, an intrusive ref that is released on drop
Problem
bun_ptrhas no handle that can sit in a field or container and release its ref on drop.RefPtr<T>owns a ref but has noDrop, so every holder callsderef()by hand on every path;ScopedRef<T>releases on drop but cannot be dereferenced or stored.- Code that parks a ref in a queue, a task, or an in-flight socket or timer therefore stores a bare
NonNull<T>or*mut Tand must release it at every site that removes it. Review on #37594 and #37618 asked for an RAII type instead. RefPtr's debug-build holder tracker took&mut selfon acquire and release, so aRefPtrused on one thread while another handle to the same object was released on a second thread would have formed a&mutover a count the other thread was reading.- #31173 proposed this type in May and never landed; this is the same type on current main, plus what the stacked consumers need.
Fix
- Adds
OwnedRef<T>: one non-null pointer that takes a ref on construction (newallocates,acquiretakes a ref,from_rawadopts one), derefs to&T, takes another ref onclone, and releases on drop.into_rawis the only way to give the ref up without releasing it.RefPtr::into_ownedandThisPtr::owned_refconvert existing handles. - Property to check: every constructor pairs one taken or adopted ref with the one
rc_derefinDrop, andinto_rawis the only escape, so a holder cannot forget or double release without naming it. Those are the same calls the hand-written sites make, so there is no runtime cost. It gives upRefPtr's debug holder tracking, asScopedRefalready does. SendandSyncfollowArc's bounds (T: Send + Sync). Only a host with the atomic count can meet them, sinceRefCountandCellRefCountedare!Sync. So that aRefPtron another thread is safe too, the debug tracker's methods now take&selfand its tables sit behind the mutex it already had; debug builds only.- Verification: nine new unit tests, two of them cross-thread, and
cargo miri test -p bun_ptrpasses all 27. No call sites outsidebun_ptrchange;cargo check -p bun_runtimeconfirms the derive output still compiles, and #37618, rebuilt on this branch, exercises it through the MySQL suites.
Background
- Intrusive refcount: the count is a field inside the object (
RefCount,CellRefCounted, or the atomicThreadSafeRefCount, added by derive macros).rc_ref/rc_derefbump it through a raw pointer, and the release that reaches zero runs the destructor and frees the allocation.AnyRefCountedis the derive-emitted trait that lets one generic handle work over all three counts. ScopedRef<T>takes a ref for one scope and releases it on drop; it keeps an object alive across a re-entrant call and cannot be dereferenced or moved.RefPtr<T>is the storable handle, released by an explicitderef(); its docs say dropping one leaks.ThisPtr<T>wraps thethispointer a callback is dispatched with and asserts it points at a live object.ref_guard()turns it into aScopedRef; the newowned_ref()is the storable version.- Debug holder tracking: in debug builds
RefPtrrecords an id per holder in tables inside the count'sDebugData, so a double release can be diagnosed. It is compiled out of release builds, which is why the locking change costs nothing there. - Miri with Tree Borrows is how this crate's tests run. It rejects a destroying release made through a pointer derived from a shared borrow, which is why the constructors take
*mut Trather than&T(the contractScopedRef::newalready has), and it checks the cross-thread tests for data races.
Original description
What
bun_ptr has two handles over an intrusive refcount, and neither is the one a struct field or a container element wants:
RefPtr<T>owns a ref but has noDropimpl. Its own docs say that dropping one leaks, so every holder releases by hand on every path.ScopedRef<T>releases on drop, but cannot be dereferenced or stored; it brackets one re-entrant call.
So code that parks a ref somewhere (a request queue, a task, an in-flight socket or timer) stores a NonNull<T> or *mut T, remembers that it stands for a ref, and calls deref() / deref_nn() at every site that removes it. Review on #37594 and #37618 asked for an RAII type instead. #31173 proposed exactly this type in May and never landed; this PR is that type again, with the same name and constructor vocabulary (Jarred is credited as co-author), on current main and with the three things the reworks stacked on it need: Send / Sync for thread-safe hosts, as_non_null / this_ptr for the call sites that key on the pointer, and RefPtr::into_owned for migrating existing handles. If you would rather land #31173 itself I can restack the consumers on it instead; the consumers only use the shared surface.
pub struct OwnedRef<T: AnyRefCounted>(NonNull<T>);
impl OwnedRef<T> {
pub fn new(value: T) -> Self; // allocates; owns the initial ref
pub unsafe fn acquire(ptr: *mut T) -> Self; // takes a ref (same contract as ScopedRef::new)
pub unsafe fn from_raw(ptr: *mut T) -> Self; // takes over a ref the caller owns
pub fn into_raw(self) -> *mut T; // hands the ref off; inverse of from_raw
pub fn as_ptr(&self) -> *mut T;
pub fn as_non_null(&self) -> NonNull<T>;
pub fn this_ptr(&self) -> ThisPtr<T>;
}
impl Deref for OwnedRef<T> { type Target = T; } // live by construction: it holds a ref
impl Clone for OwnedRef<T> // takes another ref
impl Drop for OwnedRef<T> // releases it
impl RefPtr<T> { pub fn into_owned(self) -> OwnedRef<T>; }
impl ThisPtr<T> { pub fn owned_ref(self) -> OwnedRef<T>; }
unsafe impl Send / Sync for OwnedRef<T> where T: Send + Sync
It is ScopedRef as a value: the same rc_ref on construction and rc_deref on drop, plus Deref, Clone and the hand-offs, so it works for all three count flavours (RefCounted, CellRefCounted, ThreadSafeRefCounted) through the AnyRefCounted bridge the derives already emit. It is one pointer with a null niche (Option<OwnedRef<T>> is pointer sized; there is a test for that). ThisPtr::owned_ref() sits next to ref_guard() as the safe way for a callback to take one. There is no conversion from OwnedRef back to RefPtr; nothing needs one.
Send / Sync follow Arc's bounds. That is what lets a work-pool task carry the ref it holds on a ThreadSafeRefCounted host and release it on the other thread, which is the shape the stat watcher (#37591) needs. The bounds are sound because the only count that can live inside a Sync host is the atomic one: RefCount and the Cell<u32> behind CellRefCounted are !Sync, so hosts embedding them never satisfy the bounds; and ThreadSafeRefCount's ref_ / deref are already the operations such hosts use from several threads today. The one thing that was not ready for this was RefPtr's debug holder tracking: its acquire / release took &mut self (and debug_data_ptr took &mut of the whole count), so a RefPtr driven on one thread while an OwnedRef to the same object was released on another would have formed a &mut over a count the other thread was reading, which review pointed out. Those methods now take &self and the two tables live behind the lock the struct already had (DebugData is debug-build only, so this costs nothing in release); noop_debug_data becomes a plain static. Two tests cover the threading: one clones an OwnedRef on a thread-safe host into four threads, drops the clones there and destroys it from a fifth, the other keeps a RefPtr recording and releasing holder entries on the main thread while four threads release OwnedRef clones of the same object. The whole test module runs under Miri, which also checks these for data races.
The constructors take *mut T rather than &T on purpose. A first version took &T; Miri (Tree Borrows, which is how this crate's tests run) rejected it, because a ref taken through a pointer derived from a shared borrow may be the one whose release destroys the object, and the destroying release writes to the count's debug data and frees the allocation. ScopedRef::new and RefPtr::init_ref have the same *mut T contract for the same reason, so the per-type ref_guard(&self) style helpers that already exist for ScopedRef are where the one audited cast belongs (the first consumer, #37618, adds JSMySQLQuery::owned_ref(&self) next to its ref_guard). new(value) needs no unsafe at all and replaces the Box plus initial-count hand-off that types like the proxy tunnel do today.
What it gives up relative to RefPtr (and to the #31173 version) is the debug-build holder tracking, which ScopedRef does not have either; in the current port that tracker only records ids, and it is not written for concurrent use, so being able to send the ref is worth more than keeping it. RefPtr itself gains into_owned, the &self tracker described above and a doc pointer here.
No call sites outside bun_ptr change (the derives' generated code calls debug_data_ptr through a reference, which now auto-borrows shared; bun_runtime, which has thread-safe hosts, was checked). The PR adds the type, its docs and nine unit tests (take/release, last one out destroys, new / into_raw / from_raw round trip, Cell<Option<_>>::take releases exactly once, RefPtr::into_owned leaves the count alone, ThisPtr::owned_ref outlives the callback, the two cross-thread tests, the niche).
Why
With the ref represented as a value, "this field holds a ref" is stated by its type, releasing it is = None, take(), or popping it out of a container, and a path that forgets to release, or releases twice, does not compile unless it names into_raw. The follow-ups that use it are the reworks requested in review: #37618 (the MySQL request queue becomes a fifo of OwnedRef<JSMySQLQuery>, removing the five deref_nn sites), #37591 (the stat watcher's tasks carry the ref instead of the callbacks adopting it) and #37594 (the WebSocket upgrade client's socket, C++ owner and tunnel refs become Option<OwnedRef<Self>> fields); they are stacked on this branch. There is no runtime cost relative to the hand-written pairs: construction and Drop are the same rc_ref / rc_deref calls the hand-written sites make, inlined.
Part of a series of small type-system hardening changes.
Verification
cargo check, cargo clippy and cargo fmt --check are clean for bun_ptr. cargo miri test -p bun_ptr (Tree Borrows, as bun run rust:miri runs it): 27 passed, including the 9 new tests. cargo check -p bun_runtime confirms the derive-generated code is unaffected by the &self change; #37618, rebuilt on top of this branch, exercises it through the MySQL suites.