PR #635 - oven-sh/WebKit

Two commits (the mechanism; the follow-up to the second review round), stacked on #632 (its tests use $vm.markedBlockStatistics() from there, and both add $vm functions at the same place). No dependency on #628. Nothing here runs unless the testing option or the $vm function asks for it.

This PR adds the mechanism, $vm.evacuateAuxiliaryBlocks(), and a testing mode
(Options::evacuateAuxiliaryBlocksAfterEveryFullCollection, default off); in production nothing calls it. The two-line
hook into VM::shrinkFootprintNow (which only exists on #628's branch; main only has upstream's
shrinkFootprintWhenIdle()) is a separate follow-up, claude/gc-evacuate-on-shrink-footprint, described at the end.

Why

After an interactive session of a large bundled CLI application the Auxiliary subspace (butterflies) holds 2.8 MB of live
cells in 2,270 pages (8.9 MB): what the session's peak left behind is spread thinly over 900 blocks, and page-granular
decommit cannot help because nearly every page has a survivor. Allocation policy does not help either (resident-first
picking, sealing blocks below 25 % / 50 % / all: every variant measured neutral or worse); the fragmentation is created by
the first session's peak and then stays flat.

JSC does not move cells, but a butterfly is special: exactly one pointer to it is known to the collector, its object's
m_butterfly.

Measured with the follow-up hook, i.e. the embedder's deep-idle footprint shrink calling this (20 inputs / idle 265 s /
10 inputs / idle 265 s, five interleaved pairs, off -> on):

	off	on
anonymous memory after the shrink, first idle	126.8 MB	119.9 MB
anonymous memory after the shrink, second idle	130.2 MB	123.4 MB (-6.8 MB)
pages holding a live cell	61.3 MB	54.8 MB
candidate blocks / evacuated		640-730 / 600-690
butterflies moved		13.6-15.3 k, 1.3-1.4 MB copied, none pinned
cost		two heap walks, 18-22 ms on a quiet machine (up to 116 ms under load); 90 M instructions for 292 k live objects and 41 k moves in the jsc shell
user instructions, whole process	34.95 G	35.05 G (inside the +-0.6 G spread between identical runs)

At a fresh prompt (no session yet) it is worth 0.7-1 MB.

What

Heap::evacuateSparseAuxiliaryBlocks(maximumOccupancy), for a program at rest:

Candidates: MarkedBlocks of the Auxiliary subspace whose live bytes (mark count, once per block) are at most
maximumOccupancy of a block, except blocks an allocator is in the middle of, and except size classes where moving
would not free a block (a single candidate, or survivors that would fill as many blocks as they came from). That rule
is what gives repeated calls a fixed point: without it each call finds the block the previous one half filled sparse
again. Candidates are taken out of the allocatable set while cells move, so copies never land in them; the ones that
are not evacuated in the end are put back.
Owners: one walk over every live JSObject (mark bits per block, cell kind decided per block; precise allocations
separately) finds, for each live cell of a candidate, whether it is the (non copy-on-write) butterfly of exactly one
object.
Pins: a conservative scan of everything the collector scans conservatively (below).
A block is evacuated only if every live cell in it can move. Compiler threads are parked for the move loop only. Each
cell is copied whole (pre-capacity, out-of-line properties, IndexingHeader, vector or ArrayStorage move together) into a
cell of the same size class allocated the ordinary way, and installed with JSObject::setButterfly.
The old copy stays marked and untouched; the next full collection finds it unreferenced and frees it, and with it the
block. The caller follows the evacuation with a full collection; a second call before that collection returns at once.

It never asserts on the caller's state. If this is not a moment at which it can run (no API lock, heap access released, a
collection in progress, collection deferred or prevented by the caller, heap iteration, a second thread registered with the
VM, the CLoop) it returns an empty result whose skipped field says why. The header comment spells out what it asks of an
embedder that keeps raw pointers into out-of-line storage.

$vm.evacuateAuxiliaryBlocks(occupancy = 1, holder) runs it on demand and returns the counters, skipped and the time.
With a holder, the native frame keeps a raw pointer to the storage of holder.target (and nothing else of the target)
across the call and reports heldStorageStayed: that is how the tests reach the pointer rule without the owner rule.

Safety argument

Exactly one owner. A cell of the Auxiliary subspace moves only if the heap walk found exactly one live JSObject whose
butterfly()->base(structure) is that cell and whose indexing mode is not copy-on-write. Everything else in the subspace
(JSPropertyNameEnumerator buffers, ScopedArguments overflow storage, StructureChain vectors: 1 % of the bytes) has no such
owner, stays, and keeps its whole block in place. Copy-on-write butterflies are cells of another subspace
(JSCellButterfly) and are not candidates; neither are precise allocations. So after step 2, for every cell that may move,
the only heap reference is one known m_butterfly.

No other reference anywhere else. Raw pointers to an ordinary butterfly exist only transiently:

generated code loads m_butterfly each time it needs it and bakes no storage pointer for ordinary objects:
ConstantStoragePointer is only made for typed-array vectors (DFGConstantFoldingPhase), which live in another subspace
and are never touched here; the butterfly constants that DFG/FTL do embed are JSCellButterfly (copy-on-write) cells;
inline caches and profiles hold offsets, structure IDs and cells; suspended generators and async frames hold JSValues;
what remains is what the collector itself scans conservatively: machine registers and the machine stack (JIT code that
hoisted the load; C++ that keeps a Butterfly* local across a call that can allocate), active DFG scratch buffers and
OSR exit side state, and the copies of machine stack and callee-saves that a suspended JSPI computation keeps
(PinballCompletion). All of these are scanned: registers and stack through callWithCurrentThreadState (so the whole
of this function's own frame is above the scanned stack top), the buffers through one VM function that
gatherScratchBufferRoots now shares.

The conservative pin rule. A candidate cell is pinned if a scanned word points into it or up to
sizeof(IndexingHeader) past its end (two probes, p and p - sizeof(IndexingHeader) - 1: the butterfly pointer of an
object without indexed storage sits that far beyond its out-of-line properties); this is the arithmetic of
ConservativeRoots::genericAddPointer. In addition, the storage of every object that a scanned word refers to is
pinned, whether the object is a cell of a MarkedBlock (interior pointers included) or a precise allocation (large cells and
the first cells of every IsoSubspace, found in a table sorted by address): code that has the object at hand may hold a
pointer derived from its butterfly that no longer points into the allocation. A pinned cell stays, and so does its block.
If another thread is registered with the VM (its stack is not scanned) nothing is done at all.

Collector and compiler threads. The evacuation runs under PreventCollectionScope (no marker is looking at a
butterfly while it is replaced) with allocation-triggered collection deferred. Compiler threads read the butterflies of
constant objects, so they are parked with Heap::suspendCompilerThreads() while cells move: that waits until each is at a
safepoint, where a plan holds cells only.

Barrier. The copy is installed with JSObject::setButterfly, i.e. the ordinary write barrier: the owner goes into the
remembered set and the next collection (eden or full) marks the new storage through it, exactly as when an old array's
storage is reallocated on growth. Concurrent-marking races do not arise because no collection is in progress.

Old copy. Nothing is freed by hand. The old cell stays marked and intact until the next full collection clears the mark
bits and finds it unreferenced, so even a missed reader would read stale but valid data until then; the testing mode below
removes that slack on purpose.

The testing mode

Options::evacuateAuxiliaryBlocksAfterEveryFullCollection evacuates every Auxiliary block (occupancy 1, no fixed-point
rule) from the first allocation that takes the slow path after each full collection, i.e. under arbitrary JS, JIT and C++
frames; scribbles the old copies with 0xbadbeef0 (as a JSValue: a cell at an address that is never mapped; as a length:
out of bounds); then walks the heap again with the generic iterator and release-asserts that no butterfly points into an
evacuated cell and that every butterfly is live.

It does not also evacuate where a synchronous collection (gc()) returns to its caller. Once cells have moved nothing
moves again before the next full collection, so an evacuation there takes the place of the one on the slow path: with such
a call in Heap::collectNow (after its DeferGCForAWhile scope, where it is live) all 120 evacuations of
auxiliary-evacuation-under-jit-frames.js happen at the return of gc(), none inside the compiled loops, and the test
passes with the whole stack scan taken out (without the call it fails at iteration 9 with a wrong sum).

Changes since the first version of this PR (review round)

The owner rule now covers owners in precise allocations (it silently skipped them: isAtomAligned / block-set test).
No page-size gate any more: evacuation frees whole blocks and works where an OS page is as large as a block.
Candidates that are not evacuated get their allocatable bit back (they used to stay frozen until the next sweep).
No RELEASE_ASSERT on caller state: one predicate, shared with the testing hook, and a skipped reason in the result.
$vm.evacuateAuxiliaryBlocks: exception check after toNumber, occupancy clamped to [0, 1] (NaN = 0).
Every use is under #if USE(BUN_JSC_ADDITIONS) like the declarations; memcpySpan; include order.
Tests rewritten so that each mechanism has teeth (table below); the dead collectNow hook and its comment are gone.
m_isEvacuatingAuxiliaryBlocks, lastAuxiliaryEvacuation() and its member are gone; duration is set before the
return and exposed; m_isCollectionPrevented stays as a skip condition because preventCollection() does not nest and
an allocation slow path can run inside a caller's PreventCollectionScope (Heap::deleteAllCodeBlocks).
(partly) Scans OSR exit side state and PinballCompletion slices too; one scratch-buffer walker shared with
gatherScratchBufferRoots (a ScopedLambda, no allocation); callWithCurrentThreadState; the redundant p - 1 probe is
gone. Not done: factoring the pointer lookup out of ConservativeRoots::genericAddPointer and driving the scan from
gatherStackRoots/gatherVMRoots with a custom sink (touches the hot conservative-scan path; see "Not done").
Fixed point and repeat cost: see step 1 and step 5 above. A dense heap now reports 0 candidates from the second
gc(); evacuate round on.
(partly) Candidate selection by mark count; the owner walk iterates mark bits with the cell kind decided per block;
compiler threads are parked around the move loop only; the allocator is fetched once per block. 8,937 cells out of 94
blocks with 60 k live objects: 2.4 ms in the release shell.
Refuses to run without heap access; the embedder contract is in the header comment.

Changes in the second review round (second commit)

The two findings written against 6a9b828f no longer applied at 8d30d954: objects in precise allocations are found
through the sorted table (re-checked: with that lookup taken out pins-storage-in-use pins 3 instead of 4 and both
lines fail), and the dead call in collectNow is gone. Making that call live instead was tried and rejected, see
"The testing mode".
The precise-allocation lookup of the owner rule also accepts a pointer one past the end of the cell, which is what
ConservativeRoots::genericAddPointer accepts for such a cell.
RegisterState saves rbp on x86-64 and x29 on ARM64. In code compiled without frame pointers (the JSCOnly release
shell; Bun's own builds keep them) these are ordinary callee-saves, and a pointer that lived only there was seen by
neither the collector's scan of the current thread nor this one unless a frame in between happened to push it. Found
by an adversarial read of the pin set whose other results were negative (stack-word shapes, derived pointers in
DFG/FTL/B3, the collector's other conservative roots, non-stack holders of raw storage pointers, compiler-thread
parking, objects in the middle of a transition, the collector handshake).
One baseOfMovableStorage(JSObject*) for the ownership walk and the validation pass.
auxiliary-evacuation-with-concurrent-compiles.js moved nothing in its first run line (one block per size class is
never a candidate without the testing option). It now makes the blocks of the constant objects sparse again before
every collection, keeps the constant objects off the stack, compares the storage addresses of the constants that the
generation's function uses before and after each evacuation, and fails unless they moved (120 of 120 do, in both
lines; without the survivors 5 of 120, with the constants on the program's frame 18 of 120). Half the generations,
so it costs what it did (0.2 s release, 7-8 s debug + ASan).
Tests for this round (6263b1b3, JSCOnly shells: release, and debug + ASan with
ASAN_OPTIONS=detect_stack_use_after_return=0): the 16 run lines of auxiliary-evacuation-*.js and
marked-block-decommitted-pages-are-not-read.js pass on both. 1,522 JSTests/stress files about arrays, storage and
properties, release, with --evacuateAuxiliaryBlocksAfterEveryFullCollection=1 --slowPathAllocsBetweenGCs=53 against
the same shell without them, once with default tiering and once with --useConcurrentJIT=0 and eager thresholds: the
same exit codes, except two memoryHog OOM tests (own-property-names-oom.js, get-own-property-descriptors-oom.js)
that also run into the 600 s limit with --slowPathAllocsBetweenGCs=53 alone. check-webkit-style: 0 errors.
$vm.markedBlockStatistics() reports blockSize. MarkedBlock::blockSize is 64 KB on Linux arm64
(CeilingOnPageSize), and "more than a block's worth of garbage" was a constant sized for 16 KB: the Linux arm64 test
lane failed leaves-unevacuated-blocks-usable (84 candidate blocks where it asks for 100) and the rewritten
with-concurrent-compiles (both survivors in one block: 1 of 120 evacuations moved the constants); the x64 test lane
had 0 failures. Those two tests and pins-storage-in-use scale their counts by blockSize / 16 KB. Reproduced and
checked with the arm64 shell that CI built, run under qemu: the two failures reproduce exactly, and with the scaled
counts all 13 run lines of the six tests pass there. Not an arm64 problem in the mechanism (the x29 capture and the
pin rules are in that shell).

Tests

jsc shells, release and debug + ASan (ASAN_OPTIONS=detect_stack_use_after_return=0), of this branch (7e2212d0a0ef) and of
main (cf1b36ec8703); base lists retaken in the same session. "Testing mode" = --evacuateAuxiliaryBlocksAfterEveryFullCollection=1.
check-webkit-style on the commit: 0 errors in 13 files.

New tests, 13 run lines, 13/13 on release and on ASan (every line 20-130 ms on release except the two compile-thread lines
at ~200 ms; loop counts derive from testLoopCount; LLInt-only lines use --useBaselineJIT=0, not --useJIT=0, so that
the collector stays generational):

test	what it does
`auxiliary-evacuation-moves-butterflies.js`	out-of-line properties and int32 / double / contiguous / ArrayStorage arrays, sparse survivors; explicit evacuation; contents checked after the move, after an eden collection, after 16,000 new objects and arrays took whatever that collection freed, after a full collection; the evacuated blocks are freed; objects then grow, transition and shrink; a second evacuation finds nothing to do
`auxiliary-evacuation-pins-storage-in-use.js`	explicit evacuation with, on the stack: an array that is a precise allocation, one that is a block cell, a plain array, an object with out-of-line properties only (their storage must stay, everything else must move); then a native frame holding only a raw pointer into an array's storage, then only a pointer `sizeof(IndexingHeader)` past an object's out-of-line properties
`auxiliary-evacuation-under-jit-frames.js`	testing mode; compiled loops (default tiering, eager DFG+FTL, eager DFG only) that load an array's or object's storage once and keep using it across an allocation that, after a `gc()`, takes the slow path and evacuates: element loads, out-of-line loads, out-of-line stores on plain objects and on functions
`auxiliary-evacuation-under-allocating-natives.js`	testing mode with `slowPathAllocsBetweenGCs` 21 (LLInt) and 11 (eager JIT) and with collectContinuously: slice, concat, spread, push, splice, unshift, Object.assign, JSON, fill + grow
`auxiliary-evacuation-leaves-unevacuated-blocks-usable.js`	150 sparse blocks that each hold a for-in enumerator's buffer cannot be emptied; 20,000 arrays allocated afterwards must go into their free cells (block count +25, not +125)
`auxiliary-evacuation-with-concurrent-compiles.js`	concurrent compiles folding loads from constant objects during 120 explicit evacuations that each move the storage of those objects (asserted by comparing addresses), with and without the testing mode

Teeth. Release builds with one mechanism taken out at a time, the six tests above run against each:

removed	caught by
the whole scan of stack, registers and buffers	7 run lines: pins-storage-in-use (2), under-jit-frames (3: wrong sums), under-allocating-natives (2: SIGSEGV in the LLInt line, a wrong `push` result in the eager-JIT line; its collectContinuously line is not deterministic and does not count)
the pointer rule (both probes), owner rule kept	pins-storage-in-use, both lines (`heldStorageStayed` false)
the `p` probe only	pins-storage-in-use, both lines (array held by the native frame)
the `p - sizeof(IndexingHeader) - 1` probe only	pins-storage-in-use, both lines (object with out-of-line properties only)
the owner rule, pointer rule kept	pins-storage-in-use, both lines
the precise-allocation lookup of the owner rule	pins-storage-in-use, both lines (3 pinned instead of 4)
the barrier after `setButterfly` (plain store)	moves-butterflies, both lines (`{}` where an array was)
restoring the allocatable bit	leaves-unevacuated-blocks-usable (198 -> 323 blocks)
parking the compiler threads	nothing (see "Not done")

After the last variant the source was restored and the rebuilt binary is byte-identical to the tested one.

JSTests/stress, release, all 5,838 files (197 skipped by their own directives), failures / passes:

mode	this branch	`main`, same mode without the testing option	only failing here
default options	35 / 5606	37 / 5597	`recursive-try-catch.js` (intermittent "Used too much heap", fails on `main` in 2 of its 3 other runs)
testing mode + `--useGenerationalGC=0`	37 / 5604	35 / 5599	`auxiliary-evacuation-moves-butterflies.js` (it evacuates explicitly and finds that the testing mode already did), `int8-repeat-in-then-out-of-bounds.js` (flaky everywhere)
testing mode + `--useGenerationalGC=0 --collectContinuously=1`	37 / 5604	36 / 5598	`auxiliary-evacuation-moves-butterflies.js`
testing mode + `--slowPathAllocsBetweenGCs=20 --useConcurrentJIT=0`	35 + 2 timeouts / 5604	34 / 5600	the three tests that evacuate explicitly (moves-butterflies, pins-storage-in-use, leaves-unevacuated-blocks-usable: a collection + evacuation lands between their steps); timeouts (300 s): `async-from-sync-throw-closes-and-rejects.js` (85 s on `main` in this mode), `marked-block-decommitted-pages-are-not-read.js`: the mode is ten times slower by construction

JSTests/stress, debug + ASan, every fourth file (1,458):

mode	this branch	`main`, same mode without the testing option	only failing here
testing mode + `--useGenerationalGC=0`	12 / 1393	12 / 1393	none (same 12)
testing mode + `--useGenerationalGC=0 --collectContinuously=1`	11 + 2 timeouts (1,200 s) / 1392	11 + 1 timeout / 1393	no failure; one more timeout, `array-shift-intrinsic.js` (466 s under ASan + collectContinuously without the option; release 2.7 s -> 26 s, passes)

(The first version of this commit had two more tests at that limit in this mode, regress-158411.js and
sampling-profiler-bound-function-name.js; with the cheaper walk they finish.)

On top of #628's branch with the shrinkFootprintNow hook (the follow-up below): all 18 run lines of the seven tests pass;
release default 35 / 5624 (base 36 / 5615), testing mode + useGenerationalGC=0 36 / 5623 (35 / 5616: only
moves-butterflies), + collectContinuously 38 / 5621 (37 / 5614: moves-butterflies and codeblock-aging-ftl-idle.js, a timing
test that fails 5/20 on the base shell in that mode).

Not done / open questions

Prototype: nothing calls it by default, single-threaded VMs only (a second registered thread disables it, and the result
says so), Auxiliary subspace only. The other large holes in live pages after two sessions (JSCell 12.0 MB, JSArray 2.4,
JSFunction 2.4, JSRopeString 1.4, ImmutableButterfly 1.2 of ~33 MB) are cells with arbitrarily many referrers and are out
of reach of this technique.
Typed-array vectors (Gigacage / primitive subspace) are never moved: DFG bakes their address.
Blocking is per block, not per byte. The three kinds of non-butterfly cells that share the Auxiliary subspace
(JSPropertyNameEnumerator buffers, ScopedArguments overflow storage, StructureChain vectors) are 1 % of its bytes on the
measured application, but each one keeps its whole block in place: a for-in over many shapes leaves 60 % of the candidates
unevacuated (464 candidates, 185 evacuated in the reviewer's measurement). Giving them their own CompleteSubspace is a
separate small change and not in this PR.
The pointer lookup is still this function's own copy of the arithmetic in ConservativeRoots::genericAddPointer (two
probes and the owner table), and the set of scanned areas is assembled by hand to mirror gatherStackRoots /
gatherVMRoots / the PinballCompletion constraint. Factoring one forEachCellPossiblyReferencedBy(pointer) out of
genericAddPointer and driving both from the same gatherers with a custom sink would make it impossible for a future
root to be missed; it means restructuring the hot conservative-scan path (its asserts on worldIsStopped() and
isMarking(), its mark hooks, its per-collection precise-allocation snapshot) and was left for a follow-up.
Cost: one pass over block headers, one walk over live objects, one over the stack; proportional to the live heap, not to
what moves (about 10 ns per live cell before this round; the mark-bit iteration removes the per-cell isLive() that was
over half of it). Still not grouped: the directory bit-vector lock is taken per candidate block. There is no explicit
cost/benefit threshold beyond "a size class must free at least one block". Acceptable once per deep idle, not per
collection.
Parking the compiler threads is the one mechanism without a test that fails when it is removed (the detector runs
compiles of functions that fold loads from constant objects during 120 evacuations that move their storage and, with
parking removed, passed 10 out of 10 times; without the
testing option a stale read from the intact old copy is harmless by construction).
The three tests that call $vm.evacuateAuxiliaryBlocks themselves assume that nothing else evacuates in between, so
they fail when the whole suite is run with the testing option on.
The hook into VM::shrinkFootprintNow and the embedder glue (calling the shrink at deep idle) are not part of this PR.

Follow-up: the hook into VM::shrinkFootprintNow

Separate branch claude/gc-evacuate-on-shrink-footprint (needs #628 for VM::shrinkFootprintNow and this PR for the
mechanism): 2 lines in shrinkFootprintNow, Options::evacuateSparseAuxiliaryBlocksWhenShrinkingFootprint (default off) and
auxiliaryEvacuationMaximumOccupancy (0.5), and the fourth test auxiliary-evacuation-on-shrink-footprint.js. Text:
wk-evacuate-on-shrink-footprint.md. That is the configuration the -6.8 MB was measured in.