Evict code a parked process does not use: idle by allocation, module graph at 2 min, the executable's own pages and rebuildable bytecode at 10 min
Contains #42329's commits until that lands (it makes main's idle collections one ladder and stops Worker VMs from running
it; the three commits here change its controller). The base is main; once #42329 is merged this
rebases down to its own three commits, which are the last three on the branch. The WebKit it needs (Heap::totalBytesAllocated(),
VM::shrinkFootprintNow(flags)) is on main since #42822 (Heap::allocationBudgetThisCycle(), which the first commit reads, is
not read any more after the third).
GC controller: a program is idle when it has stopped allocating, not when its heap has stopped growing.GC controller: in a standalone executable the last idle rung pages out its own code and drops what JSC can decode again
(the page-out of the executable image used to be in #42329; it is here because it needs commit 1 to be safe for a program that
is serving).GC controller: loud is measured against what the program itself allocates when it works(the line the first commit drew was
a share of the collector's budget; this replaces it).
The GC controller: one signal, and what the last rung evicts
#42329 makes main's idle work one ladder: BUN_IDLE_GC_SECONDS (default 10,110,480) lists the rungs, the controller
remembers when a tick last saw the program busy and how many rungs have run since, and a busy tick starts over. Busy there is
main's "the heap grew by more than 2 MB", which cannot tell a server from a parked program: a server's heap does not grow, and
a program that works hard on a flat footprint reads as quiet. This PR replaces that one term and then, on top of it, adds what
would hurt a working program if it were mistaken for a parked one. When collections run is otherwise what main and #42329
do: the ladder's requested idle collections and the collection the timer requests on every tick. An earlier state of this PR
also decided a "burst has ended" collection from the collector's budget and kept a byte bucket for the rungs; measured against
main on a pausing server, a spike and WebSocket bursts it made no difference worth its code (the same collections inside
bursts, the same RSS in the gaps, a collection 5 s after a burst where main has one after about 10 s), and the bucket made the
last rung slip to 20 minutes after a session. Both are gone. GarbageCollectionController.rs is 376 lines (main 260, #42329
290).
Loud (first and third commit). The controller reads Heap::totalBytesAllocated() (the cycle in progress included, so there
is no lag of a collection) and keeps one number, working_rate: the highest allocation rate it has seen between two looks at
the program, fading over the ladder's length (the last rung's time: 10 minutes by default) until the ladder has run to its
end. The program is loud when its rate is more than working_rate / 16, and never for less than 2 KB a second, or when a
tick came more than two seconds late (#42329's rule). The rungs run their entries after the last loud look; a loud one puts the
timer on its 1 s tick and 30 ticks in a row that are not loud put it on the 30 s one. Four details:
- A look is a tick. On the 30 s tick the next one is brought forward once the program has allocated 8 MB, to a second after
the last one at the earliest (process_gc_timerre-arms the timer; it judges nothing itself). The tick judges what was
allocated over the time it took. Loud: the timer is on its fast tick. Not loud: it goes back to where it was. So a burst in the
middle of half a minute is judged by its own rate, whatever the line (the server with 800 MB live, whose line is 22 MB a
second, is back on its fast tick within a second of its traffic coming back after a minute of silence), also when the
program parks right after the burst (the look comes from the timer, not from a turn of an event loop that may have nothing to
turn for), and a steady rate reads the same on either tick. (An earlier state judged every window as if it had
been 5 s long, which made a steady rate read six times higher on the slow tick than on the fast one: a trickle between 1/96
and 1/16 of the working rate flipped between loud and quiet every 70 s and never got past the first rung. The application's
job, 1.9 MB in one 30 s window, never reaches 8 MB and is judged as the 64 KB a second it is.) - The yardstick stops fading when the last rung has run. What a program went all the way idle against stays the line
until something is loud against it (which starts the ladder, and the fading, over). Left to fade, the application's job
would have been over the line 17 minutes after a session, and from 34 minutes on every tick would have been loud. - The first tick and the tick after a rung do not judge what was allocated, and run no rung (one flag). What they see is
starting up, or the rung's doing: freed blocks handed out again, and after the last rung the code it dropped decoded again,
0.7 MB in the application, which at its first prompt is more than a sixteenth of anything it has allocated since starting
up. Rungs a second apart in the list therefore run two ticks apart. - A tick reads the clock itself, for lateness, for the sample and for arming. The dispatcher's reading is taken once per
drain, before the callbacks it runs: a timer of the program's that fires in the same drain ahead of the tick and sits in a
synchronous call for seconds left the tick "on time", those seconds idle ones and their allocation one tick's worth. Every
tick is late when it comes late, the two that do not judge included (a synchronous call is not idle time whichever tick
it delays); the timer is armed from the clock again at the end of the tick, so the time a rung's collection took is not time
the next tick is late by. - The time a tick is late by does not fade the yardstick: it fades up to when the tick was due. A synchronous call, a
stopped process or a machine asleep for longer than the ladder took the whole of it away, and what the program allocated
afterwards, its background job, was its normal rate from then on: loud for good, never idle again. - With a debugger attached nothing is due, as without a ladder: the timer goes to its 30 s tick, where it used to fire every
second for a rung that never ran. - An earlier state drew the line at 1/64 of the collector's budget per second. The budget follows the heap, so a server with
300 MB live that allocates 2 MB a second all the time read as parked, and would have had a synchronous collection, its bytecode
dropped and its code paged out while it served. And no fixed line works either, because a server answering "hello" five
times a second allocates 4 KB a second, less than the 60 KB a second of the background job of a parked application. Only the
program's own history tells them apart. The budget is not read any more: the floor was 1/4096 of it, 2 KB a second for a
new heap, and with a constant 2 KB a second every program below is classified as before, so
JSC__VM__allocationBudgetThisCycleis gone.
The consequence, by design: a program with a modest steady allocation at its own normal rate is running, not idle. A small
server at a few requests a second is never evicted; neither is a program that redraws something every second at the rate it has
always allocated at. A program that runs no JS allocates nothing and goes down the ladder on schedule.
What this rule cannot tell apart. "A server that was busy and now serves modest steady traffic" and "an application that
ran a turn and now runs a modest steady job" are the same shape and differ only in the ratio to the peak; 1/16 is where the
line is drawn. The application's job is 1/300 to 1/1500 of a turn's rate. A server whose modest traffic is above a sixteenth of
its peak stays busy; one below it goes down the ladder once (bounded: three collections, the last one synchronous, and its
code read back in from the page cache as it is used) and then, because the yardstick no longer fades, nothing more happens to
it until something is loud against that line. (The fade is working *= 1 - dt/T per look, so e^-1 = 37 % of the peak is left
at the last rung: the line there is 1/43 of the peak. Fading linearly to zero by the last rung's time instead would make the
application's job, 57-63 KB a second against a line of 2 MB a second x (1 - t/600 s), loud at about 582 s, just before the
last rung: it would start the ladder over and the application would never be evicted, which is what this PR is for.)
Starting up is counted, except for what fits in the first tick. A real server's start-up spans several ticks, so its
working rate starts at its start-up peak; low steady traffic after that looks idle for up to ln(peak / 16 / rate) x T, and such
a server goes down the ladder once in its first ten minutes and then stays classified idle until something is loud against the
line it stopped at. The 5-requests-a-second rows below are for a server whose whole start-up fits in the first tick. Not
counting start-up does not work as it stands: at its first prompt the application has nothing else to be measured against, and
its ticks there (3.0, 1.4 and 0.7 KB a second at the three rungs) are no quieter than that server's whole load (3.7 KB a second).
How the measured programs are classified (release build; the 5 / 50 / 2 MB rows re-measured on this state for 66 s each with the default list, the others unchanged from the previous state):
| program | its rate | against | result |
|---|---|---|---|
| hello server, 5 requests a second | 3.7 KB/s, steady | its own 3.9-4.1 KB/s | loud on 66 of 66 ticks under traffic; no rung |
| hello server, 50 requests a second | 35-37 KB/s | its own 36-37 KB/s | loud on 66 of 66 |
| 300 MB live, a steady 2 MB/s | 1.9-2.0 MB/s | its own 2.0 MB/s (budget/64 would be 10 MB/s) | loud on 69 of 69; no rung |
| 800 MB live under 2,000 requests a second | 351 MB/s | its own 351-406 MB/s | loud on every tick of the load; quiet from the first tick after it |
| server whose traffic pauses (20 s on, 40 s off), WebSocket bursts | 27 MB/s, 0.5 MB/s in a burst; ~0 between | the burst's rate | loud exactly during the bursts; first rung 10 s after each |
| large CLI application parked at its first prompt | 0.7-3 KB/s on 1 s ticks, under 0.2 KB/s on 30 s ones | 0.7 MB/s line from its first seconds (11 MB/s on its second tick), 0.25 MB/s where it stops | loud on its second tick only; rungs at 12 s, 2 min, 10 min; nothing after the last |
| the same after a session (60-91 MB/s while it worked) | its own job: 1.9 MB in one 30 s window = 64 KB/s | 2.05 MB/s line (32.8 MB/s / 16, where it stopped fading) | quiet; rungs 12 s, 2 min, 10 min after the session; in a 40-minute run no loud tick after the session and the timer on its 30 s tick from 30 s after the last rung |
| a process that runs no JS | 0 | down the ladder on schedule |
| rung | default | what runs |
|---|---|---|
| 1 | 10 s after the last loud tick | idle full collection (requested) |
| 2 | 2 min | idle full collection (requested) + page-out of a standalone executable's module graph |
| last | 10 min | idle full collection; in a standalone executable JSC first drops the code it can decode again, the collection is synchronous when a page-out follows, and the module graph and the executable's own code and constants are paged out |
The last rung (second commit; the only one of a list of one). All of this is for standalone executables (bun build --compile,
the applications it was built for): a process started as bun file.js gets the requested collection it gets on #42329, nothing
is dropped and nothing is paged out. Before its collection, JSC__VM__shrinkFootprintNow has JSC drop
its parser caches and the unlinked bytecode of functions that have no linked code any more (the earlier rungs' collections
unlinked what had not run) and that it can decode again from the executable's embedded bytecode; the rung's collection frees
it. Nothing that would need a re-parse, and nothing at all with JS on the stack: the binding returns whether it did anything,
and if it did not the rung is not counted and comes again on the next tick. The collection is synchronous when a page-out
follows, because the collector is the last thing that would read the executable back in (0.04 ms for the code drop plus 68 ms
for the collection on a 237 MB JS heap, 348 ms on a 1.0 GB one; measured on the previous state, the code is the same). The
executable is paged out with MADV_PAGEOUT over the program's read-only PT_LOAD segments (bun_sys::page_out_range;
page_out_program_image finds the image with find_loaded_module, which now also returns the program headers, and issues
the calls after that walk has returned, outside the loader lock). While a Worker is
alive the rungs run when they are due (their collections are for this thread's heap; the last one's is only requested then) and
what they could not page out is owed: the first tick that finds no Worker does it, and a loud tick forgets the debt with the rest
of the ladder; BUN_FEATURE_FLAG_DISABLE_STANDALONE_MADVISE turns the page-outs off. The page-out of the
executable used to be in #42329; it is here because it needs the first commit: under the heap-growth rule a server under load was idle.
process.memoryUsage().heapUsed reads Heap::sizeAfterLastCollection(); the HeapObserver that reconstructed it and
JSC__VM__blockBytesAllocated are gone.
A server at scale: what is given back after a burst, and when
Half an LRU Map of objects, half retained 16-512 KiB buffers; 2,000 requests a second for 60 s (JSON parse, LRU churn and a
16-512 KiB scratch buffer per request), then silence. Anonymous RSS in MB, main (f937cf4) and this PR side by side, one run each (this state):
| live | under load avg / max | end | +10 s | +15 s | +20 s | +40 s | +70 s | back at | p50 / p99 | |
|---|---|---|---|---|---|---|---|---|---|---|
| 800 MB | main | 1678 / 2447 | 2447 | 2381 | 803 | 803 | 803 | 802 | +14 s | 0.16 / 11.1 ms |
| 800 MB | this PR | 1677 / 2454 | 2454 | 2386 | 800 | 800 | 799 | 799 | +13 s | 0.16 / 11.2 ms |
| 80 MB | main | 206 / 355 | 237 | 104 | 103 | 103 | 103 | 102 | +1 s | 0.17 / 8.2 ms |
| 80 MB | this PR | 211 / 369 | 230 | 107 | 107 | 107 | 106 | 105 | +1 s | 0.16 / 7.8 ms |
The server is loud on every tick of the load (351 MB a second; its working rate peaks at 406) and quiet from the first tick
after it, so the first rung comes 10 s after the load, as on main. An earlier state of both PRs had the first rung at 30 s,
which is when the timer goes to its 30 s tick: the rung's requested collection then started on time and ended a tick later, and
both held 2.36 GB until 92 s after the traffic had stopped. After any rung the timer now stays on its fast tick for 30 ticks
(#42329 has the details and the test).
The application (release build, --lto=off; RSS in MB, anonymous + file-backed)
| before | first rung | 2 min | 10 min | later | |
|---|---|---|---|---|---|
| parked at its first prompt | 216 (118 + 97) at 10 s | 216 (118 + 97) at 27 s | 143 (101 + 42) at 137 s | 100 (89 + 11) at 622 s | 104 (88 + 15) at 752 s |
| after the 20-request session (two runs) | 426 / 437 | 370 / 363 at +20 s | 232 / 231 (188 / 187 + 44) at +2.5 min | 169 / 168 (158 / 156 + 11) at +10 min | 171 / 170 (155 / 154 + 15 / 16) at +13 min; 183 (155 + 27) at +41 min |
Rungs at 12 s, 2 min and 10 min at the first prompt and 12 s, 2 min and 10 min after the session, in all three runs; nothing
starts the ladder over. The application's own job ten minutes after the session is 1.9 MB in one 30 s window, 64 KB a second
against a line of 2.05 MB a second, and what the last rung dropped being decoded again falls in the tick that only looks. One
of the runs went on for 41 minutes after the session: not one loud tick, no rung after the third, the timer on its 30 s tick
from 30 s after the last rung to the end. (With the first rung at 30 s and the budget-relative line these rows were 213 ->
185 -> 128 -> 93 and 416 / 427 -> 334 / 355 -> 228 / 233 -> 171 / 172.)
Numbers
A large bundled CLI application built as a standalone executable with embedded bytecode, scripted 20-request session against a
local fake API, release builds (--lto=off). Total RSS in MB (anonymous + file-backed in brackets).
(The two tables that follow were measured on an earlier state of this PR with the same rungs at the same times; the table above is this state.)
What each rung gives, default thresholds (2 runs each; the before/30 s/2 min/10 min columns are within a few MB of the
previous state of this PR, 4-5 runs):
| before | after 30 s | after 2 min | after 10 min | after 20 min | |
|---|---|---|---|---|---|
| parked at its first prompt | 215 (118 + 97) | 203-204 (108-109 + 95) | 135-137 (94-95 + 41-42) | 95-96 (87-88 + 8) | 108-109 (87-88 + 21) |
| parked after the 20-request session | 429-437 (325-334 + 103-104) | 363-371 (259-267 + 104) | 238-247 (193-202 + 45) | 175-181 (164-170 + 11) | 182-188 (164-170 + 18) |
After the session the application runs a job of its own ten minutes after the last request, a few seconds before the last rung;
the code that job ran is in use and stays, and over the following minutes it reads 7-13 MB of the executable back in. The last rung
does not come round again (#42329), so that stays until the application has worked and gone idle again: with the repeat the
previous state of this PR was at 161-173 (152-156 + 9-17) after 20 minutes. With the previous thresholds (10,65,65) that state
ended at 168-176 (156-160 + 12-16) 8 minutes after the session, and the state before it (shrink on its own clock, page-out
repeated on demand) at 171-174 (154-157 + 14-18) after the session and 105-108 (89-90 + 16-18) at the prompt. With a preloaded job
that allocates 6 MB every 5 s the application parked at its prompt still went down the ladder (107-109 (96-98 + 11) after 10
minutes; not re-measured on this state, the busy predicate is unchanged).
What coming back costs: the first request after the pause, a fresh process each time, user-mode instructions of that request and
of all 20 (perf stat -I lined up with the request's frames; wall time on a shared machine):
| pause | first request | 20 requests | |
|---|---|---|---|
| 9 s (no rung) | 640-740 ms | 2.78-2.94 G | 19.0-19.4 G |
| 51 s (rung 1) | 800 ms | 2.78 G | 19.7 G |
| 151 s (rungs 1-2) | 1210-1480 ms | 3.31-3.48 G | 19.9-20.1 G |
| 641 s (all three) | 1380-2590 ms | 3.57-3.69 G | 20.1 G |
(1-2 runs each on a machine at load 80-130, so the wall times say little. The previous state of this PR, measured alongside:
614 ms / 2.97 G / 19.4 G with no rung, 1215 ms / 3.29 G / 20.9 G after two, 1458 ms / 3.35 G / 19.9 G after all three: the first
request after the last rung costs 0.2-0.3 G more now, the session as a whole the same.)
The second rung is the one that costs most: about +0.5 G instructions and half a second on the first request. The last one adds
0.2-0.3 G to that.
What the WebKit upgrade gives on its own (fresh prompt, idle, peak, --help) is measured in #42822.
Tests
Release build (bun scripts/build.ts --profile=release --lto=off) on main after #42822.
test/js/bun/gc/gc-controller-cadence.test.ts: 50 tests, all pass on the release build (3 runs, 67 s, and once with
BUN_DESTRUCT_VM_ON_EXIT=1; on the ASAN build CI made of this head, with the lane's environment, 23 pass / 28 skip twice, on the previous head; the three tests added since are for release lanes, one of them also runs on ASAN). The three
tests that expect the executable to stay resident now run their child again whenever its code is gone, not only when it is
gone before a rung could have run: on a loaded machine the kernel takes freshly read pages now and then (one in six runs
here), a page-out that should not be there does it every time. That is far over a file's budget, because what it measures is seconds
of idleness: the "idle release" block (20 s of wall clock) runs its children side by side; the standalone-executable block
(45 s) is one at a time becauseMADV_PAGEOUTskips pages another process maps, and every child there maps the same 100 MB
executable. The file is out of the parallel batch (#42329 put it inparallel-allowlist.json'sexcludeFiles, which is
what the runner reads, and inparallel-denylist.txt, which is whatscripts/update-parallel-allowlist.mjsregenerates the
former from: both are needed for the entry to survive a regeneration). The children are observed
without allocating: a child that allocated a little on every stamp would be a program at its normal rate, so the stamps are
written withfs.writeSyncfrom a buffer the child keeps, and the FTL and bytecode children look at the heap (8 KB a look)
only once their rungs have had their time. No test waits for a time: when the first rung comes depends on how many ticks
starting up is spread over (two on a release build, four on an ASAN one), so the positive ones end a moment after the rungs they
wait for have been seen. Lists of one, two and three (3,2,2,2,1,1: the time the yardstick fades over is seconds
there, not a tick or two) run that many collections and none in the 2.5 s after (a tick that only looks and one that would
run a rung); rungs a second apart in the list run two seconds apart (a second apart when the tick after a rung runs
rungs); malformed lists and0run none; programs that allocate 5 MB and 40 MB a second on a flat heap get none; a 10 ms timer
and a timer that builds a spinner frame every 80 ms do not keep a program from going idle; 4 s in a synchronous call are not
idle time, nor are 3 s that make the tick after a rung late (the ladder starts over; taking that tick for an idle one had
the second rung run on the next), nor 3 s in a call made from a timer that fires in the same turn of the event loop as the
tick, ahead of it (the rung of2ran 7 s in, a second after the call; it is due two seconds after); a process stopped
(SIGSTOP) for 3 s, longer than the ladder of1,1, has both rungs after it goes on, with 1/200 of its peak rate (none
before: the 3 s had taken the whole yardstick away); with--inspectthe timer reaches its slow tick (no collection after the first
4 s; once a second before; on ASAN lanes that child runs withdetect_leaks=0, because LeakSanitizer reports 376 bytes of
the debugger's own start-up at exit and aborts); a burst on the 30 s tick brings the fast tick back, also when it is
over within a second of the last tick and the program parks (one collection in the 1.5 s after it before, a hundred now). The rule, on a 20 ms tick with lists of seconds (release
builds only: the rates mean nothing at a tenth of the speed): after a second at 80 MB a second, 0.5 MB a second goes down
1,1,1and 16 MB a second does not (a 20 ms tick sees a timer's allocation in lumps, twice the rate or none, hence ratios
of 1/160 and 1/5 and not 1/40 and 1/8; and since 37 % of the peak is left at the last rung, nothing above 1/43 can finish a
ladder); after half a second at 320 MB a second, a job of 4 MB a second that runs for 2.5 s out of every 5 does not start
1,1over once it has run to its end (it does, after every job, when the yardstick goes on fading: checked by taking
the condition out); 2 MB a second after half a second at 160, on the slow tick, where it is looked at every 8 MB, gets the
rung of6when it is due; a second of work (6 MB, under what makes the controller look early) in the middle of a slow
tick starts the ladder over; work at 320 MB a second that comes back on the slow tick is loud against a line of 20 MB a
second (looking every 8 MB over at least a second could report 8 MB a second at most: the rung came in the middle of the
work). On the default tick: a server answering five requests a second reaches no rung and the same
server without requests does; a standalone executable with 300 MB live that allocates 2 MB a second gets no collection from
the controller and keeps its code (the large-heap case). A burst's garbage (50 MB alive, 300 MB old-generation garbage) is
given back within seconds of the rung
(#42329). Page-out (Linux release lanes): the module graph goes with the second rung of three and the code does not; the code
goes with the last rung of two and with a list of one; neither goes after1,99999999999, with
BUN_FEATURE_FLAG_DISABLE_STANDALONE_MADVISE=1, or while a Worker is alive (both go right after it has; with a Worker alive
for the whole of a list of three all three collections run); the same executable run as plainbun(BUN_BE_BUN=1) keeps its
code. These stay one at a time:MADV_PAGEOUTskips pages another process maps. It is the test that looks, at
/proc/<pid>/smaps; it tells the child to exit before its deadline, counts a mapping as gone when it has less than a quarter
of what it had, runs a child again when the mapping was gone before the rung in question could have run, and says so when a
child exits before it is ready. On a--compile --bytecodeexecutable the last rung of1,1drops the re-decodable unlinked
code and the results are the same afterwards, the first of1,30does not.
Also on this build:require-cache18/18,bundler_bytecode_portable22/22 andcompile-bytecode-tooling4/4 (both as in
#42822),html-rewriter-leak13 pass / 1 skip,heapStats-mimalloc; clippy onbun_jsc,bun_sysand
bun_standalone_graph,cargo checkforx86_64-pc-windows-msvcandaarch64-apple-darwin(on this state).