Add runtime stats
This introduces a runtime-stats subsystem that coordinates per-process vCPU exit accounting, shim counter snapshots, and syscall histograms under a single env-var gate (ELFUSE_RUNTIME_STATS). Modes: summary (stderr table), json (one JSON object on exit), jsonl (one JSON object per interval or signal). ELFUSE_STATS_SIGNAL=USR1 delivers a snapshot on demand; final dumps suppress subsequent signal dumps via rt_final_done.
Fix a recorder-guard race in syscall_hist_record: the old code entered the hist_active_recorders guard inside record() itself, leaving a window between syscall_hist_now_ns() and record() where the dump could flip mode to OFF, observe guard==0, and read the table before the in-flight recorder landed its updates. Replace the enter/record split with syscall_hist_enter() which does a cheap disabled-mode probe then increments the guard and rechecks mode before returning the start timestamp. record() takes (nr, start_ns, end_ns) and owns the guard release on every exit path including clock-failure.
Fix unserialized concurrent dumps: runtime_stats_dump() now holds rt_dump_mutex for the duration of each dump so JSON/human output from a SIGUSR1 handler cannot interleave with the process-exit dump.
Fix impossible JSON ratios in proc_dump_vcpu_exit_stats_json: load vcpu_exit_total last to maximize the denominator under concurrent increments, then clamp the null-exit numerator to total so null_exit_share stays in [0, 100].
Fix duplicate "reserved" JSON keys in shim_globals_counters_dump_json: unnamed counter slots now emit "reserved12", "reserved13", etc.
Fix silent counter wrap in hist_snapshot_rows and cost-bucket sums: use saturating add (sat_add_u64) for total_count, total_ns, and all inter-bucket sums including process_lifecycle_ns.
Add proc_reset_vcpu_exit_stats, syscall_hist_reset, and runtime_stats_reset_baseline so fork children start with clean counters rather than inheriting the parent's recording window.
Summary by cubic
Adds an optional runtime stats subsystem that aggregates shim counters, vCPU exit reasons, and a syscall histogram into per-process reports, gated by ELFUSE_RUNTIME_STATS with summary, JSON, and JSONL modes plus optional USR1 snapshots. Also adds steady-state syscall recording, fork-child accounting, and a converter to viewer formats.
-
New Features
- Runtime stats coordinator aggregates shim counters, vCPU exit stats, and syscall histograms; modes: summary/json/jsonl;
ELFUSE_STATS_SIGNAL=USR1triggers snapshots; JSON includes schema, pid, argv, wall_ns, shim, vmexit, syscalls, and phase/cost buckets. - Steady-state syscall histogram via
ELFUSE_STARTUP_TRACE=syscalls-steadyor whenELFUSE_RUNTIME_STATSis on; each fork child keeps its own counters and emits its own final record. - vCPU exit accounting tracks totals, VTIMER masks, spurious cancels, and null-exit share.
scripts/runtime-stats-convert.pyturns JSONL output into folded, speedscope, perfetto, and csv; docs updated indocs/usage.md.- Added
test-runtime-statsto validate output shapes and signal dumps.
- Runtime stats coordinator aggregates shim counters, vCPU exit stats, and syscall histograms; modes: summary/json/jsonl;
-
Bug Fixes
- Removed recorder-guard race by splitting
syscall_hist_enter()andsyscall_hist_record(); guard always releases, even on clock failure. - Serialized concurrent dumps with a mutex; final dump suppresses later signals; JSON mode drops non-final signal dumps.
- Corrected vCPU-exit JSON ratios by loading totals last and clamping; used saturating adds for histogram totals and cost buckets.
- Shim counters JSON now emits unique
reservedNkeys for unnamed slots.
- Removed recorder-guard race by splitting