#28015 acpmux: per-session usage (usageStats) in ls --wide, --json and session info

Leo (2026-10-30): "that visibility into [one fleet host's] guys was (and is?) pretty bad i feel like we can make more improvements to acpmux/cmux-next", and on acpmux ls: "and acpmux ls should probably not be overly... verbose? i guess" / "to keep everything efficient... tho idk".

Part of HQ#1920 (agent metrics), HQ#5226, and the HQ#5123 telemetry lane. As that design says, a field acpmux cannot read is unknown (null in --json, ? in --wide), never a synthesized zero.

This PR is the data underneath. Four deputies on one fleet host ran plain claude with no autocompact cap for hours, at 290-330k tokens of context per call and 67-90 Opus subagents each. Nothing in acpmux showed it; finding it took a hand audit of ps lines and jsonl files. Now every session row says what its harness spends and holds, read from the harness's own records. The cmux NEXT views (#27923) and the runaway warning (#27899) build on these fields.

What changes

• SessionSummary.usageStats (every _acpmux/sessions row, _acpmux/info, session_changed, and peer rows, since a peer sends its own summary):
  {launcher, autocompact, autocompactCap, contextTokens, contextWindow, compactions24h, subagentsLive, subagents1h, subagents24h, tokens24h, costUsd24h, updatedAt}.
  • Unknown is null, never 0. The 24h counts, live subagents and cost stay null until the reads cover the whole last day: a first read cut short after the day began, a backlog left for the next read, or a file it could not read. A chat whose transcript is not found (or a harness acpmux doesn't read) has no usageStats at all.
  • launcher: the executable the harness runs, by file name (claude, claude-sr2, codex-acp); null when this daemon didn't start it (an adopted, watched or revived chat). harness (the profile), model and effort are the existing summary fields.
  • autocompact: capped | none | unknown, with autocompactCap holding the cap's tokens when known. Claude Code reads --autocompact, else CLAUDE_CODE_AUTO_COMPACT_WINDOW, else CLAUDE_AUTOCOMPACT_PCT_OVERRIDE of the window, else autoCompactWindow in settings.json. Codex reads -c model_auto_compact_token_limit=N, else that key in config.toml.
    • none (nothing caps it) is reported only when acpmux started the harness and could read everything that decides it.
    • unknown covers an argv this daemon never ran (adopted, watched, revived; HQ#5266) and a value or settings file it can't read. #27899 warns about no cap only on none.
  • contextTokens: the last main-loop call's input + cache read + cache write.
  • Spend comes from Claude Code's transcript usage blocks (one per message.id), its compact_boundary lines, and its subagents' own transcripts (<sid>/subagents/agent-*.jsonl, live until .meta.json says finished). For Codex it comes from the rollout's token_count (last_token_usage, model_context_window), compacted items and spawn_agent calls.
  • costUsd24h is those tokens at the model catalog's list prices (1-hour cache writes at 2x input); null when no model has a price.
• acpmux ls (#28046's compact view) prints exactly what it did before, and the daemon reads nothing for it. acpmux ls --wide adds LAUNCHER CAP CTX CMP SUBS(live/24h) TOK24H $24H to #28046's wide table, right before LAST; unknown prints ?, and CAP prints the tokens, none, capped or ?. --wide, --json and --fields usageStats… send usage: true.
• _acpmux/sessions {usage: true} reads each chat's records first: one pass at most every 5 s (sooner when a turn ended since), waited for at most 750 ms. Without it, rows carry the last pass's values.
• A turn's end reads only its own chat and appends a mux usage_stats record, which watchers hear as session_changed kind usage_stats. That keeps the sidebar current without anyone listing. The record is bookkeeping: it never counts as activity, so it doesn't keep an idle chat from hibernating.
• Every metric has an action, in the sibling PRs:
  • autocompact none, or a large contextTokens: #27899 warns (sidebar badge and ls WARN), and the fix is to relaunch with a cap (claude-sr2).
  • A spike in subagents1h: warn, then look at the lane.
  • costUsd24h and tokens24h: #27923's Session Info and fleet view, for the spend audit that found those deputies.
• Protocol IR, docs/acpmux-schema.json, and webviews' generated types.ts are regenerated. README and the acpmux skill are updated.

Cost: reads, watchers, cleanup

• Incremental: each chat's tracker keeps its file offsets and reads only appended complete lines, at most 64 MiB per file per pass. A first read takes at most the newest 256 MiB.
• Lines older than 24 h are skipped after a byte scan for their timestamp; only the newest one is parsed, for the context. Search uses memchr.
• Spend sits in minute buckets for the last day, so the 1 h and 24 h sums never rescan.
• No watcher, no timer, no thread outlives a pass. A pass runs on one spawn_blocking thread and stops when every listed chat's new lines are read.
• Cleanup: a full pass drops trackers and stats for chats that are gone (purged, or closed more than a day ago). A subagent that finished and was read to its end is never opened again, and subagent files untouched for a day are never read.

Performance

ls_latency is a new ignored hub_integration bench: N fake-Claude chats with real transcripts, one turn each; R _acpmux/sessions calls per mode, each on a new connection, as each acpmux ls makes one. The plain mode is what acpmux ls sends; the usage: true mode is what --wide and --json send. The base build ignores usage, so both its modes are a plain list. Runs were interleaved base/head on one Linux lane VM; each cell is the median over the rounds of each round's p50 or p95. Command: LS_LATENCY_SESSIONS=N LS_LATENCY_RUNS=R cargo test -p acpmux --test hub_integration ls_latency -- --ignored --nocapture.

metric	before	after	delta	harness or run link
acpmux ls (default), 30 sessions, p50	29.1 ms	28.7 ms	-0.4 ms (noise)	ls_latency, N=30, R=50, 4 rounds
acpmux ls (default), 30 sessions, p95	44.3 ms	40.4 ms	-3.9 ms (noise)	same
ls --wide / --json, 30 sessions, p50	29.1 ms	30.6 ms	+1.5 ms (+5%)	same
ls --wide / --json, 30 sessions, p95	44.1 ms	49.0 ms	+4.9 ms	same
acpmux ls (default), 200 sessions, p50	180.5 ms	190.0 ms	+9.5 ms (+5%; base rounds alone spread 138-181 ms)	ls_latency, N=200, R=30, 3 rounds
acpmux ls (default), 200 sessions, p95	293.7 ms	258.4 ms	-35.3 ms (noise)	same
ls --wide / --json, 200 sessions, p50	187.0 ms	190.5 ms	+3.5 ms (+2%)	same
ls --wide / --json, 200 sessions, p95	270.0 ms	296.9 ms	+26.9 ms (+10%)	same
first read, 18 MB Claude transcript + 106 subagent files	702 ms (this PR's first cut)	121 ms	-83%	ACPMUX_USAGE_PROBE=claude:<sid>:<cwd> cargo test -p acpmux --lib usage_probe -- --ignored --nocapture
first read, 22 MB Claude transcript	not read on base	159 ms	new	same
first read, 16 MB Codex rollout	not read on base	69 ms	new	same
incremental read (nothing appended)	not read on base	0.04-0.10 ms	new	same

The default acpmux ls does no new work: it sends no usage, and its rows carry the stats the last pass left. What bounds --wide on many sessions:

• At most one pass every 5 s (sooner only after a turn ended); a list arriving meanwhile reuses it.
• A list waits at most 750 ms for its pass; past that it answers with the last pass's values and the pass finishes on its own thread.
• A pass reads at most 64 MiB of each file, and a first read starts at most 256 MiB from the end.
• An incremental pass costs O(bytes appended since the last pass); lines older than the day are skipped with a byte search, not parsed.
• So a fleet's first --wide against many cold, large transcripts can answer at 750 ms with nulls (unknown, never 0) and be complete on the next call.

Tests (red, then green)

• 1fffe290bc4 adds fork_through::usage_stats::the_session_list_and_info_carry_what_the_chats_transcript_says before the feature. At that commit it fails left: Null, right: 201003 (no usageStats). At head it passes, and it also checks that the second turn adds only its own call and that each turn's end is followed by its usage_stats record.

• watch::a_watched_chats_launcher_and_cap_are_unknown_not_none (end to end): a chat acpmux didn't start reports autocompact: unknown and launcher: null, and its transcript's context is still read.

• Unit tests in usage_stats_tests.rs:

  • unknown, never 0: a first read cut after the day began, and a backlog read in steps, report null counts until the reads cover the day, then the whole count; a transcript that can't be read again is null too;
  • a chat whose transcript is not found has no stats until it is;
  • one count per streamed message, and complete lines only;
  • minute buckets aging out;
  • Claude subagent spawns, spend and live state;
  • Codex rollout found by day, with each token_count counted once;
  • cap precedence and the three states for flag, env and settings (Claude) and argv and config.toml (Codex): auto and no setting are none; an unreadable value or settings file, or no argv of this daemon's, is unknown;
  • an all-old transcript keeps its last real call's context (skips <synthetic> and sidechain lines, and a tool input that names its own timestamp).

• cli::ls_table::tests::wide_usage_cells_say_unknown_as_a_question_mark and wide_prints_every_column.

• Review fixes, each with a test that fails at the commit before it and passes at head:

  • 51d96467: a_first_read_cut_at_a_line_start_keeps_that_line, a_line_being_written_or_too_long_leaves_the_counts_unknown, a_record_that_cannot_be_read_is_unknown_not_zero;
  • 800ce2b4: the untimed <synthetic> and info-null rows of a_record_that_cannot_be_read_is_unknown_not_zero;
  • aec8d5f7: an_untimed_repeat_of_a_counted_record_is_not_lost, plus the table test every_record_kind_is_classified_the_same_timed_or_not, which covers every record kind each parse reads, timed and untimed, through the one classify decision;
  • 8e682b72: the_tracker_applies_every_classification (untimed turn context, a lost first line of a message, an untimed repeat of an old call, an escaped kind), plus new table rows.

Observed / Not run

• Ran on this lane VM:
  • cargo test -p acpmux --lib at head: 1992 pass. 16 fail (tui::* x14, daemon::state_file_tests::managed_unit_start_avoids_a_symlinked_default_home, store::startup_tests::listing_open_chats_does_not_read_their_log_directories). The same 16 fail by name on base e3355039fc7 on this host: a temp-dir permission refusal (mode 775) on this VM.
  • --test hub_integration: 564 pass. 4 fail (session_git_state::*), and the same 4 fail by name on base on this host. The 7 session_hibernate tests that an earlier cut of this PR broke now pass.
  • protocol_ir_is_current, api-compat-check (revision 12 matches), scripts/web-lint.sh and prepush-gate.py all pass.
  • The probe reads real transcripts on this host.
• Not run: a live acpmux ls --wide against the fleet daemon (that needs a release with this change), and the fleet's Rust/Swift CI here. Peer rows carry usageStats as the peer last read it: at its own turn ends, or when someone lists it with usage.
• Base merge: #28046 moved ls into cli/run/ls.rs and cli/ls_table.rs; the usage request now lives in ls.rs and the columns in ls_table.rs (review chain in the PR comments).

No capture: daemon, protocol and CLI text only; the acpmux ls --wide columns are pinned by cli::ls_table::tests::wide_usage_cells_say_unknown_as_a_question_mark, and the UI that shows these fields is #27923's (which carries its own captures).

Review chain

Codex gpt-6-astra (high, fast), each step's receipt in the PR comments:

1. full review at 3e0dd276: BLOCK, 4 findings, fixed in 51d96467;
2. delta 3e0dd276..51d96467: the 4 fixes sound, 1 new finding (untimed uncountable records marked lost), fixed in 800ce2b4;
3. delta 51d96467..800ce2b4: no earlier behaviour changed, 1 new finding (untimed repeats marked lost). Fixed in aec8d5f7 by ONE classify function deciding every record;
4. review at aec8d5f7: 3 findings introduced by it and 3 older ones, fixed or documented in 8e682b72;
5. delta at 8e682b72: see the final receipt.

Changelog

• Added: acpmux ls --wide and usageStats on every session (context, autocompact cap, compactions, subagents, last day's tokens and cost) for Claude Code and Codex sessions.

Risk: normal. The changes are additive protocol fields and one new opt-in list param; default acpmux ls output is byte-identical.

Agent session: agent session

🤖 Generated with Claude Code