cmux-host: report the machine's workload so a Cloud idle pause never freezes a dev server, build or cron job

Merged
TE
1d ago
+951-15

Why

Leo, on Cloud work: "If I tell you to do something, don't stick it into dev. Put it into prod. There's nobody else using prod."

The Cloud's idle decision looked only at input and running agent turns. A machine with a dev server, a build or a cron job an agent left running read as idle, and an idle pause would freeze it. This PR makes the guest report what it runs, so the platform can call a machine idle only when nothing is running (the platform side follows in manaflow-ai/cmux-platform).

What changed

cmux-host (the Cloud agent, cloud/workload.rs) probes the machine every 30 s and adds activity.workload to cloud.vm.status.report, with the daemon capability activity-workload while the probe reads:

  • busy_processes: processes above 2% of one CPU between two probes. Reaped children count toward their parent, so a make whose compilers each live a second still reads busy. A process new since the last probe counts all its ticks; pid reuse is caught by the start time.
  • listening_ports: TCP sockets in LISTEN (IPv4 and IPv6, loopback included). A socket with no owner found counts.
  • timers: user crontabs, /etc/cron.d entries, at jobs, transient (systemd-run) timers, and timer units under /etc/systemd/system or a user's ~/.config/systemd/user (masked units skipped).
  • Each count carries at most 8 names: a process's 15-character comm, a port number, or a timer name. Never a command line, so no secrets travel. last_busy_at is the last probe that found the machine busy, so the idle period starts no earlier.
  • System work never counts, by documented allowlists (SYSTEM_PROCESSES, SYSTEM_PROCESS_PREFIXES, SYSTEM_TIMERS, SYSTEM_CRON_D): kernel threads, systemd and its daemons, sshd, cron and atd, journald, dbus, cmux-host, the terminal daemon and its hosts, acpmux, machine health, the image's desktop and the distribution's housekeeping timers.
  • A probe that cannot read /proc makes the workload unknown. The report then drops activity-workload, and the platform treats the machine as busy.
  • A report goes out when the machine turns busy or quiet, when its ports or timers change, or when a quiet period starts again. A change in which process is using CPU while the machine is busy waits for the next report, so a busy machine does not report on every probe.

The block is an extra key in activity (whose schema allows additional properties), like phases and last_person_prompt_at, so the op catalog and the generated clients are unchanged.

Changelog

  • Fixed: a Cloud machine running a dev server, a build or a scheduled job is no longer read as idle.

Tests

cargo test -p cmux-host --lib cloud:: (fleet): new workload_tests.rs (stat and /proc/net/tcp parsing, the CPU threshold, pid reuse, the allowlists, the timers reader under a fake root with crontabs, systemd units, masked units and at jobs) and a session.rs test for the report and capability rules.

Leaned on

Agent session: agent session

๐Ÿค– Generated with Claude Code


Autofix with [code]smith
Open in [code]smith ยท Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Stops the Cloud's idle pause from freezing a machine that's actually working: cmux-host now probes the machine every 30 s and reports busy processes, listening ports, and user timers as activity.workload under the activity-workload capability, so a dev server, build, or cron job keeps the machine awake.

  • Busy processes are those using more than 2% of one CPU between probes (reaped children count toward their parent), plus TCP LISTEN sockets and user timers (cron, cron.d, at, systemd timers, including systemd-run --user). System work is excluded via documented allowlists.
  • Only names are reported (process comm, port number, timer name), never command lines; anything unreadable in /proc โ€” a listener table or a process stat โ€” marks the workload unknown and drops the capability, so the platform treats the machine as busy. Only an exited process or a missing IPv6 table is skipped; IPv4's listener table is required.
  • last_busy_at is monotonic, so a stepped-back clock never lengthens the idle period. Reports go out when the machine turns busy or quiet or when ports or timers change, not on every probe.