runner-hook: a signing job waits out an earlier job's leftover processes in any runner's _work

Merged
TE
8d ago
+128-0

A signing job (the exclusive class) is now admitted only when no leftover job process is still working on the mini, not just when every capacity unit is free.

Leo, 2026-10-28, on the release signing jobs running on owned Macs: "ok. idk what the "risk" was before but like.. we can just read the code and stuff." The rule: the job claims the Mac exclusively, so nothing else runs beside it.

Why: cmux's signing picker is about to offer a mini whose offline runners GitHub calls idle. glaeda parks most of a mini's runners offline, and counting those as busy left no mini ever drained. A runner whose listener died mid-job looks the same: offline, not busy, and its units released with its Runner.Worker. But the steps it started can still be running.

Change:

  • After the exclusive class takes every unit, it lists this user's processes that work in (lsof cwd) or name (ps) any runner's _work (actions-runner-glaeda*/_work, glaeda-runner/_work).
  • Excluded from that list: this job's own processes (under its Runner.Worker), its ancestors, and the hook's own processes.
  • If any are left, it lets go of what it took and keeps waiting like a capacity wait, at most EXCLUSIVE_CAPACITY_WAIT_S (300 s). Then it is refused, and the job's re-run takes Blacksmith. When ps or lsof can't tell, it is refused as well.
  • A held host.lock still refuses it (now covered by a test).
  • The hook's new sha256 is pinned in glaeda-disk.

Test: test_a_signing_job_waits_out_an_offline_runners_leftover_job_and_the_host_lock.

  • An orphaned process working in a fake runner's _work gets the job refused, with the leftover's pid named, no GLAEDA_JOB_CLAIM written, and everything it took released.
  • A held host.lock gets the job refused.
  • With neither, the job is admitted with the claim.
  • test_a_signing_job_takes_the_whole_mini still passes.

Agent session: agent session

šŸ¤– Generated with Claude Code