fix(cli,docs): run the host under systemd with KillMode=process

Merged
KI
2mo ago
+283-40

What & why

A customer ran superset start in the foreground as superset-host.service under systemd's default KillMode=control-group. When the unit stopped, systemd sent SIGTERM to the whole cgroup. That included the pty-daemon: the daemon is detached into its own session but stays in the same cgroup. The daemon's SIGTERM handler kills every PTY, so each restart killed every terminal and agent. With KillMode=process, only the main process gets the signal, the daemon survives, and the next host adopts its sessions.

This PR makes every systemd setup we ship or recommend use KillMode=process:

  • superset start (foreground) now stops its host when it gets SIGTERM/SIGINT. Before this change the runner turned the signal into an abort, and start returned "Host service stopped" with the host still running. Under KillMode=process that broke systemctl restart: the old host stayed up, and the new superset start found a live manifest PID and exited 0 with "already running". It now SIGTERMs the host child, waits up to 10 s, sends SIGKILL if the host is still running, and then removes the manifest (only if the manifest still belongs to that PID). The pty-daemon never gets a signal. --daemon mode does not change.
  • No leaked or hanging hosts. spawnHostService now SIGTERMs its child if the manifest write fails after the health check. Before, the host kept running and, under KillMode=process, the restart started a second one. The polling in terminateHost now has a deadline, so superset stop cannot hang on a zombie host. stop removes the manifest only if it still names the PID it stopped.
  • The failure hint now recommends Restart=on-failure and KillMode=process.
  • Docs: cli/host-server gets a "Run it under systemd" section with a complete unit and one line on why KillMode=process matters.
  • apps/review-host/setup.sh: the review-host unit adds KillMode=process.

How I tested it

  • New packages/cli/src/commands/start/stopHost.test.ts spawns real child processes. It checks that the helper SIGTERMs the host and clears its manifest, sends SIGKILL to a host that ignores SIGTERM, and leaves alone a manifest that belongs to a different PID.
  • I checked that the tests fail without the fix. With a no-op stopHost (the old behavior), 2 of 3 fail. With the SIGKILL escalation removed, the escalation test times out.
  • bun test in packages/cli: 284 pass, 0 fail.
  • bun run typecheck in packages/cli: pass.
  • bunx biome check on the changed files: clean.
  • Vale (marketing vale.ini) on host-server.mdx: 0 errors, 0 warnings. The 2 suggestions are on lines that were already there.
  • systemd in a privileged Ubuntu 24.04 container, with a stand-in host that spawns a setsid-detached daemon the way DaemonSupervisor does:
    • default KillMode=control-group: systemctl restart replaced the daemon (PID 122 became 131), so every session was lost.
    • KillMode=process: the same daemon PID 164 survived the restart.
    • With KillMode=process, the 1.30.2 CLI made systemctl restart hang about 90 s until systemd killed it, and the old host kept running. This branch restarts in about 0 s.
  • Not tested: the real superset binary under systemd on a Linux box.

Checklist

  • PR title follows conventional commits (type(scope): subject)
  • bun run lint and bun run typecheck pass (CI fails on lint warnings too): ran biome on the changed files and typecheck in packages/cli only
  • "Allow edits from maintainers" is checked on fork PRs

Summary by cubic

Fixes the host server killing terminal daemons when systemd restarts the service, by running the unit with KillMode=process and making foreground superset start stop its host before exiting.

  • Under systemd's default KillMode=control-group, stopping the service sends SIGTERM to the detached pty-daemon too, which kills every terminal and agent.
  • start and stop share a terminateHost helper: SIGTERM, up to 10s wait, then SIGKILL, and it clears the manifest only if it still belongs to that PID. It ignores pid 0 and negative pids that would signal a process group, and bounds the wait so stop can't hang on a zombie host.
  • The liveness watch now follows the command's signal via AbortSignal.any, fixing a hang when SIGTERM arrived while the host was still starting.
  • spawnHostService now SIGTERMs the host when the manifest write fails, so it can't leave a healthy host orphaned.
  • Host docs get a complete systemd unit, noting the daemon outlives the service and that this needs a CLI after 1.30.2; the failure hint recommends KillMode=process; apps/review-host/setup.sh adds KillMode=process to its unit.

Summary by CodeRabbit

  • Documentation
    • Added guidance for running the host as a systemd service, including setup steps and CLI version requirements.
  • Bug Fixes
    • Improved host shutdown and restart behavior, including escalation when a host does not stop promptly.
    • Prevented host shutdown from affecting unrelated processes and preserved host terminals during systemd service stops or restarts.
    • Ensured failed host startup cleans up the spawned process and that shutdown removes a manifest only when it belongs to that host.