fix(vm): refuse a guest-agent rootfs that does not match the resolved smolvm

Root cause (RCA, verified on cpane)

On a smolvm host, machine create --from <pack> boots the guest agent from AgentManager::default_rootfs_path(), whose priority is SMOLVM_AGENT_ROOTFS env → embedded → <exe_dir>/agent-rootfs → data dir. Preloop always exports SMOLVM_AGENT_ROOTFS for every smolvm child (preloop-vm), choosing the first existing of agent_rootfs_candidates(), which put the data-directory copy first. That copy is only refreshed by preloop update.

An operator upgraded the smolvm prefix by hand to 1.30.1 and never ran preloop update, so the engine kept pinning a pre-1.16 agent. Host smolvm ≥ 1.16 requires the quiesced-shutdown-v1 capability before it will stop a machine and otherwise fails with

running guest agent does not support safe shutdown; VM left unchanged; update the guest agent before retrying

which the runner pool retried, with the same opaque message, for hours.

What changed

New module crates/preloop-vm/src/agent_rootfs.rs owns the guard; preloop-vm, preloop-cli consume it.

• Guard (the check). It reads the rootfs Preloop would export (SMOLVM_AGENT_ROOTFS, else the resolved candidate) and the rootfs that ships next to the resolved smolvm binary (<exe_dir>/agent-rootfs, symlink-following so ~/.local/bin/smolvm lands on the prefix). Match is the sha256 of usr/local/bin/smolvm-agent; when a digest is unavailable the fallback signals are the agent version (stamp, else the boot banner baked into the binary) and the presence of quiesced-shutdown-v1. A refusal is one error naming both rootfs paths, both agent hashes (and versions/capabilities), the resolved smolvm version, and the remedy (preloop update --ensure-runtime, or cp -a <prefix>/agent-rootfs <data dir>/agent-rootfs, and "unset SMOLVM_AGENT_ROOTFS" when the pin is the stale tree). A digest match is not treated as proof of capability either: when the agent Preloop would boot advertises no quiesced-shutdown-v1 while the resolved host is new enough to require it, the boot is refused even though the two trees hold identical bytes — that is what a smolvm upgraded without its rootfs looks like, and it is the same unstoppable pair.
• Where it runs. (a) At engine start, once, before the pool spawns: on refusal the pool is disabled with that one message (the engine keeps serving, so preloop status/recovery still work) instead of retry-looping. (b) SmolVmProvider::sandboxed_command runs the same guard (its verdict cached under a fingerprint of the resolved binary and every candidate rootfs tree, so a runtime repair is picked up without restarting the engine) so every boot-capable path — including the golden bake — fails the first smolvm launch, not hours later. Recovery commands (status/list/stop/delete) are deliberately not gated.
• Preference (the actual fix). The rootfs shipped next to the resolved binary is now a candidate, and on Linux it leads; on every platform a candidate that agrees with it is preferred (in candidate order, so identical trees keep the precedence they had), and when none agrees the shipped one wins. So a stale data-dir copy can no longer outrank a version-matched agent. Layouts with no sibling rootfs keep the old data-dir behavior (macOS untouched in ordering; ~/.smolvm/$XDG_DATA_HOME candidates unchanged).
• preloop update --ensure-runtime. Now checks the agent against the installed smolvm version even when the binary is compatible, and repairs both trees the engine can boot from: the rootfs the engine exports (<data dir>/agent-rootfs, where <data dir> is SMOLVM_DATA_DIR or the effective Preloop home's smolvm, not smolvm's own platform data dir) and the rootfs beside the resolved binary (which the engine prefers). The source is a tree that already holds the host's agent — no network, no unverified bytes — else the matching release archive, checksum-verified, with the version recorded in <data dir>/agent-rootfs/.preloop-agent-version. A sibling whose version the earlier check already proved stale is never copied into place and never stamped as current: that is how the stale agent became permanent. The prefix is resolved from PATH (following symlinks), so a binary installed outside ~/.smolvm is repaired where it lives, and an operator SMOLVM_AGENT_ROOTFS pin that does not hold the host's agent makes the command fail with the remedy instead of reporting a repair nothing boots from.
• Shutdown errors carry the diagnosis. A refused stop/delete now appends [guest-agent diagnostics: …]: the resolved smolvm, the rootfs Preloop exported (with hash/version/capability), and the guest agent's boot version from <machine data dir>/agent-console.log.
• Docs: docs/vm-images.md troubleshooting entry for the exact error text + remedy, a note in docs/vm-substrates.md § Shutdown, and a CHANGELOG entry.

Refuse vs warn — the decision

Prefer the matched rootfs; refuse only what is provable. The resolver auto-corrects to the shipped agent whenever it exists, so a mismatch reaches the refusal in three cases: the operator explicitly pinned a diverging SMOLVM_AGENT_ROOTFS; no shipped rootfs exists and the exported agent lacks quiesced-shutdown-v1 under a host ≥ 1.16; or the two trees agree but the agent they both hold lacks that capability under a host ≥ 1.16. Both are exactly the incident's shape, and the failure mode is not a degraded VM but an unstoppable one plus a retry loop, so failing fast with the fix in the message beats booting. Everything unprovable — an unreadable tree, a host whose version cannot be read — warns and proceeds: a false refusal would take a working engine down, which is worse than the failure being guarded against. Mismatch in the benign direction (host older than the pinned agent, i.e. no capability handshake) is not refused when the host version proves it is pre-1.16.

macOS considerations

• Candidate ordering is unchanged on macOS (data dir → ~/Library/Application Support/smolvm → ~/.smolvm → binary sibling last); only the matched-preference applies, so a leftover ~/.smolvm/agent-rootfs from the official installer can no longer shadow a fresh copy, and the wrapper's own SMOLVM_AGENT_ROOTFS (set before falling back to its sibling) keeps working.
• smolvm --version is probed through the same runtime env the engine exports (DYLD_LIBRARY_PATH for a bare smolvm-bin, wrapper layout unaffected).
• preloop update on macOS installs the runtime into ~/.smolvm and the agent into ~/Library/Application Support/smolvm; the sibling refresh keys on a pre-existing <prefix>/agent-rootfs, so it never creates a duplicate where the official installer did not lay one down.

Tests

• agent_rootfs unit tests: hash/version/capability inspection, digest match/mismatch classification over fixture rootfs trees (with and without the capability string), candidate ordering and sibling preference, message contract (both paths, both hashes, host version, both remedies), refusal on a capability-less agent under a ≥ 1.16 host, identical-but-incapable trees refused on such a host and left alone under 1.15.x, an incomplete candidate never exported over the complete shipped tree, warn-and-run when the host version is unreadable, a replaced rootfs invalidating the cached verdict, and a multi-byte console log not panicking the boot-version reader.
• preloop-vm integration (tests/provider.rs, fake launcher, no VM): a pinned mismatched agent refuses create before any smolvm command runs (asserted from the launcher's invocation log); the binary's own agent boots; a refused stop reports the pinned rootfs and the guest's boot version.
• preloop-cli update tests: the refresh source is only ever a tree that holds the host's agent (a stale pair has none, so the release is required), a stale sibling repaired from the data-dir copy without network, a stale rootfs refreshed from the binary's sibling without network, a matching rootfs left untouched, a stamp whose bytes were replaced still counting as stale, and the install layout test asserting the version stamp plus the sibling refresh.

Review round (2026-10-18)

The 18 open devin/codex/coderabbit threads were triaged and closed. Fixed in the guard: an identical-but-incapable pair is now refused rather than allowed by the digest match; a candidate must be a rootfs (sbin/init) before it can be preferred or exported — which is also the Linux-only rust shard 3 of 4 failure (agent_rootfs::tests::resolution_falls_back_to_the_data_dir_when_the_sibling_is_not_a_rootfs, left: Some(".../agent-rootfs") right: Some(".../data/agent-rootfs")); the console-tail slice is character-boundary safe; the verdict and resolution are cached under a stat fingerprint so a repair is picked up without a restart and the command path stops rehashing whole trees. Fixed in preloop update --ensure-runtime: it repairs every tree the engine can export (the engine's data dir, the binary's sibling, smolvm's own data dir), never installs or stamps a tree it has proven stale, resolves the prefix from PATH through symlinks, and fails loudly on an operator pin that does not hold the host's agent. Fixed at engine start: the guard runs before the server is constructed, so a disabled pool no longer leaves the control plane demanding a provision token nothing will mint, and it reaps the hypervisors the unstoppable machines left behind. Full dispositions are in the thread replies.

Verified

• cargo fmt --all --check — clean.
• cargo clippy --locked -p preloop-vm -p preloop-cli --all-targets -- -D warnings — clean.
• cargo nextest run --locked -p preloop-vm -p preloop-cli — 379 passed, 0 failed (2 excluded, see below).
• cargo nextest run --locked -p preloop-vm --partition hash:3/4 — 32 passed, 0 failed; the shard-3 test above passes.
• Reproduction of the fixed testing gap on a real host: smolvm --version → 1.18.1 with quiesced-shutdown-v1 present in ~/.smolvm/agent-rootfs/usr/local/bin/smolvm-agent and absent in ~/Library/Application Support/smolvm/agent-rootfs/usr/local/bin/smolvm-agent (agent 1.14.2) — the pair the guard refuses.

Not verified

No live smolvm/host VM on this machine, so the guard was exercised only against fixture rootfs trees and fake launchers: the real machine stop refusal text is matched by its documented phrasings, and a real end-to-end run (hand-upgrade the prefix, boot, confirm the engine exports the sibling) still needs a smolvm host.

Two preloop-vm tests are host-layout dependent on macOS: tests::start_failure_with_the_irq_signature_becomes_the_dedicated_variant and tests::start_failure_without_the_signature_keeps_its_command_error (both from #672's IRQ work) build a fake smolvm with no sibling rootfs, so the guard resolves the candidate from the developer's real HOME — on a macOS host with a pre-1.16 agent in ~/Library/Application Support/smolvm/agent-rootfs the guard refuses create before start and the tests fail. They pass with an isolated HOME (HOME=$(mktemp -d) cargo nextest run -p preloop-vm), which is what the Linux CI shards have, and the failure is not introduced by this round: with no sibling rootfs the guard's decision is unchanged. Worth isolating those fixtures (SMOLVM_DATA_DIR in the fixture) in a follow-up.

---

---

Summary by cubic

Fixes the failure where a stale pinned guest-agent rootfs made every VM unstoppable: smolvm ≥ 1.16 refuses machine stop without the quiesced-shutdown-v1 agent capability, and the pool retried the same opaque error for hours. The engine now refuses a provably mismatched rootfs before boot and prefers the rootfs shipped beside the resolved binary.

Guard

• Compares the rootfs Preloop exports with the one shipped next to the resolved binary (agent sha256, falling back to version and capability), refusing a proven mismatch with both paths, both hashes, the host version, and the remedy.
• A matching digest does not skip the capability check, and a candidate must look like a rootfs (sbin/init) before it is selected.
• Runs at engine start before the server exists (the pool is disabled with the message while the engine keeps serving, and leftover hypervisors are reaped) and on the first boot-capable command of each provider; recovery commands are not gated.
• Verdicts are cached under a stat fingerprint, so a repair is picked up without a restart; a cached warning is logged once, not on every boot-capable command. Unprovable cases warn and proceed.

Update and diagnostics

• preloop update --ensure-runtime refreshes <data dir>/agent-rootfs when it does not match the installed smolvm, from the sibling rootfs or the matching release, and never installs or stamps a tree it has proven stale.
• The archive install writes only to ~/.smolvm; the PATH-resolved prefix is tracked separately and never used as an install destination, so a real ~/.local/bin/smolvm no longer becomes a self-symlink. Rootfs refreshes repair the sibling and managed prefix only when they already exist.
• The prefix is resolved from PATH through symlinks, and an operator SMOLVM_AGENT_ROOTFS pin that does not hold the host's agent fails the command with the remedy.
• Refused stops and deletes report the exported rootfs and the agent's boot version from the machine's agent-console.log. Residual-VM cleanup runs only after the engine socket is owned, so a second serve whose guard fails cannot kill a live engine.

Summary by CodeRabbit

• New Features

  • Added guest-agent rootfs compatibility checks for SmolVM, including version, integrity, and shutdown capability validation.
  • preloop update --ensure-runtime now refreshes and repairs available guest-agent rootfs copies.
  • Added clearer diagnostics showing selected rootfs paths, versions, and mismatch details.
  • Engine startup now cleans up orphaned virtual machines and mounts when pool assignment is unavailable.

• Bug Fixes

  • Prevented machines from booting with incompatible guest agents.
  • Improved shutdown and deletion errors with actionable agent and rootfs information.

• Documentation

  • Added troubleshooting guidance for stale or overridden guest-agent rootfs files.