Cloud: recover frozen machines with provider restart and health alerts

Merged
AU
23d ago
+998-84

Summary

This PR keeps Cloud recovery truthful when the Freestyle control plane has no documented memory-discarding reboot operation. The driver no longer calls an undocumented endpoint or presents pause/start as a hard restart; restart is capability-gated and returns a typed unsupported result until a provider contract can verify completion. Guest heartbeat health is classified separately from provider lifecycle state, and alert snapshots use one consistent database read.

The change also carries restart capability metadata through the web API, macOS, CLI/socket, and iOS recovery paths; invalidates the iOS cached connection after a successful supported restart; preserves automatic restart on restore; and keeps snapshot, fork, and restore behavior intact. The latest origin/main is merged without force-pushing.

Related: #20256

Testing

  • Focused Bun suites: 355 passed, 86 database-dependent skipped, 0 failed across capability, provider, workflow, alert, route, and deadline tests.
  • bun run typecheck and bun run lint:complexity: passed.
  • python3 scripts/verify-local.py: 8/8 applicable checks passed, including Swift syntax, catalog structure/parity, localization defaults, app-source wiring, test wiring, package groups, and feature flags.
  • git diff --check: passed.
  • Final exact head: 218183bd28e2dde309e27937dfdcd9f9a873a91e.
  • Conflict-only merge gate against origin/main aad835069de78d76a43820c2821248e602bca727: passed.
  • Hosted CI for this head: web, package, simulator build, guards, static checks, and migrations passed; macOS compile and iOS simulator jobs were still running at handoff. The Cloud task-local lifecycle job failed during runner setup (Set up runner, cmux-austin-mini-1-glaeda-1) before checkout; no code step ran.
  • Tagged fleet build pr-20361-frozen-cloud-v6 job 9ceae6ed16c3bbae7f51666b is queued for this exact head; its approved wait is still in progress.

Changelog

  • Changed: Cloud recovery reports unsupported when the provider cannot perform a verified hard reboot, while guest health and connection state remain truthful.

Review notes

  • Freestyle documentation and the pinned SDK expose pause/start but no memory-discarding restart operation. Pause freezes memory and start resumes it, so this PR fails closed until the provider contract supports a completed, retry-safe hard reboot. A disposable provider VM was not created because ~/.secrets/cmux.env and FREESTYLE_API_KEY are unavailable; no live Cloud machine was mutated.
  • The merged link recovery from #20376 is already present in current main. Its timed-out CloudMachineLink child reap, cancellation handling, stale inventory guards, and bounded registry reopen were verified read-only; this PR adds no duplicate cmux-tui implementation.
  • All existing review threads were resolved before this repair pass. The PR remains open and unmerged pending the infrastructure and provider-contract blockers above.