feat(daemon): connection-file handshake and CLI attach-or-spawn

Merged
KH
2mo ago
+3585-55

Summary

Phase 1 of "package flows so it's frictionless": relayflowd currently requires a human to build it and run relayflowd serve by hand before any flows run/flows check/flows resume can connect — the CLI fails closed on a missing socket rather than starting one. This gives relayflowd a connection-file handshake and gives the flows CLI attach-or-spawn logic, ported from ../relay's broker-lifecycle pattern, so flows run x.yaml needs no manual daemon step.

  • Kernel (kernel/relayflowd/src/server/lifecycle.rs): relayflowd serve holds an exclusive flock(2) on <data-dir>/relayflowd.lock for its whole lifetime, only sweeps leftover socket/connection-file residue after acquiring that lock, binds, and only then atomically publishes <data-dir>/connection.json (temp-file + rename). Clean shutdown (SIGTERM/SIGINT) unlinks the file and socket from an async-signal-safe handler; kill -9 leaves the file behind by construction. A second serve on an already-served data dir loses the race and exits 3 without touching anything.
  • SDK: daemon-connection.ts validates connection.json against a live-pid check and a socket hello probe (the socket is always the authority; EPERM correctly counts as alive). daemon-lifecycle.ts is the attach-or-spawn algorithm — attach if something answers, otherwise resolve the binary (relayflowd-path.ts, multi-anchor resolution) and spawn it detached, polling for readiness. Wired into check/run/resume via cli.ts. New --no-spawn / FLOWS_NO_SPAWN=1 restores today's exact fail-closed behavior for CI.
  • docs/SURFACE.md §5 updated — it previously stated "neither verb starts the daemon implicitly," which this PR makes false.
  • Full spec in kernel/DAEMON-LIFECYCLE.md; full build/test/review writeup in ops/DAEMON-LIFECYCLE-REPORT.md.

This was built and verified as workflows/daemon-lifecycle.yaml, an 8-step DAG (design → kernel impl → kernel tests → CLI impl → SDK tests → e2e smoke → adversarial review → report). The first adversarial-review pass failed: kernel/DAEMON-LIFECYCLE.md §6 specifies 6 kernel tests and only 3 existed. Two were missing — the anti-hijack case (a second serve on a live data dir exits 3, first daemon keeps serving) and the SIGKILL-successor case (a killed daemon's successor starts cleanly). Both are now written against the real relayflowd binary and pass.

Test plan

  • cargo test --workspace (kernel) — all green, including the two new real-process daemon-lifecycle tests
  • npm test (packages/sdk) — 803 passed, 3 skipped
  • e2e smoke: cold-start flows run against an empty data dir with no daemon running spawns exactly one daemon and completes
  • Adversarial review against concurrent-spawn races, stale-file false-attach, detached-spawn survival, manual-operator backward compatibility, and publish-ordering — all hold up; see ops/DAEMON-LIFECYCLE-REPORT.md

Not in scope

Phase 2 (per-platform relayflowd/flows binary packages for macOS/Windows, mirroring packages/runtime-linux-x64 which is Linux-only today) is separate follow-up work, noted in the report.

🤖 Generated with Claude Code

https://claude.ai/code/session_013Y8uLRUXqKSZsqeeUMNaS2

Summary by cubic

flows run/check/resume now work without a manually started daemon: the CLI attaches to a relayflowd that's already serving, or spawns one detached and polls until ready, instead of failing closed on a missing socket.

Daemon handshake

  • relayflowd serve holds an exclusive flock for its lifetime; a competing serve on the same data dir exits 3 without touching anything.
  • Publishes <data-dir>/connection.json only after the socket is live, and removes both on SIGTERM/SIGINT; a kill -9 leaves the file behind, which the client tolerates.
  • --no-spawn / FLOWS_NO_SPAWN=1 restores the previous fail-closed behavior for tooling and CI.

Client validation

  • The connection file is never believed alone: it's checked against a live pid and a socket hello probe, where the socket is the authority and EPERM counts as alive.
  • New failure kinds distinguish spawn failure, start timeout, protocol mismatch, and a missing binary; binary resolution tries multiple anchors so version-manager PATHs don't break.
  • JournalClient connects now have a timeout, so a listener that never answers doesn't hold the CLI for the full request timeout.