flows check never talks to the daemon, so it structurally cannot catch spec/daemon skew

Merged
AG
6d ago
+3797-29

flows check could accept a declarative spec that the installed runtime would
refuse. It now submits the compiled kernel spec to relayflowd validate-spec
when available, preserving the daemon's field and step diagnostic. The validator
creates no run or journal. --against-daemon requires a verdict;
--no-daemon-check keeps local checking available. Both have environment
counterparts, and watch mode forwards the selection.

Pass output names the validating binary or explains why acceptance is unproven;
JSON reports include the same validation status on pass and refusal. Generated
gate indices are mapped against the submitted spec and identified as compiler
lowering issues. SDK callers, run, build and deploy retain local preflight unless
explicitly opting into validation.

Scope follows reviewed-plan.md: this covers declarative YAML/JSON specs. Authored
TypeScript bodies still lower only at execution, so the authored-body case
reported in #489 is not covered. Required mode refuses to claim validation for
those bodies. Older runtimes including 2.0.19 lack this command: auto mode warns,
required mode refuses. The verdict is from the resolved binary, not a different
build already serving a socket. Release-artifact --against-daemon enforcement
and authored-body capability probing remain follow-ons; no workflow files changed.

Verification is not fully green. The complete SDK run below recorded 45 test
failures plus one unhandled error. Its new-flag coverage failure was fixed by
adding both invocations to the existing coverage table, without changing the
assertion; that file's rerun is included. Plugin-extension tests hit ENOSPC while
the first Rust workspace build filled the disk; they passed after task-created
build caches were removed. Remaining failures include bwrap network-namespace
restrictions, Bun 1.3.6 versus expected 1.4.0, live analyzer/worker-output
expectations, and event-await exit-status expectations. No claim is made that
these remaining failures were reproduced on the base commit. The full SDK suite
was not repeated after the invocation-table repair; the minimal-PATH rerun covers
the affected check/CLI suites, not the entire SDK.

The Rust workspace was retried without debug symbols/incremental artifacts after
a disk-full linker crash. The normal ops wrapper initially found no default
private rustup toolchain. SDK test preparation used /tmp/daemon-check-toolchain
with cargo and rustup symlinked to the installed /home/daytona/.cargo and
/home/daytona/.rustup. All build artifacts remained outside the repository.

Literal commands and captured output follow. Full captures, including initial
failures, are in evidence/daemon-check.

sdk-final.txt 
cd packages/sdk && npx vitest run tests/check-daemon-validation.test.ts tests/cli.test.ts tests/cli-watch.test.ts tests/relay-cli-surface.test.ts


 RUN  v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk

 ✓ tests/relay-cli-surface.test.ts (89 tests) 45ms
 ✓ tests/check-daemon-validation.test.ts (20 tests) 2853ms
   ✓ real CLI refuses skew in auto and required modes 982ms
   ✓ real CLI labels daemon and offline passes, including JSON 1363ms
   ✓ real CLI refuses an invalid RELAYFLOWD_BIN in required mode 418ms
 ✓ tests/cli.test.ts (71 tests) 3012ms
   ✓ flows check CLI > binds a checked relative wrapper to the flow directory for worker execution 484ms
   ✓ flows check CLI > resolves a bare PATH-resolved claude with no declared model, in an isolated PATH 558ms
   ✓ flows run/resume CLI over the journal protocol > follows a worker wait past a locally expired lease until the daemon settles it 1040ms
 ✓ tests/cli-watch.test.ts (11 tests) 17841ms
   ✓ flows check --watch > forwards required daemon validation through watch despite the offline test environment 979ms
   ✓ flows check --watch > rechecks syntax errors, clears once, and returns the last refusal on Ctrl-C 1394ms
   ✓ flows check --watch > streams JSON lines without ANSI, recovers after atomic saves, and exits zero after repair 1965ms
   ✓ flows check --watch > coalesces 20 concurrent saves into at most two rechecks 1894ms
   ✓ flows check --watch > watches transitive relative use imports, cycles, and nearest config changes 2492ms
   ✓ flows check --watch > refreshes the import graph and notices missing imports being created 2502ms
   ✓ flows check --watch > reloads authored TypeScript instead of reusing the first imported definition 1864ms
   ✓ flows check --watch > detects a nearer config appearing and falls back after it is deleted 1992ms
   ✓ flows check --watch > keeps watching after the target is deleted and recreated 1981ms
   ✓ flows check --watch > queues changes during a slow check without overlapping checks 773ms

 Test Files  4 passed (4)
      Tests  191 passed (191)
   Start at  14:40:27
   Duration  20.31s (transform 1.35s, setup 40ms, collect 5.28s, tests 23.75s, environment 0ms, prepare 184ms)

Full captured output.

sdk-targeted-final.txt 
cd packages/sdk && npx vitest run tests/check-daemon-validation.test.ts tests/cli.test.ts tests/cli-watch.test.ts


 RUN  v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk

 ✓ tests/cli.test.ts (71 tests) 4358ms
   ✓ flows check CLI > binds a checked relative wrapper to the flow directory for worker execution 945ms
   ✓ flows check CLI > resolves a bare PATH-resolved claude with no declared model, in an isolated PATH 699ms
   ✓ flows run/resume CLI over the journal protocol > follows a worker wait past a locally expired lease until the daemon settles it 1066ms
 ✓ tests/check-daemon-validation.test.ts (20 tests) 4465ms
   ✓ real CLI refuses skew in auto and required modes 1449ms
   ✓ real CLI labels daemon and offline passes, including JSON 2162ms
   ✓ real CLI refuses an invalid RELAYFLOWD_BIN in required mode 616ms
 ✓ tests/cli-watch.test.ts (11 tests) 19871ms
   ✓ flows check --watch > forwards required daemon validation through watch despite the offline test environment 1371ms
   ✓ flows check --watch > rechecks syntax errors, clears once, and returns the last refusal on Ctrl-C 2275ms
   ✓ flows check --watch > streams JSON lines without ANSI, recovers after atomic saves, and exits zero after repair 2472ms
   ✓ flows check --watch > coalesces 20 concurrent saves into at most two rechecks 1927ms
   ✓ flows check --watch > watches transitive relative use imports, cycles, and nearest config changes 2580ms
   ✓ flows check --watch > refreshes the import graph and notices missing imports being created 2559ms
   ✓ flows check --watch > reloads authored TypeScript instead of reusing the first imported definition 1902ms
   ✓ flows check --watch > detects a nearer config appearing and falls back after it is deleted 2014ms
   ✓ flows check --watch > keeps watching after the target is deleted and recreated 1987ms
   ✓ flows check --watch > queues changes during a slow check without overlapping checks 779ms

 Test Files  3 passed (3)
      Tests  102 passed (102)
   Start at  14:30:03
   Duration  22.30s (transform 1.83s, setup 58ms, collect 6.59s, tests 28.69s, environment 0ms, prepare 196ms)

Full captured output.

cli-surface-final.txt 
cd packages/sdk && npx vitest run tests/relay-cli-surface.test.ts


 RUN  v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk

 ✓ tests/relay-cli-surface.test.ts (89 tests) 83ms

 Test Files  1 passed (1)
      Tests  89 passed (89)
   Start at  14:36:15
   Duration  3.18s (transform 1.91s, setup 23ms, collect 2.88s, tests 83ms, environment 0ms, prepare 44ms)

Full captured output.

live-ladder.txt 
cd packages/sdk && RELAYFLOWD_BIN=/tmp/relayflows-daemon-check-target/debug/relayflowd npx vitest run tests/live-kernel.test.ts -t 'validates the canonical ladder'


 RUN  v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk

stdout | tests/live-kernel.test.ts
LIVE_KERNEL relayflowd=/tmp/relayflows-daemon-check-target/debug/relayflowd
LIVE_KERNEL flows=/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/dist/cli.js

 ✓ tests/live-kernel.test.ts (33 tests | 32 skipped) 306ms
   ✓ validates the canonical ladder with the built daemon without running it 303ms

 Test Files  1 passed (1)
      Tests  1 passed | 32 skipped (33)
   Start at  14:31:05
   Duration  4.41s (transform 2.31s, setup 62ms, collect 3.80s, tests 306ms, environment 0ms, prepare 46ms)

Full captured output.

rust-targeted-final.txt 
CARGO_TARGET_DIR=/tmp/relayflows-daemon-check-target CARGO_PROFILE_DEV_DEBUG=0 CARGO_PROFILE_TEST_DEBUG=0 CARGO_INCREMENTAL=0 /home/daytona/.cargo/bin/cargo test --manifest-path kernel/Cargo.toml -p relayflowd --test validate_spec

   Compiling relayflowd v0.1.0 (/home/daytona/.relayflow-v2-supervisor/durable/repository/kernel/relayflowd)
    Finished `test` profile [unoptimized] target(s) in 0.19s
     Running tests/validate_spec.rs (/tmp/relayflows-daemon-check-target/debug/deps/validate_spec-2e0369aa01bb2e19)

running 3 tests
test refuses_unknown_field_with_kernel_location ... ok
test accepts_without_creating_data_directory_or_journal ... ok
test refuses_semantic_errors_and_malformed_json ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

Full captured output.

rust-workspace-retry.txt 
CARGO_TARGET_DIR=/tmp/relayflows-daemon-check-target CARGO_PROFILE_DEV_DEBUG=0 CARGO_PROFILE_TEST_DEBUG=0 CARGO_INCREMENTAL=0 /home/daytona/.cargo/bin/cargo test --manifest-path kernel/Cargo.toml -j 2

test result: ok. 64 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.80s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.09s
test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s
test result: ok. 41 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 37.28s
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s
test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 17.53s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.05s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.09s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.75s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.42s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.08s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.14s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.13s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
test result: ok. 86 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.89s
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 18 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 32 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.28s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Full captured output.

typecheck-final.txt 
cd packages/sdk && npm run typecheck && npm run typecheck:tests


> @relayflows/sdk@2.0.42 typecheck
> tsc --noEmit && tsc -p tsconfig.type-tests.json


> @relayflows/sdk@2.0.42 typecheck:tests
> tsc -p tsconfig.tests.json

Full captured output.

sdk-minimal-path.txt 
cd packages/sdk && PATH=/usr/bin:/bin /usr/local/share/nvm/versions/node/v25.6.0/bin/node node_modules/vitest/vitest.mjs run tests/check-daemon-validation.test.ts tests/cli.test.ts


 RUN  v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk

 ✓ tests/check-daemon-validation.test.ts (20 tests) 3104ms
   ✓ real CLI refuses skew in auto and required modes 1124ms
   ✓ real CLI labels daemon and offline passes, including JSON 1394ms
   ✓ real CLI refuses an invalid RELAYFLOWD_BIN in required mode 420ms
 ✓ tests/cli.test.ts (71 tests) 3794ms
   ✓ flows check CLI > binds a checked relative wrapper to the flow directory for worker execution 1246ms
   ✓ flows check CLI > resolves a bare PATH-resolved claude with no declared model, in an isolated PATH 440ms
   ✓ flows run/resume CLI over the journal protocol > follows a worker wait past a locally expired lease until the daemon settles it 1045ms

 Test Files  2 passed (2)
      Tests  91 passed (91)
   Start at  14:31:06
   Duration  8.70s (transform 3.86s, setup 51ms, collect 9.02s, tests 6.90s, environment 0ms, prepare 210ms)

Full captured output.

package-gate.txt 
node scripts/cli-package-gate.mjs

$ /usr/local/share/nvm/versions/node/v25.6.0/bin/node scripts/pack-release.mjs surface /tmp/flows-cli-package-gXtQFF/tarballs
PACK_OK @relayflows/surface@2.0.42: package/dist/index.js, package/dist/index.d.ts, package/dist/runtime.js, package/dist/runtime.d.ts
$ /usr/local/share/nvm/versions/node/v25.6.0/bin/node scripts/pack-release.mjs sdk /tmp/flows-cli-package-gXtQFF/tarballs
PACK_OK @relayflows/sdk@2.0.42: package/dist/index.js, package/dist/index.d.ts, package/dist/cli.js
$ /usr/local/share/nvm/versions/node/v25.6.0/bin/node scripts/pack-release.mjs relayflows /tmp/flows-cli-package-gXtQFF/tarballs
PACK_OK relayflows@2.0.42: package/bin/flows.js
$ npm install --ignore-scripts --omit=optional --no-audit --no-fund /tmp/flows-cli-package-gXtQFF/tarballs/relayflows-2.0.42.tgz /tmp/flows-cli-package-gXtQFF/tarballs/relayflows-sdk-2.0.42.tgz /tmp/flows-cli-package-gXtQFF/tarballs/relayflows-surface-2.0.42.tgz

added 150 packages in 10s
$ /tmp/flows-cli-package-gXtQFF/local/node_modules/.bin/flows check examples/dependency-upgrade-bot.flow.ts
REQUIRES claude (agent "upgrader")
CHECK PASSED examples/dependency-upgrade-bot.flow.ts [local only: authored body, so daemon acceptance is unproven]
$ /usr/local/share/nvm/versions/node/v25.6.0/bin/node /tmp/flows-cli-package-gXtQFF/local/node_modules/relayflows/bin/flows.js check examples/dependency-upgrade-bot.flow.ts
REQUIRES claude (agent "upgrader")
CHECK PASSED examples/dependency-upgrade-bot.flow.ts [local only: authored body, so daemon acceptance is unproven]
$ npm install -g --prefix /tmp/flows-cli-package-gXtQFF/prefix --ignore-scripts --omit=optional --no-audit --no-fund /tmp/flows-cli-package-gXtQFF/tarballs/relayflows-2.0.42.tgz /tmp/flows-cli-package-gXtQFF/tarballs/relayflows-sdk-2.0.42.tgz /tmp/flows-cli-package-gXtQFF/tarballs/relayflows-surface-2.0.42.tgz

added 294 packages in 8s
$ npm install --ignore-scripts --omit=optional --no-audit --no-fund /tmp/flows-cli-package-gXtQFF/tarballs/relayflows-surface-2.0.42.tgz

added 140 packages in 2s
$ /tmp/flows-cli-package-gXtQFF/prefix/bin/flows check examples/dependency-upgrade-bot.flow.ts
REQUIRES claude (agent "upgrader")
CHECK PASSED examples/dependency-upgrade-bot.flow.ts [local only: authored body, so daemon acceptance is unproven]
$ /usr/local/share/nvm/versions/node/v25.6.0/bin/node /tmp/flows-cli-package-gXtQFF/prefix/lib/node_modules/relayflows/bin/flows.js check examples/dependency-upgrade-bot.flow.ts
REQUIRES claude (agent "upgrader")
CHECK PASSED examples/dependency-upgrade-bot.flow.ts [local only: authored body, so daemon acceptance is unproven]
CLI_PACKAGE_OK: local and global installs

Full captured output.

standalone-build.txt 
node scripts/build-standalone-cli.mjs bun-linux-x64 /tmp/flows-daemon-check

Bundled 824 modules in 64ms

  authored-node.mjs  3.54 MB  (entry point)

 [421ms]  bundle  779 modules
  [25ms] compile  /tmp/flows-daemon-check

Full captured output.

standalone-check.txt 
RELAYFLOWD_BIN=/tmp/stub-relayflowd-check /tmp/flows-daemon-check check --against-daemon --json testdata/hello-deterministic.flow.yaml

WARNING [unprovable_effects] Step "greet" command "echo" resolves, but its effects cannot be proven before execution.
WARNING [unprovable_effects] Step "shout" command "echo" resolves, but its effects cannot be proven before execution.
WARNING [editor_schema_missing] For editor validation, add this first line: # yaml-language-server: $schema=https://schema.relayflows.dev/v0.1/flows.schema.json
{"validation":{"mode":"daemon","binary":"/tmp/stub-relayflowd-check","protocol":0,"specVersion":"0.1.0"},"ok":true,"path":"testdata/hello-deterministic.flow.yaml","projectConfigPath":"/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/flows.json","gates":[{"stepId":"greet","kind":"data","checks":["exit_code","output_contains"],"evaluator":"kernel","preflightable":true,"replayable":true},{"stepId":"shout","kind":"data","checks":["exit_code","output_contains"],"evaluator":"kernel","preflightable":true,"replayable":true}],"resolutions":[],"diagnostics":[{"severity":"warning","kind":"unprovable_effects","stepId":"greet","message":"Step \"greet\" command \"echo\" resolves, but its effects cannot be proven before execution."},{"severity":"warning","kind":"unprovable_effects","stepId":"shout","message":"Step \"shout\" command \"echo\" resolves, but its effects cannot be proven before execution."},{"severity":"warning","kind":"editor_schema_missing","message":"For editor validation, add this first line: # yaml-language-server: $schema=https://schema.relayflows.dev/v0.1/flows.schema.json"}],"requirements":{"integrations":[],"harnesses":[],"harnessUses":[],"mcp":[]}}

Full captured output.

sdk-full.txt 
cd packages/sdk && RELAYFLOWS_TOOLCHAIN_HOME=/tmp/daemon-check-toolchain CARGO_TARGET_DIR=/tmp/relayflows-daemon-check-target npm test

 Test Files  9 failed | 239 passed | 3 skipped (251)
      Tests  45 failed | 3801 passed | 23 skipped (3869)
     Errors  1 error
   Start at  14:24:31
   Duration  655.90s (transform 5.84s, setup 1.28s, collect 83.28s, tests 1770.17s, environment 32ms, prepare 11.03s)

Full captured output.

plugin-extension-retry.txt 
cd packages/sdk && npx vitest run tests/plugin-extension.test.ts


 RUN  v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk

 ✓ tests/plugin-extension.test.ts (94 tests) 428ms

 Test Files  1 passed (1)
      Tests  94 passed (94)
   Start at  14:37:26
   Duration  2.84s (transform 1.47s, setup 26ms, collect 2.21s, tests 428ms, environment 0ms, prepare 44ms)

Full captured output.

Mutation verification used an isolated copy of the current SDK source and build,
so the concurrently running full suite was never exposed to mutated code. The
reproducible mutation script disables the
refusal branch, tests unit and real CLI skew, then restores both files byte for
byte. It separately changes submitted-step indexing back to authored-step
indexing, tests generated and subsequent authored steps, and restores again.
All eight complete captures follow; hashes are in the last capture.

mutation-refusal-unit-mutated.txt 
$ 'node' '/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/vitest/vitest.mjs' 'run' 'tests/check-daemon-validation.test.ts' '-t' 'refuses package-valid daemon-invalid skew through check'
cwd: /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 RUN  v2.1.9 /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 ❯ tests/check-daemon-validation.test.ts (20 tests | 1 failed | 19 skipped) 29ms
   × refuses package-valid daemon-invalid skew through check 28ms
     → expected true to be false // Object.is equality

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  tests/check-daemon-validation.test.ts > refuses package-valid daemon-invalid skew through check
AssertionError: expected true to be false // Object.is equality

- Expected
+ Received

- false
+ true

 ❯ tests/check-daemon-validation.test.ts:31:28
     29|   const result = checkAuthoredFlow(flow, '/tmp/flow.json', undefined,
     30|     { daemonValidation: 'auto', daemonValidationDeps: deps(message) });
     31|   expect(result.report.ok).toBe(false);
       |                            ^
     32|   expect(result.report.diagnostics).toContainEqual({ severity: 'refusa…
     33|     message, errors: [message], stepId: 'first' });

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

 Test Files  1 failed (1)
      Tests  1 failed | 19 skipped (20)
   Start at  14:32:32
   Duration  2.21s (transform 1.29s, setup 19ms, collect 1.94s, tests 29ms, environment 0ms, prepare 45ms)


exit=1
mutation-refusal-cli-mutated.txt 
$ 'node' '/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/vitest/vitest.mjs' 'run' 'tests/check-daemon-validation.test.ts' '-t' 'real CLI refuses skew in auto and required modes'
cwd: /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 RUN  v2.1.9 /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 ❯ tests/check-daemon-validation.test.ts (20 tests | 1 failed | 19 skipped) 496ms
   × real CLI refuses skew in auto and required modes 495ms
     → expected +0 to be 2 // Object.is equality

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  tests/check-daemon-validation.test.ts > real CLI refuses skew in auto and required modes
AssertionError: expected +0 to be 2 // Object.is equality

- Expected
+ Received

- 2
+ 0

 ❯ tests/check-daemon-validation.test.ts:106:27
    104|   for (const flags of [[], ['--against-daemon']]) {
    105|     const result = cli(f.path, f.binary, flags);
    106|     expect(result.status).toBe(2);
       |                           ^
    107|     expect(result.stderr).toContain(`REFUSED [invalid_spec] ${message}…
    108|   }

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

 Test Files  1 failed (1)
      Tests  1 failed | 19 skipped (20)
   Start at  14:32:34
   Duration  2.52s (transform 1.16s, setup 16ms, collect 1.80s, tests 496ms, environment 0ms, prepare 45ms)


exit=1
mutation-refusal-unit-restored.txt 
$ 'node' '/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/vitest/vitest.mjs' 'run' 'tests/check-daemon-validation.test.ts' '-t' 'refuses package-valid daemon-invalid skew through check'
cwd: /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 RUN  v2.1.9 /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 ✓ tests/check-daemon-validation.test.ts (20 tests | 19 skipped) 23ms

 Test Files  1 passed (1)
      Tests  1 passed | 19 skipped (20)
   Start at  14:32:37
   Duration  2.05s (transform 1.15s, setup 35ms, collect 1.80s, tests 23ms, environment 0ms, prepare 43ms)


exit=0
mutation-refusal-cli-restored.txt 
$ 'node' '/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/vitest/vitest.mjs' 'run' 'tests/check-daemon-validation.test.ts' '-t' 'real CLI refuses skew in auto and required modes'
cwd: /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 RUN  v2.1.9 /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 ✓ tests/check-daemon-validation.test.ts (20 tests | 19 skipped) 865ms
   ✓ real CLI refuses skew in auto and required modes 864ms

 Test Files  1 passed (1)
      Tests  1 passed | 19 skipped (20)
   Start at  14:32:39
   Duration  3.03s (transform 1.29s, setup 47ms, collect 1.95s, tests 865ms, environment 0ms, prepare 42ms)


exit=0
mutation-indices-generated-mutated.txt 
$ 'node' '/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/vitest/vitest.mjs' 'run' 'tests/check-daemon-validation.test.ts' '-t' 'maps generated-step indices'
cwd: /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 RUN  v2.1.9 /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 ❯ tests/check-daemon-validation.test.ts (20 tests | 1 failed | 19 skipped) 12ms
   × maps generated-step indices to submitted steps and identifies compiler bugs 11ms
     → expected { severity: 'refusal', …(4) } to match object { stepId: 'first.gate', …(1) }
(3 matching properties omitted from actual)

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  tests/check-daemon-validation.test.ts > maps generated-step indices to submitted steps and identifies compiler bugs
AssertionError: expected { severity: 'refusal', …(4) } to match object { stepId: 'first.gate', …(1) }
(3 matching properties omitted from actual)

- Expected
+ Received

  Object {
    "kind": "invalid_spec",
-   "stepId": "first.gate",
+   "stepId": "second",
  }

 ❯ tests/check-daemon-validation.test.ts:41:33
     39|     verification: { type: 'artifact_exists', path: 'answer.txt' } }, f…
     40|   const result = validateSpecWithDaemon(gated, 'required', deps('unkno…
     41|   expect(result.diagnostics[0]).toMatchObject({ stepId: 'first.gate', …
       |                                 ^
     42|   expect(result.diagnostics[0]?.message).toContain('generated by this …
     43|   expect(result.diagnostics[0]?.message).toContain('Report this rather…

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

 Test Files  1 failed (1)
      Tests  1 failed | 19 skipped (20)
   Start at  14:32:43
   Duration  2.09s (transform 1.18s, setup 14ms, collect 1.86s, tests 12ms, environment 0ms, prepare 84ms)


exit=1
mutation-indices-authored-mutated.txt 
$ 'node' '/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/vitest/vitest.mjs' 'run' 'tests/check-daemon-validation.test.ts' '-t' 'maps authored steps following generated steps'
cwd: /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 RUN  v2.1.9 /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 ❯ tests/check-daemon-validation.test.ts (20 tests | 1 failed | 19 skipped) 24ms
   × maps authored steps following generated steps without blaming the compiler 23ms
     → expected [ { severity: 'warning', …(3) }, …(2) ] to deep equally contain { severity: 'refusal', …(4) }

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  tests/check-daemon-validation.test.ts > maps authored steps following generated steps without blaming the compiler
AssertionError: expected [ { severity: 'warning', …(3) }, …(2) ] to deep equally contain { severity: 'refusal', …(4) }

- Expected: 
Object {
  "errors": Array [
    "unknown field \"nope\" at steps[2]",
  ],
  "kind": "invalid_spec",
  "message": "unknown field \"nope\" at steps[2]",
  "severity": "refusal",
  "stepId": "second",
}

+ Received: 
Array [
  Object {
    "kind": "unprovable_effects",
    "message": "Step \"first\" command \"echo\" resolves, but its effects cannot be proven before execution.",
    "severity": "warning",
    "stepId": "first",
  },
  Object {
    "kind": "unprovable_effects",
    "message": "Step \"second\" command \"echo\" resolves, but its effects cannot be proven before execution.",
    "severity": "warning",
    "stepId": "second",
  },
  Object {
    "errors": Array [
      "unknown field \"nope\" at steps[2]",
    ],
    "kind": "invalid_spec",
    "message": "unknown field \"nope\" at steps[2]",
    "severity": "refusal",
  },
]

 ❯ tests/check-daemon-validation.test.ts:178:38
    176|     { daemonValidation: 'required', daemonValidationDeps: deps('unknow…
    177|   expect(checked.report.ok).toBe(false);
    178|   expect(checked.report.diagnostics).toContainEqual({ severity: 'refus…
       |                                      ^
    179|     message: 'unknown field "nope" at steps[2]', errors: ['unknown fie…
    180| });

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

 Test Files  1 failed (1)
      Tests  1 failed | 19 skipped (20)
   Start at  14:32:45
   Duration  2.04s (transform 1.15s, setup 17ms, collect 1.80s, tests 24ms, environment 0ms, prepare 71ms)


exit=1
mutation-indices-generated-restored.txt 
$ 'node' '/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/vitest/vitest.mjs' 'run' 'tests/check-daemon-validation.test.ts' '-t' 'maps generated-step indices'
cwd: /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 RUN  v2.1.9 /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 ✓ tests/check-daemon-validation.test.ts (20 tests | 19 skipped) 6ms

 Test Files  1 passed (1)
      Tests  1 passed | 19 skipped (20)
   Start at  14:32:47
   Duration  2.55s (transform 1.49s, setup 14ms, collect 2.28s, tests 6ms, environment 0ms, prepare 47ms)


exit=0
mutation-indices-authored-restored.txt 
$ 'node' '/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/vitest/vitest.mjs' 'run' 'tests/check-daemon-validation.test.ts' '-t' 'maps authored steps following generated steps'
cwd: /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 RUN  v2.1.9 /tmp/daemon-check-mutation-avim9yyh/packages/sdk

 ✓ tests/check-daemon-validation.test.ts (20 tests | 19 skipped) 30ms

 Test Files  1 passed (1)
      Tests  1 passed | 19 skipped (20)
   Start at  14:32:50
   Duration  2.38s (transform 1.35s, setup 16ms, collect 2.02s, tests 30ms, environment 0ms, prepare 85ms)


exit=0
mutations.txt 
python3 evidence/daemon-check/mutation-run.py

mutation-refusal-unit-mutated.txt: exit=1
mutation-refusal-cli-mutated.txt: exit=1
mutation-refusal-unit-restored.txt: exit=0
mutation-refusal-cli-restored.txt: exit=0
Restored byte-for-byte: src/daemon-spec-validation.ts sha256=3aac707c344f3b83a8cef39d47df4901635b071b5b34cecb0e06340a479eae4c
Restored byte-for-byte: dist/daemon-spec-validation.js sha256=16ff10bf6f1e3d2b3fe89000f85776d7b84d90f5c8aff81fd1ed3b3382d6f764
mutation-indices-generated-mutated.txt: exit=1
mutation-indices-authored-mutated.txt: exit=1
mutation-indices-generated-restored.txt: exit=0
mutation-indices-authored-restored.txt: exit=0
Restored byte-for-byte: src/daemon-spec-validation.ts sha256=3aac707c344f3b83a8cef39d47df4901635b071b5b34cecb0e06340a479eae4c
Restored byte-for-byte: dist/daemon-spec-validation.js sha256=16ff10bf6f1e3d2b3fe89000f85776d7b84d90f5c8aff81fd1ed3b3382d6f764

Checks

The checks fail on the base commit too, so these failures were not introduced by this change: they come from the repository itself or from the environment the checks ran in. This pull request is a draft until someone looks.

What ran (.relayflow/check.sh) 
#!/bin/sh
# How this repository checks itself, the way CI does, on a fresh machine.
#
# Mirrors, in order:
#   .github/workflows/cloud-runtime-artifact.yml  (the tree-health gate: kernel
#                                                  build + kernel tests + SDK
#                                                  suite + artifact assembly)
#   .github/workflows/schema-publish.yml          (validate job only)
#   .github/workflows/surface-package.yml         (scripts/surface-package-gate.sh)
#   .github/workflows/review-swarm*.yml           (only their secret-free gates)
#
# Deliberately NOT here, because this machine cannot run them honestly:
#   * review-swarm.yml's actual review run. It launches a cloud swarm with
#     CLOUD_API_KEY / RELAY_WORKSPACE_KEY and polls agentrelay.com; no secrets
#     here, and a review verdict is not a test of this tree.
#   * review-swarm-wrapper-guard.yml's swarm-wrapper-guard.sh. Needs GH_TOKEN
#     and a live PR number.
#   * publish.yml and schema-publish.yml's publish/version jobs. They bump
#     versions, publish to npm through OIDC trusted publishing and push tags —
#     a deployment, not a check.
#   * actions/upload-artifact. GitHub-only; the archive is still built and
#     verified below.
#   * The hosted extension sandbox provisioning (apt-get bubblewrap +
#     `sysctl -w kernel.apparmor_restrict_unprivileged_userns=0`). The workflow
#     does that on an ephemeral VM it owns. Here `sysctl -w` is denied, and even
#     with bubblewrap installed a new network namespace cannot bring up its
#     loopback:
#         bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted
#     so the sandbox-executing SDK test files are excluded by name below. That
#     list now includes tests/hosted-extension-protocol.test.ts: eight of its
#     twenty-seven tests exec the sandbox and fail with that same bwrap error,
#     and vitest has no way to skip part of a file, so the other nineteen are
#     lost here too. Everything else in the SDK suite still runs.
set -e

repo_root=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd -P)
cd "$repo_root"

say() { printf '\n=== %s ===\n' "$1"; }

# ---------------------------------------------------------------------------
# Toolchain
# ---------------------------------------------------------------------------
say 'toolchain'
for tool in node npm git tar; do
  command -v "$tool" >/dev/null 2>&1 || {
    echo "check.sh: $tool is required and is not on PATH" >&2
    exit 1
  }
done

# CI gets Rust from dtolnay/rust-toolchain@stable. A fresh machine may ship
# none, so install the same stable toolchain and then use plain `cargo`, which
# is what cloud-runtime-artifact.yml invokes. ops/cargo.sh is deliberately not
# used here: it redirects RUSTUP_HOME to a private toolchain home, so a cargo
# that is already on PATH as a rustup shim has no toolchain to choose — the
# failure that workflow documents on its kernel step.
if ! command -v cargo >/dev/null 2>&1; then
  if [ ! -x "$HOME/.cargo/bin/cargo" ]; then
    command -v curl >/dev/null 2>&1 || {
      echo "check.sh: no cargo and no curl to fetch a toolchain" >&2
      exit 1
    }
    curl --proto '=https' --tlsv1.2 -sSf --max-time 300 https://sh.rustup.rs \
      | sh -s -- -y --default-toolchain stable --profile minimal --no-modify-path
  fi
  PATH="$HOME/.cargo/bin:$PATH"
  export PATH
fi

# CI gets bun from oven-sh/setup-bun@v2, and every workflow mirrored here pins
# it to the same exact version (cloud-runtime-artifact.yml, surface-package.yml,
# schema-publish.yml). The pin is load-bearing rather than cosmetic: the SDK
# suite asserts it, so a machine carrying any other bun fails
# tests/authored-node-runtime.test.ts in beforeAll — "expected '1.3.6' to be
# '1.4.0'" — before a single one of its sixteen tests runs. Install the pinned
# bun into its own home and put it first on PATH, the way the cargo block above
# handles a missing toolchain.
bun_pin='1.4.0'
if [ "$(bun --version 2>/dev/null)" != "$bun_pin" ]; then
  bun_home="$HOME/.bun-$bun_pin"
  if [ ! -x "$bun_home/bin/bun" ]; then
    command -v curl >/dev/null 2>&1 || {
      echo "check.sh: bun $bun_pin is required and no curl is available to fetch it" >&2
      exit 1
    }
    curl -fsSL --max-time 300 https://bun.sh/install \
      | BUN_INSTALL="$bun_home" bash -s "bun-v$bun_pin"
  fi
  PATH="$bun_home/bin:$PATH"
  export PATH
fi

node --version
npm --version
bun --version
cargo --version

# ---------------------------------------------------------------------------
# Dependencies
# ---------------------------------------------------------------------------
say 'install: authoring surface'
# --ignore-scripts: the build is the next line, so nothing is skipped.
( cd packages/surface && bun install --frozen-lockfile --ignore-scripts && bun run build )

say 'install: SDK'
# --ignore-scripts again, or npm runs the file: dependency's prepare before its
# own devDependencies exist.
npm ci --prefix packages/sdk --ignore-scripts
# `npm ci` installs @relayflows/surface from the REGISTRY, not the copy built
# above. Any SDK change importing a symbol that has not been published yet then
# fails typecheck. Override with the local directory; --no-save keeps
# package.json / package-lock.json unchanged. The `./` prefix is load-bearing:
# without it npm reads the path as a GitHub org/repo shorthand.
npm install ./packages/surface --prefix packages/sdk --no-save --ignore-scripts

# workflows/*.flow.ts import @relayflows/surface from the repo root, where a
# fresh checkout has no node_modules at all. Link the same local surface there,
# which is also what `flows check` needs to read a workflow file.
mkdir -p node_modules/@relayflows
ln -sfn ../../packages/sdk/node_modules/@relayflows/surface node_modules/@relayflows/surface
node -e "console.log(require.resolve('@relayflows/surface'))"

# Committed 100755, so git already restores this; the guard is one line and the
# failure it prevents is an opaque EACCES deep inside a preflight test.
[ ! -d testdata/preflight ] || find testdata/preflight -name '*-cli' -type f -exec chmod +x {} +

# The agent-CLI fixtures in testdata/preflight are extensionless ESM scripts
# (`#!/usr/bin/env node`, top-level `import`). Node decides their module type
# from the NEAREST package.json, and this repository deliberately has none — on
# a CI runner the search finds nothing and the fixture loads as ESM. On this
# machine the search walks out of the checkout and finds
# /home/daytona/package.json, which declares "type": "commonjs"; the fixture is
# then loaded as CommonJS and exits 0 having printed nothing, so every worker
# that spawns one journals
#   verification: { gate: "execution", verdict: "fail" }
#   "... exited before completing the relayflows-agent-cli-v1 same-process handshake."
# and seven live-kernel tests fail on an output of null. That is the machine's
# home directory, not the tree: restore the CI module resolution for the
# duration of the run with a fixture-local manifest, and remove it afterwards
# so the working tree is never left dirty. Verified both ways in
# .relayflow/repair-notes.md.
preflight_manifest="$repo_root/testdata/preflight/package.json"
if [ -d testdata/preflight ] && [ ! -e "$preflight_manifest" ]; then
  printf '{"type":"module"}\n' > "$preflight_manifest"
  trap 'rm -f "$preflight_manifest"' EXIT HUP INT TERM
fi

# ---------------------------------------------------------------------------
# Builds and code generation
# ---------------------------------------------------------------------------
say 'build: relayflowd (release)'
( cd kernel && cargo build --locked --release -p relayflowd )

say 'codegen: JSON schema is regenerable and committed'
# schema-publish.yml's validate job: generate, assert the committed file did not
# move, generate again and assert the generator is deterministic.
node scripts/generate-json-schema.mjs
git diff --exit-code -- packages/schema/flows.schema.json
cp packages/schema/flows.schema.json /tmp/flows.schema.first.json
node scripts/generate-json-schema.mjs
diff -q /tmp/flows.schema.first.json packages/schema/flows.schema.json
# The surface's generated provider helpers are checked by
# scripts/check-generated-helpers.mjs, which runs inside the surface package
# gate near the end of this script.

# ---------------------------------------------------------------------------
# Tests
# ---------------------------------------------------------------------------
say 'test: cloud artifact contract'
node --test scripts/cloud-artifact.test.mjs

say 'test: kernel workspace'
( cd kernel && cargo test --workspace )

# This machine has a 10G disk; `cargo test --workspace` leaves a ~3.8G
# kernel/target/debug tree behind, and the SDK suite below needs more scratch
# space than what remains. Each authored-node-runtime case dereferences a whole
# @relayflows/surface install into /tmp, and with ~2.7G free seven tests failed
# with "ENOSPC: no space left on device, mkdtemp '/tmp/authored-node-runtime-…'"
# plus one 5s timeout in authored-root.test.ts. Nothing after this point reads
# the debug artifacts — the suite runs against the release binary through
# RELAYFLOWD_BIN — so reclaim them here. CI's runner has the headroom and does
# no such thing; the cost is that the next run rebuilds them.
rm -rf kernel/target/debug
df -h . | tail -1

say 'test: SDK (typecheck, build, typecheck:tests, vitest)'
# `npm test` expanded, minus test:prep. test:prep shells out to ops/cargo.sh for
# the sole purpose of producing a relayflowd binary; the release build above
# already made one, so point RELAYFLOWD_BIN at it instead of compiling a second
# debug copy through a wrapper that cannot run here. The build is still
# required: several test files fail at collection without packages/sdk/dist.
(
  cd packages/sdk
  npm run typecheck
  npm run build
  npm run typecheck:tests
  # RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 is what the workflow sets, and it is an
  # explicit statement rather than a convenience: live-kernel.test.ts runs one
  # case against the real Claude analyzer and fails closed when it cannot, so
  # **this script is not gate-2 acceptance evidence**. That case says so in its
  # own output. Everything else in the suite still gates.
  #
  # The whole suite, not a list of files — a test added to any other file would
  # never run — except the three files that exec the bubblewrap sandbox, for the
  # reason given in the header.
  RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 \
  RELAYFLOWD_BIN="$repo_root/kernel/target/release/relayflowd" \
    ./node_modules/.bin/vitest run \
      --exclude '**/tests/hosted-extension-isolation.test.ts' \
      --exclude '**/tests/babysitter-native-extension.test.ts' \
      --exclude '**/tests/software-garden-babysitter-composition.test.ts' \
      --exclude '**/tests/hosted-extension-protocol.test.ts'
)

say 'test: schema parity and smoke'
# Imports packages/sdk/src directly, so it needs the SDK install above.
( cd packages/schema && bun run test )

say 'test: standalone CLI, artifact assembly, and artifact smoke'
mkdir -p dist/cloud-artifact-input
node scripts/build-standalone-cli.mjs bun-linux-x64 dist/cloud-artifact-input/flows
node scripts/cloud-artifact.mjs build \
  --relayflowd kernel/target/release/relayflowd \
  --flows-executable dist/cloud-artifact-input/flows \
  --output-dir dist/cloud-artifact \
  --source-commit "$(git rev-parse HEAD)"
archive="$(find dist/cloud-artifact -name '*.tar.gz' -type f -print -quit)"
checksum="$(awk '{print $1}' "$archive.sha256")"
node scripts/cloud-artifact.mjs verify --archive "$archive" --sha256 "$checksum"
mkdir -p dist/cloud-artifact-smoke
tar -xzf "$archive" -C dist/cloud-artifact-smoke
dist/cloud-artifact-smoke/bin/relayflowd --help >/dev/null
flows_output="$(dist/cloud-artifact-smoke/bin/flows check --json testdata/hello-deterministic.flow.yaml)"
printf '%s\n' "$flows_output"
node -e '
  const report = JSON.parse(process.argv[1]);
  if (report.ok !== true) throw new Error("flows smoke report was not ok");
  if (report.path !== "testdata/hello-deterministic.flow.yaml") {
    throw new Error(`flows smoke reported unexpected path: ${report.path}`);
  }
' "$flows_output"

say 'test: surface source, regressions, and packed consumers'
# surface-package.yml. Runs last because it re-installs: it packs the surface
# and reinstalls packages/sdk against the tarball rather than the directory.
# It also carries the surface's own vitest suite and typecheck:regressions.
bash scripts/surface-package-gate.sh

say 'test: review-swarm gate self-tests'
# The secret-free halves of review-swarm.yml / review-swarm-wrapper-guard.yml:
# the gate proving it can still say no, and the two drift checks between the
# pre-swarm runner and the swarm definition. Run against the in-repo copies;
# CI runs them from the PR base, which needs a second checkout.
bash .github/workflows/scripts/swarm-gate.test.sh
sh ops/preswarm-check/lens-parity-check.sh
# PRESWARM_ALLOW_MISSING_CLI=1 as the guard workflow sets it: the CLI-presence
# half of the check belongs to the pre-swarm runner's own environment.
PRESWARM_ALLOW_MISSING_CLI=1 sh ops/preswarm-check/lens-cli-parity-check.sh
if command -v ruby >/dev/null 2>&1; then
  bash .github/workflows/scripts/swarm-definition.test.sh
else
  # swarm-definition.sh parses YAML with Ruby's Psych. CI's ubuntu-latest image
  # ships Ruby; this machine does not, and installing one is not this script's
  # job. The validator is skipped rather than run against a missing interpreter.
  echo 'skip: swarm-definition.test.sh needs ruby, which is not installed here'
fi

say 'all checks passed'
Output on this branch (last 80 lines) 
(pass) structural parity: empty command [0.16ms]
(pass) structural parity: positive timeout [0.15ms]
(pass) structural parity: fractional retry [0.14ms]
(pass) structural parity: negative transport retry [0.13ms]
(pass) structural parity: fractional transport retry [0.09ms]
(pass) structural parity: zero transport retry [0.07ms]
(pass) structural parity: u32 max transport retry [0.06ms]
(pass) structural parity: transport retry above u32 [0.10ms]
(pass) structural parity: wrong step field [0.19ms]
(pass) structural parity: nonzero exit gate [0.43ms]
(pass) structural parity: legacy zero exit gate [3.85ms]
(pass) structural parity: boolean schema [1.54ms]
(pass) structural parity: nested invalid schema [8.79ms]
(pass) structural parity: bad memory budget [0.16ms]
(pass) structural parity: unsafe memory budget [0.09ms]
(pass) structural parity: empty memory query [0.09ms]
(pass) structural parity: unsafe duration [0.10ms]
(pass) structural parity: bad money [0.12ms]
(pass) structural parity: bad input index [0.18ms]
(pass) structural parity: blank input name [0.18ms]
(pass) structural parity: trigger silence budget [0.14ms]
(pass) structural parity: workspace: single grant [0.14ms]
(pass) structural parity: tools.fs: single grant [0.11ms]
(pass) structural parity: workspace: grant array [0.08ms]
(pass) structural parity: tools.fs: grant array [0.07ms]
(pass) structural parity: workspace: non-string item [0.06ms]
(pass) structural parity: tools.fs: non-string item [0.06ms]
(pass) structural parity: workspace: nested array [0.06ms]
(pass) structural parity: tools.fs: nested array [0.07ms]
(pass) structural parity: llm: output object [14.64ms]
(pass) structural parity: llm: boolean output [0.24ms]
(pass) structural parity: llm: output and verification [10.62ms]
(pass) structural parity: llm: exit gate [0.25ms]
(pass) structural parity: llm: trimmed model [0.11ms]
(pass) structural parity: llm: control in model [0.07ms]
(pass) structural parity: agent: output object [12.34ms]
(pass) structural parity: agent: boolean output [0.20ms]
(pass) structural parity: agent: output and verification [9.77ms]
(pass) structural parity: agent: exit gate [0.20ms]
(pass) structural parity: agent: trimmed model [0.11ms]
(pass) structural parity: agent: control in model [0.07ms]
(pass) step examples compile and validate [0.34ms]
(pass) generated schema satisfies the existing bounded-reference rule [6.84ms]
(pass) semantic checks remain explicit runtime responsibilities [0.60ms]
(pass) embedded dialect http://json-schema.org/draft-04/schema# [10.65ms]
(pass) embedded dialect http://json-schema.org/draft-06/schema# [13.62ms]
(pass) embedded dialect http://json-schema.org/draft-07/schema# [11.42ms]
(pass) embedded dialect https://json-schema.org/draft/2019-09/schema [17.79ms]
(pass) embedded dialect https://json-schema.org/draft/2020-12/schema [17.13ms]
(pass) header hint is warning-only, first-line aware, and never edits input [19.17ms]
(pass) canonical surface parity: "repo" [0.28ms]
(pass) canonical surface parity: "/repo/src" [0.11ms]
(pass) canonical surface parity: "pr://github/example" [0.09ms]
(pass) canonical surface parity: "/" [0.11ms]
(pass) canonical surface parity: "pr://" [0.10ms]
(pass) canonical surface parity: "" [0.13ms]
(pass) canonical surface parity: " repo" [0.10ms]
(pass) canonical surface parity: "repo " [0.09ms]
(pass) canonical surface parity: "repo//src" [0.07ms]
(pass) canonical surface parity: "repo/../src" [0.06ms]
(pass) canonical surface parity: "repo/." [0.07ms]
(pass) canonical surface parity: ":/bad//path" [0.08ms]
(pass) named declarations and selected input paths use authoring shapes [50.04ms]

tests/smoke.test.ts:
(pass) all exported spec type nodes have documented definitions [21.99ms]
(pass) regeneration is byte-stable and committed schema has not drifted [467.53ms]
(pass) npm tarball contains only data and documentation with no runtime dependencies [291.04ms]

 85 pass
 0 fail
 4019 expect() calls
Ran 85 tests across 2 files. [1.76s]

=== test: standalone CLI, artifact assembly, and artifact smoke ===
Bundled 810 modules in 73ms

  authored-node.mjs  3.42 MB  (entry point)

Terminated
Output on the base commit (last 80 lines) 

=== toolchain ===
v25.6.0
11.8.0
1.4.0
cargo 1.99.0 (5f94df478 2026-08-27)

=== install: authoring surface ===
/tmp/relayflow-recipe.kR9As8: 104: cd: can't cd to packages/surface

What the repair agent found

Check failures on relayflow/flows-software-garden-774f819f

.relayflow/check.sh failed in the SDK vitest step: 3 files, 15 tests, and a
second run surfaced a fourth cause behind them. None of it came from this
branch's change; every cause is a property of this machine, and three of the
four are now handled inside .relayflow/check.sh (uncommitted, as instructed).
After those four fixes the script passes end to end:

=== all checks passed ===
EXIT=0
 Test Files  246 passed | 1 skipped (247)
      Tests  3772 passed | 4 skipped (3776)

(sh .relayflow/check.sh, captured in full at /tmp/check-rerun2.log; the
working tree is clean afterwards — git status --porcelain prints nothing.)

What follows is what each cause is and how it was established.

1. tests/authored-node-runtime.test.ts — bun version, fixed in check.sh

The file's beforeAll asserts the exact bun CI pins:

AssertionError: expected '1.3.6' to be '1.4.0' // Object.is equality
 ❯ tests/authored-node-runtime.test.ts:18:77

Every workflow check.sh mirrors installs bun through
oven-sh/setup-bun@v2 with bun-version: 1.4.0
(.github/workflows/cloud-runtime-artifact.yml:56, surface-package.yml:38,
schema-publish.yml:41); this machine ships 1.3.6 and check.sh only asserted
that some bun was on PATH. It now installs the pinned bun next to the way the
existing block installs a missing cargo, and puts it first on PATH. Verified:

$ bun_pin=1.4.0; bun_home=$HOME/.bun-$bun_pin
$ curl -fsSL --max-time 300 https://bun.sh/install | BUN_INSTALL="$bun_home" bash -s "bun-v$bun_pin"
$ PATH="$bun_home/bin:$PATH" bun --version
1.4.0

2. tests/live-kernel.test.ts — an ancestor package.json, fixed in check.sh

Seven tests failed with a null step output, e.g.

 FAIL  tests/live-kernel.test.ts > built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo)
+   "output": null,
+     "gate": "execution",
+     "verdict": "fail",

The journalled detail names the cause:

"stderr_tail":"CLI \"…/testdata/preflight/analyze-story-stub-cli\" exited before
completing the relayflows-agent-cli-v1 same-process handshake."

Every one of the seven drives an agent step through an extensionless fixture in
testdata/preflight (analyze-story-*-cli, echo-model-cli,
tick-slot-report-cli). Those fixtures are #!/usr/bin/env node scripts with a
top-level import, and Node decides an extensionless file's module type from
the nearest package.json. The repository has none, so on a CI runner the
search finds nothing and the file loads as ESM. Here the search leaves the
checkout and finds /home/daytona/package.json, which declares
"type": "commonjs" — the fixture then produces no output at all and exits 0,
so the handshake never happens.

Controlled repro, same fixture, only the ancestor manifest differing:

$ printf '{"protocol":"relayflows-agent-cli-v1","instruction":"x"}' | node fixture-probe/no-ancestor/preflight/analyze-story-stub-cli --relayflows-adapter-v1
relayflows-agent-cli-v1
relayflows-agent-cli-v1-execute
{"story_title":"stub","relevance_score":5,"reasoning":"stub agent runtime — deterministic output for gate-2 clause-2 demo"}(exit 0)
$ printf '{"protocol":"relayflows-agent-cli-v1","instruction":"x"}' | node fixture-probe/with-cjs-ancestor/preflight/analyze-story-stub-cli --relayflows-adapter-v1
(exit 0)

(fixture-probe/with-cjs-ancestor/package.json is {"type":"commonjs"}; the
other tree has no ancestor manifest. The fixture and wrapper-session.mjs are
byte-identical copies of the repository's.)

And end to end, by restoring CI's module resolution with one fixture-local
manifest:

$ echo '{"type":"module"}' > testdata/preflight/package.json
$ cd packages/sdk && RELAYFLOWD_BIN=…/kernel/target/release/relayflowd ./node_modules/.bin/vitest run tests/live-kernel.test.ts
 Test Files  1 passed (1)
      Tests  32 passed | 1 skipped (33)

versus the same command without that file: Tests 7 failed | 25 passed | 1 skipped.

check.sh now writes that manifest before the suite and removes it on exit
(trap … EXIT HUP INT TERM), so the working tree is never left dirty. It is
deliberately not committed: in CI the file would be a no-op, and the repo's
fixtures are not what is wrong here — the home directory of this machine is.
A maintainer who wants the fixtures hermetic against any ancestor manifest can
commit that one line; this branch is not the place for it.

3. tests/hosted-extension-protocol.test.ts — bubblewrap, not fixable here

Eight of its 27 tests exec the hosted-extension sandbox and fail with the error
check.sh's own header already documents for three other files:

Error: Hosted extension sandbox exited without a valid completion (exit 1): bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted

Confirmed to be the environment, not the tree:

$ bwrap --unshare-net --dev-bind / / true
bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted
bwrap exit=1

The workflow provisions this on an ephemeral VM it owns
(sysctl -w kernel.apparmor_restrict_unprivileged_userns=0); here sysctl -w
is denied. This file is a fourth instance of an already-documented limit, so it
joins the three --exclude entries, which costs the 19 tests in it that do not
touch the sandbox — vitest cannot skip part of a file. Those 19 and the 8
sandbox tests are unverified on this machine and are verified by CI.

4. Disk exhaustion in the SDK suite — handled in check.sh

With bun pinned, tests/authored-node-runtime.test.ts ran for the first time —
and six of its cases plus one authored-root.test.ts case then failed on space,
not logic:

 FAIL  tests/authored-node-runtime.test.ts > … > refuses manual then rather than reporting terminal success
Error: ENOSPC: no space left on device, mkdtemp '/tmp/authored-node-runtime-XXXXXX'
 ❯ fixture tests/authored-node-runtime.test.ts:44:21

The disk is 10G. cargo test --workspace leaves a 3.8G kernel/target/debug
tree, and each authored-node-runtime case copies a dereferenced
@relayflows/surface install into /tmp. Nothing after the kernel step reads the
debug artifacts — the suite runs against the release binary through
RELAYFLOWD_BIN — so check.sh now removes that tree right after the kernel
tests and prints df -h ., which took the suite from 2.7G of headroom to 6.7G:

overlay          10G  3.4G  6.7G  34% /

The next run rebuilds the debug artifacts; that is the whole cost. CI's runner
has the headroom and needs none of this.

Not an environment issue

Nothing above touches relayflowd validate-spec, flows check --against-daemon, or any file this branch changed. The branch's own gates pass
here: cargo test --release -p relayflowd --test validate_spec (3 passed),
vitest run tests/check-daemon-validation.test.ts (20 passed), and the two
live-daemon cases in tests/live-kernel.test.ts — all of them inside the green
run above as well.

Summary of the (uncommitted) check.sh changes

  1. Install and prefer bun 1.4.0, the version every mirrored workflow pins.
  2. Write testdata/preflight/package.json ({"type":"module"}) for the
    duration of the run and remove it on exit, restoring CI's module resolution
    for the extensionless agent-CLI fixtures.
  3. rm -rf kernel/target/debug after the kernel tests, for disk headroom.
  4. Exclude tests/hosted-extension-protocol.test.ts, the fourth
    bubblewrap-executing SDK test file, beside the three already excluded.

Fixes #502

Summary by cubic

Makes flows check catch spec/daemon skew by submitting the compiled kernel spec to the new relayflowd validate-spec subcommand when a supported binary is available, and refusing specs the installed runtime rejects using the daemon's own field and step diagnostics. Validation is stateless: no run or journal is created, and the data dir is left untouched. The branch also merges main including #621 --explain-warnings, so watch mode now forwards both --against-daemon/--no-daemon-check and --explain-warnings, and check --json reports the new validation field.

  • --against-daemon requires a verdict; --no-daemon-check keeps local checking; both have environment counterparts.
  • Pass output names the validating binary or explains why acceptance is unproven, and --json reports the same validation status.
  • Acceptance requires an accepting envelope with exit 0; a nonzero exit or signal yields abnormal_exit, refusing in required mode and warning in auto mode, while refusals are honored regardless of exit status.
  • Generated gate indices are mapped against the submitted spec and flagged as compiler lowering issues.
  • Declarative YAML/JSON specs only; authored TypeScript bodies still lower at execution, so required mode refuses to claim validation for them.
  • Older runtimes without validate-spec warn in auto mode and refuse in required mode.

[!NOTE]
Medium Risk
Changes the flows check contract and spawns relayflowd for validation, so CLI/SDK skew handling and binary discovery affect whether specs pass locally; runtime execution paths are unchanged unless users opt into daemon validation.

Overview
flows check can now prove declarative YAML/JSON specs against the installed runtime, not just local compile/schema work. After lowering, the CLI pipes the same canonical kernel spec run.start would submit to a new stateless relayflowd validate-spec path (parse + validate, no socket, run, or journal).

Modes and UX: default auto validates when the resolved binary supports it; --against-daemon / FLOWS_CHECK_AGAINST_DAEMON=1 require a verdict (exit 2 on daemon_validation_unavailable); --no-daemon-check / FLOWS_NO_DAEMON_CHECK=1 skip binary discovery. Watch mode forwards the choice. Human CHECK PASSED lines and --json reports gain a validation object (daemon vs local plus binary/protocol/reason). Daemon refusals surface as invalid_spec with step/field mapping; indices on generated gate steps are mapped to submitted steps and labeled as compiler lowering bugs when appropriate.

Scope limits: authored .flow.ts bodies still only get local checks (authored_body); older binaries without validate-spec warn in auto and refuse in required mode. Acceptance requires an OK envelope and exit 0.

Also adds validate_spec module export on the daemon, SDK tests (check-daemon-validation, CLI surface flags), FLOWS_NO_DAEMON_CHECK=1 in vitest so the default suite stays spawn-free, docs in SURFACE.md / EDITOR.md, and evidence/daemon-check capture artifacts.

Reviewed by Cursor Bugbot for commit 02230c5792bd9ccfadb0880ddc4410b63b997634. Bugbot is set up for automated code reviews on this repo. Configure here.