PR #3733 - callstack/agent-device

Summary

Rebased onto origin/main after #3711 extracted src/remote into @agent-device/remote and #3712 moved provider plugins into a private package. The remote diagnostics conflict now targets @agent-device/remote/remote-request-diagnostics; the artifact-directory client test moved beside packages/remote/src/daemon-artifacts.ts.
Rebuilt the provider-scenario harness on startDaemonRuntime, the root composition, the real daemon HTTP server, and a real client. Provider scenarios use the public runtime/client and no daemon testing imports.
Moved the HTTP server provider-scenario test beside packages/daemon/src/server/http; moved the Node concurrency-torture lane and its helpers into packages/daemon/test/nightly.
Added package-local tests for the touch response payload and action recording. Updated command-coverage path evidence. Output-economy and help-conformance now start the real root composition through startDaemonRuntime and use real HTTP RPC responses to exercise the production response views and error producers.
The proxy parity fixture forwards the diagnostics GET used by its missing-ref case and asserts the remote log is materialized. Provider-admission cleanup now closes the daemon, provider runtime, and regression server even if lease release fails.
Removed 15 daemon testing exports from the rebased origin/main manifest (34 total exports → 19; 33 test-only exports → 18); the retained ./testing/context line remains for the final-pin owner.

Placement

(b) Composed behavior: provider scenarios start the public daemon runtime and drive it through a real client or HTTP RPC. They do not inspect daemon state.
(a) Daemon logic: the HTTP server scenario is beside the server module; the touch response and action recorder invariants are tested beside their daemon modules. The concurrency suite is package-local.
(c) Root behavior: command-coverage declarations cite test paths only. Output-economy and help-conformance stay in root and drive the public daemon route through createProviderScenarioHarness (startDaemonRuntime, real HTTP server, real RPC client). Their digest samples come from live route responses; their error samples come from public RPC responses passed through the root CLI error renderer. Neither root gate imports packages/daemon/test, a testing export, or daemon internals. No R11 exception was added.
remote-test-artifacts-directory now tests @agent-device/remote's daemon-artifacts module in packages/remote/src/__tests__, beside the module it exercises. The proxy diagnostics, tenant, parity, and materialization scenarios stay in test/integration/provider-scenarios because they cover composed proxy, daemon, replay-port, and remote-client behavior; the package-local remote tests already cover its individual transfer modules. Provider scenarios remain integration tests around the real daemon and private plugin packages, rather than copies of their package-local runtime tests.
The scripts/layering files were checked as text: model.ts has a comment, while package-boundaries-facade.test.ts and test-kit-isolation.test.ts use fixture strings. None imports daemon test support or calls vi.mock with it.

Test counts

Counts are test cases at the main path and at the head destination. Helper files have no test cases.

Main path	Main	Head path	Head
`test/integration/provider-scenarios/harness.ts`	0	same	0
`test/integration/provider-scenarios/stale-provider-runtime-admission.fixtures.ts`	0	same	0
`test/integration/provider-scenarios/cloud-webdriver-lease-http.test.ts`	1	same	1
`test/integration/provider-scenarios/cloud-webdriver-runtime.test.ts`	7	same	7
`test/integration/provider-scenarios/daemon-http-server.test.ts`	10	`packages/daemon/src/server/http/http-server-provider-scenarios.test.ts`	10
`test/integration/provider-scenarios/ios-fold.test.ts`	2	same	2
`test/integration/provider-scenarios/ios-wait-selector.test.ts`	4	same	4
`test/integration/provider-scenarios/limrun-ios-snapshot-owner.test.ts`	1	same	1
`test/integration/provider-scenarios/remote-proxy-artifact-tenant.test.ts`	2	same	2
`test/integration/provider-scenarios/remote-proxy-parity.test.ts`	5	same	6
`test/integration/provider-scenarios/remote-proxy-request-diagnostics.test.ts`	1	same	1
`test/integration/provider-scenarios/remote-test-artifacts-directory.test.ts`	3	`packages/remote/src/__tests__/daemon-artifacts-directory.test.ts`	3
`test/integration/provider-scenarios/remote-test-artifacts-materialization.test.ts`	2	same	2
`test/integration/provider-scenarios/scroll-until.test.ts`	3	same	3
`test/integration/provider-scenarios/cloud-webdriver-ios-text-entry.test.ts`	5	same	5
`test/integration/provider-scenarios/web-desktop.test.ts`	1	same	1
`test/integration/provider-scenarios/stale-provider-runtime-admission.test.ts`	3	same	3
`test/integration/provider-scenarios/provider-device-runtime.test.ts`	4	same	5
`test/integration/nightly/concurrency-torture/bindings.ts`	0	`packages/daemon/test/nightly/concurrency-torture/bindings.ts`	0
`test/integration/nightly/concurrency-torture/harness.ts`	0	`packages/daemon/test/nightly/concurrency-torture/harness.ts`	0
`test/integration/nightly/concurrency-torture/invariants.ts`	0	`packages/daemon/test/nightly/concurrency-torture/invariants.ts`	0
`test/integration/nightly/concurrency-torture/real-scope-serialization.ts`	0	`packages/daemon/test/nightly/concurrency-torture/real-scope-serialization.ts`	0
`test/integration/command-coverage/declarations.ts`	0	same	0
`test/integration/interaction-contract/native-ref.contract.test.ts`	9	same	9
`test/output-economy/render-fixtures.ts`	0	same	0
`scripts/__tests__/help-conformance-sample-producers.ts`	0	same	0
`test/integration/provider-scenarios/help-conformance-error-routes.ts`	0	same	0
New	0	`test/integration/provider-scenarios/help-conformance-error-routes.test.ts`	1
`test/output-economy/output-economy.baseline.test.ts`	13	same	15
`scripts/layering/model.ts`	0	same, string evidence only	0
`scripts/layering/package-boundaries-facade.test.ts`	5	same, string fixtures only	5
`scripts/layering/test-kit-isolation.test.ts`	12	same, string fixture only	12

Additional moves and split tests:

Main path	Main	Head path	Head
`test/integration/nightly/concurrency-torture.test.ts`	5	`packages/daemon/test/nightly/concurrency-torture.test.ts`	5
`test/integration/nightly/concurrency-torture/claim-registry.ts`	0	`packages/daemon/test/nightly/concurrency-torture/claim-registry.ts`	0
`test/integration/nightly/concurrency-torture/deterministic-scheduler.ts`	0	`packages/daemon/test/nightly/concurrency-torture/deterministic-scheduler.ts`	0
`test/integration/nightly/concurrency-torture/envelope.ts`	0	`packages/daemon/test/nightly/concurrency-torture/envelope.ts`	0
New	0	`packages/test-kit/src/lane-envelope.ts`	0
`test/integration/nightly/concurrency-torture/prng.ts`	0	`packages/daemon/test/nightly/concurrency-torture/prng.ts`	0
New	0	`packages/daemon/test/nightly/concurrency-torture/lane-envelope.ts`	0
Existing	29	`packages/daemon/src/core/__tests__/session-action-recorder.test.ts`	31
New	0	`packages/daemon/src/execution/interaction/internal/interaction-touch-response-contract.test.ts`	1
`packages/daemon/src/execution/interaction/internal/interaction-touch-routing.test.ts`	7	same	8

The native-ref contract remains at 9 cases; its daemon-only response-builder assertion moved to the new package test, the response builder keeps capture evidence out of serialized touch payloads. The recorder test adds ordinary action history preserves authored commands, positionals, and replay flags and --no-record suppresses a mutating coordinate press from action history. The provider device test adds the public session-to-device-and-lease binding case. Root script-publication scenarios retain their public error, saved-file, and replay assertions; daemon recording and publication state assertions are covered beside the owning daemon modules, including parameterized fills keep literals out of recording state with deterministic placeholders, publishes without close, returns the path/count, and leaves a terminal live session, #1533: a re-open leaves an aborted authoring lifecycle aborted, and not recording, and #1533: an ordinary armed authoring session still publishes on bare close. The scroll pass-limit assertion is covered by packages/daemon/src/execution/scroll-until.test.ts, test the default budget is the shared constant. The stale-provider route now asserts a typed owner-capability-missing refusal before the counted provider bind or provider execution; packages/daemon/src/sessions/handlers/__tests__/session-reinstall.test.ts remains the daemon-level fail-closed assertion. No test case was dropped from a moved test.

Selection

Provider scenarios remain selected by Vitest project provider-integration (vitest.config.ts, include test/integration/provider-scenarios/**/*.test.ts).
Package-local daemon src/**/*.test.ts tests are selected by unit-core via packages/*/src/**/*.test.ts.
packages/remote/src/__tests__/daemon-artifacts-directory.test.ts is selected by unit-core through that same package-source include and by CI's unit-ci coverage gate (test:coverage:ci). Its three tests pass after the relocation.
Concurrency torture is a Node node:test lane, not a Vitest project. The test:concurrency-torture script selects packages/daemon/test/nightly/concurrency-torture.test.ts; both the PR CI lane and nightly workflow call the existing concurrency-torture gate. The package test/ path is outside unit-core.
test/output-economy/**/*.test.ts is selected by the output-economy Vitest project. The help sample output and recovery coverage tests are selected by unit-core, including the explicitly listed scripts/__tests__/help-conformance-sample-outputs.test.ts. The route helpers are imported by these selected tests and are not standalone tests.
pnpm exec vitest related --project output-economy --run --passWithNoTests packages/daemon/src/server/response-views.ts selects test/output-economy/output-economy.baseline.test.ts; the route driver is a static import chain from render-fixtures.ts, so response-view changes reach the output-economy gate without a child process.
pnpm check:gate-manifest: passed, 57 checks wired across 25 lanes. pnpm test:concurrency-torture: passed, 4 tests.

Moves

Pure move commits are separate from edits: 20972a21f (concurrency tree), f402a5f00 (HTTP test), and 10c48bf63 (remote artifact-directory test). The move commits contain no line edits; follow-up edits are separate. git diff -M --summary origin/main...HEAD reports the concurrency test 100%, concurrency helpers 77–98%, HTTP test 98%, and remote artifact-directory test 69% similarity.

Mocks and spies

limrun-ios-snapshot-owner.test.ts: @limrun/api mock is observed by createInstance.mock.calls.length === 1.
limrun-ios-snapshot-owner.test.ts: @limrun/api/ios-client mock is observed by elementTree.mock.calls.length === 1.
stale-provider-runtime-admission.test.ts: @limrun/api mock is observed by one allocation before release and no additional allocation before admission.
stale-provider-runtime-admission.test.ts: @limrun/api/instance-client mock is observed by the release-time disconnect.
stale-provider-runtime-admission.test.ts: the counted provider-module bind spy remains unchanged after the public route's typed admission refusal.
cloud-webdriver-runtime.test.ts: the Date.now spy advances past lease expiry; the server records WebDriver DELETE.
remote-proxy-parity.test.ts expiry case: the Date.now spy advances past expiry; the proxy returns LEASE_NOT_FOUND and a fresh lease has no prior comparison state.
remote-proxy-parity.test.ts heartbeat case: the Date.now spy proves renewed expiry moves forward and the session baseline remains usable.
ios-fold.test.ts: the Date.now spy advances by accumulated tool timeout; the final permitted hinge read succeeds and stays within the client envelope.

The dead ios-app-session-hint and Apple runner operations mocks were removed.

Mutation checks

Each new or moved destination test was mutated and then restored:

HTTP body limit changed from 1 MiB to 2 MiB: the moved HTTP server test failed.
Recorded touch capture evidence replaced with an empty object: the touch response contract test failed.
Action history changed from append to prepend: the recorder ordering assertion failed.
Fresh-session device locking disabled: the concurrency-torture lane failed all four cases on overlap.
downloadRemoteArtifact forced to treat directories as files: the moved remote artifact-directory test failed on extraction and rejected symlink coverage; the implementation was restored.
The Apple output-guard fake's runner authority changed to scoped-provider: the iOS guard hit its record timeout; the fake was restored to local-lease.
In an isolated worktree at eb5528854, removing artifacts from the production screenshot digest made test/output-economy/output-economy.baseline.test.ts fail its reviewed byte/shape baseline (1 failed, 25 passed).
In the same isolated worktree, changing the production DEVICE_IN_USE recovery hint made DEVICE_IN_USE_SAMPLE fail in scripts/__tests__/help-conformance-sample-outputs.test.ts (1 failed, 13 passed).
Changing the production live-claim hint made DEVICE_CLAIM_IN_USE_SAMPLE fail in that same test (1 failed, 13 passed).
Changing the production stale-ref recovery hint made STALE_REF_SAMPLE fail there (1 failed, 13 passed).
Changing the production ambiguous-match message made AMBIGUOUS_MATCH_SAMPLE fail there (1 failed, 13 passed).
Removing appBundleId from the production open response made all five cloud iOS text-entry scenarios fail at appBundleId === 'com.example.demo'.
Adding fabricated x/y to a recorded ref click made the interaction-route test fail; suppressing type action recording made its recorded-command assertion fail.
Dropping the lease from a published provider session made the provider device-runtime test fail because close no longer released the allocated lease.
Binding the unsupported install route before returning the typed refusal made the public admission test fail its zero-bind assertion (observed 1 bind).
Removing the production artifact-download authorization header made the public materialization test fail against its token-protected endpoint.
Omitting params.tenantId made the proxy integration test fail for agent_device.command; removing the upstream token header made its RPC-handler observation fail.
Removing encodeURIComponent from the diagnostics session path made the proxy test fail its exact PLAIN_SESSION URL assertion (cwd: versus cwd%3A).
Disabling diagnostics GET forwarding in the parity fixture made the direct/proxied scenario fail because the missing-ref response had no localized logPath (1 failed, 4 passed).
At eee378dd5, a disposable worktree was installed and built. Removing session encoding made the diagnostics request test fail with HTTP 404 (1 failed); preventing snapshot-generation increments made the settle-generation test fail on equal pre-action and settle generations (1 failed, 14 passed).
In the same disposable worktree, making daemon close unconditional replaced the expected startup error with a TypeError; disabling either temp-directory removal left an entry in the temp root; and skipping the claims-directory env restore failed the previous-value assertion. Each mutation was reverted, the restored tests passed, and the worktree was clean before removal.

Exports

packages/daemon exports: 34 on rebased origin/main (. plus 33 testing exports) → 19 on this branch (. plus 18 testing exports). This PR removes:

./testing/daemon-command-registry
./testing/device/device-claim-conflict
./testing/handlers/session-app-deployment
./testing/interaction/internal/interaction-touch-response
./testing/interactor-resolution
./testing/ios-app-session-hint
./testing/request-binding
./testing/request-execution-scope
./testing/request-router
./testing/response-views
./testing/scroll-until
./testing/selector-match-errors
./testing/session-observability/__tests__/*
./testing/session-recovery-hints
./testing/web-session-names

Remaining exports and their owners:

Daemon-composition owner mcp:57b5c3dd (15): ./testing/__tests__/support/session-factories, ./testing/__tests__/support/store-factory, ./testing/__tests__/support/unavailable-device-runtime-gateway, ./testing/artifact-tracking, ./testing/interaction/index, ./testing/lease-registry, ./testing/provider-device-admission, ./testing/ref-frame, ./testing/request-dispatch-ledger, ./testing/request-finalization, ./testing/request-runtime-binding, ./testing/server/http-server, ./testing/session-snapshot, ./testing/session-state, ./testing/session-store.
Other root-src sweep owner (2): ./testing/provider-lease-expiry, ./testing/session-action-recorder. These remain outside this PR's importer ownership.
String-only guard evidence (1): ./testing/context is named only by scripts/layering/test-kit-isolation.test.ts; it is not a live importer. Its released export remains for the final-pin owner, thread mcp:57b5c3dd, to resolve.

Earlier validation at f2592cb3f248c3cf9acdf26629ed76fff4908d00, based on origin/main 4c1c786: CI=true pnpm install --frozen-lockfile, pnpm build, repository-wide pnpm format (through affected), pnpm check:quick, pnpm check:layering, pnpm check:fallow, and pnpm check:gate-manifest passed. Its exact-head affected receipt is .tmp/validation/run-kVszWM/receipt.json (36 passed, 25 GitHub-authoritative or parked checks skipped, 2 deduplicated). Provider integration passed 64 files / 191 tests; concurrency-torture passed 4 tests. Earlier output-economy and help-conformance counts and mutation evidence remain recorded above. GitHub CI, including CodeQL, must pass on the current pushed head.

Earlier review follow-up at 63df0ae1ccbb71b8a988b6b5fa92466f21f18b38, based on origin/main 4c1c786: this commit strengthened materialization evidence by asserting the shared recorder sees a real daemon test RPC followed by an authenticated artifact GET. The materialization, tenant, and diagnostics scenarios use the shared proxy-to-real-daemon fixture in harness.ts. Provider integration passed 64 files / 191 tests; format, quick, layering, Fallow, and gate-manifest checks passed. Disabling artifact forwarding failed materialization; disabling the daemon tenant-owner check failed the tenant scenario with Missing expected rejection. Both mutations were reverted. No test cases were added or removed by that follow-up.

The exact-head affected run for 63df0ae1ccbb71b8a988b6b5fa92466f21f18b38 did not complete: the sandbox first denied its external reclaim lock, and the escalated retry waited on the host gate lock owned by another live validation process (PID 75149, started 22:27) before being aborted. The earlier exact-head run at f2592cb3f248c3cf9acdf26629ed76fff4908d00 passed. The current rebased head has a new exact-head pass recorded below.

Review follow-up at 85857f5009706a432c83ef5a11df037f6aabac0d:

4ece298a0 moves the recording proxy and its forwarding helpers from harness.ts to harness-proxy.ts; the separate commit precedes all behavior changes. Shared command/result conversion lives in harness-rpc.ts.
f7a2f9081 replaces the copied Android audio contract with ANDROID_AUDIO_CONTRACT_EVIDENCE in test/integration/command-coverage/evidence.ts, pointing to the literal title in session-audio.test.ts; it deletes session-audio.coverage.ts and the unused wildcard export. The disposable evidence-check run passed with the original title, failed after changing it to Android audio helper starts on emulator (audio evidence is not named), then passed after restoration.
f7a2f9081 makes daemon and proxy callCommand return the HTTP status from the response. The initial shared error assertion derived statuses from error codes; commit 35dd42016 below removes that derivation. The help route pins its session-use conflict, stale-ref, ambiguity, and cross-daemon claim responses to 500; the missing-session diagnostics request is 404, and the provider failure is 500. The existing daemon human-control HTTP test asserts the real 423 response.
The first help-conformance run exposed an incorrect 423 expectation for the session-use conflict. 85857f500 corrects it to the server's 500; no error assertion was loosened.
pnpm test:integration:provider passes 64 files / 192 tests. Help-conformance passes 5 files / 99 tests. pnpm format passes across 4,677 files.
pnpm check:affected --run --head 85857f5009706a432c83ef5a11df037f6aabac0d passes with exactCommit: true, checkoutChanged: false: 36 passed, 25 GitHub-authoritative or parked checks skipped, 2 deduplicated, 0 failures. Receipt: .tmp/validation/run-4AdMJH/receipt.json.
The --flag=value thread remains resolved; it was scoped file-wide and no one-off matcher was added.

Review follow-up at 35dd42016bdfd0354de8a77decbce716f0ad65ae:

assertRpcError now requires an explicit 400 | 401 | 404 | 423 | 500 status argument and compares it with the actual HTTP response. All 49 call sites pass their scenario's literal; there is no code-to-status mapping in the helper. The provider test trees contain no REPAIR_SESSION_EXPIRED scenario.
Mutation proof in a disposable worktree at this commit: the focused batch-flags.test.ts scenario passed (1/1), failed after changing its expected status from 400 to 500 (400 !== 500), then passed after restoration (1/1). The disposable worktree was removed.
pnpm test:integration:provider passes 64 files / 192 tests. pnpm format passes across 4,677 files. pnpm check:affected --run --head 35dd42016bdfd0354de8a77decbce716f0ad65ae passes with exactCommit: true, checkoutChanged: false: 36 passed, 25 GitHub-authoritative or parked checks skipped, 2 deduplicated, 0 failures. Receipt: .tmp/validation/run-6mtmnQ/receipt.json.

CodeQL stack-trace follow-up: commit 125a5c2986d9e0cfacd5da5d6f5c97311b673319 records only an upstream error code on the test proxy entry, never the caught Error object; its response remains the fixed upstream request failed body. The fixed-body assertion passes in the focused parity suite. The first isolated mutation setup hit registry DNS ENOTFOUND; a subsequent isolated install succeeded for the proxy-cleanup mutation. No separate stack-trace mutant was run. CodeQL must be confirmed on this pushed head.

Review follow-up: commit 070be35470 moves daemon and proxy setup under the protected try and makes each cleanup conditional. A startup-failure regression uses a real daemon and injected proxy startup error, then asserts close ran. The provider parity suite passed 7/7; a disposable baseline passed (1 passed, 6 skipped) and the mutation skipping daemon close failed (1 failed, 6 skipped, closeCalls was 0 instead of 1). pnpm format and pnpm check:quick passed. The existing CodeQL fix from 125a5c2986 is included in the pushed branch.

Rebase validation at 025fd8470e5391e09f9f34757b1c6b5d5912a85b: rebased onto origin/main bb05c630252e227b851d3eca333d2f4634be3176. Conflict resolutions kept both upstream deletion sets, this PR's 15 export deletions, and ./testing/context; the @agent-device/test-kit named exports from both sides remain sorted. pnpm install --frozen-lockfile, pnpm build, pnpm format, pnpm check:quick, pnpm check:layering, pnpm check:fallow, and pnpm check:gate-manifest passed. Provider integration passed 64 files / 192 tests; concurrency-torture passed 4 tests. Exact-head pnpm check:affected --run --head 025fd8470e5391e09f9f34757b1c6b5d5912a85b passed with exactCommit: true, checkoutChanged: false: 36 passed, 25 GitHub-authoritative or parked checks skipped, 2 deduplicated, 0 failures. Receipt: .tmp/validation/run-QMfwTs/receipt.json. The daemon manifest is now 34 exports on main to 19 on this branch; the 18 remaining testing exports and their owners are listed above.

🤖 Generated with Claude Code