Sample engine requests and bound message tracking dimensions

Draft
WI
6d ago
+386-15

Engine HTTP requests lack an outcome-independent denominator, and message observations lack recipient/harness dimensions. Add uniform 1% request.completed sampling before routing/authentication and bounded message dimensions from already-resolved rows, with no new database queries or public response fields.

Request facts: route_template, method, status_code, duration_ms, sample_rate. Only registered Hono templates are exported; unmatched paths become unmatched. Workspace identity is optional only for HTTP completions. telemetryRequestSampleRate is configurable (0 disables, finite clamp 0..1). The types catalog adds relaycast_server_request.

The denominator is non-heartbeat HTTP requests reaching the engine, including 401/404/finalized 500 outcomes. POST /v1/agents/heartbeat is excluded before sampling, including failures. WebSocket heartbeat frames and hosting-gateway rejections/interceptions before the engine are outside this denominator. Latency ends when the response is finalized, not when a streaming response ends. Every sample carries its inclusion probability; this is not a full-traffic-coverage claim.

All four message observations carry sender_harness, recipient_type, recipient_harness. Broker metadata.cli, Desktop session_kind, and exact known harness/orchestrator_harness values map to bounded canonical families, including antigravity; custom commands and model suffixes remain unknown. Sender metadata already comes from the authenticated actor or existing node-sender query, without another lookup. Channel sends identify channel; direct DM, group DM and DM-thread recipients use persisted agent types. Homogeneous agent/human audiences identify that type; mixed, empty or unavailable audiences remain unknown. A shared canonical recipient harness is retained; heterogeneous harnesses remain unknown.

Group/thread observations conditionally project type/metadata into both existing successful-delivery and missing-recipient queries, preserving joins, membership predicates and public result maps. Request-local callbacks are exception-isolated and are never serialized into responses or idempotency storage. Existing replay and credential-authenticated provenance remain intact; arbitrary owner/user metadata never establishes identity.

Validation on reviewed follow-up:

  • npm run build --workspace=@relaycast/engine, npm run typecheck --workspace=@relaycast/engine, changed-source ESLint and git diff --check succeeded.
  • npm run test --workspace=@relaycast/engine — 161 files / 1844 tests; local socket access enabled for fixture listeners.
  • ./node_modules/.bin/vitest run packages/engine/src/lib/__tests__/serverTelemetryCatalog.test.ts packages/types/src/__tests__/telemetry.test.ts — 12 tests.
  • Recipient conformance: 7 cases, covering admitted and rejected recipients, homogeneous/mixed groups, DM threads, identical query counts with observation on/off, isolated callback throws, public/replayed response equality, and captured KV writes free of recipient metadata.
  • Negative controls: four conformance cases fail when recipient observer delivery is removed; five original request tests fail when completion emission is removed. Sources restored; relevant tests are green.
  • Independent source review signed off the engine and companion changes.
  • PATH=/Users/will/.nvm/versions/node/v22.14.0/bin:$PATH npm run test:engine:regression — SUCCESS, all seven built-output regression stages, including real workerd-D1 controls, TLS redirect controls, packed public consumer compile and task invocation fencing. Final toolchain: Node 22.14.0 and npm 11.7.0. Initial npm 12 run encountered a harness JSON-shape incompatibility in npm pack; final full rerun passed without changing the harness.

Draft / release hold: requires Barry review and Will approval. No merge, publish or deployment is authorized. A real engine release containing this source must precede hosted dependency uptake. Do not invent an unpublished package version. Follow-up must update root/workspace dependencies and lockfile plus the engine bundling marker in packages/relaycast/src/entrypoints/cloudflare.ts, then rerun hosted CI. No plan migrations or deployment configuration changed.

Companion hosted transport/policy: https://github.com/AgentWorkforce/relaycast-cloud/pull/296.
Cloud receiver/analytics consumer: https://github.com/AgentWorkforce/cloud/pull/5909.

Final remote validation — 2026-11-02 01:44 UTC, head 82b5494ab111f37fc3a3a185a1e0df3395c6a329:

  • All 14 execution checks are SUCCESS: primary Lint/Build/Test (including the complete regression gate), engine binary Build, real-image integration, both architecture containers, Bun SQLite suite, Rust SDK and all seven platform binary smokes.
  • Preview cleanup, release attachment and CodeRabbit draft review are intentionally SKIPPED. No bot rerun was requested.
  • Draft remains MERGEABLE/CLEAN, with no submitted reviews or inline findings; the only bot conversation notice says draft reviews are skipped. Independent source review is complete, but Barry review and Will approval remain required by this task's release hold.
  • Worktree is clean. No merge, package publication or deployment occurred. CI monitoring stops after this handoff; no background watcher remains.

Agent Relay sessions

  • codex session 01a19972 · contributor · ran gh pr commands · last active 2026-11-02