Stream D: Cloudflare usage monitor

Merged
KH
4mo ago
+553-0

Summary

Cloudflare spend/usage monitor for the relayfile-cloud migration. Watches D1 rows read/written, R2 storage costs, queue throughput, and Worker error rates via the relayfile VFS usage feeds, posting Slack alerts when thresholds are exceeded.

Files

  • cloudflare-monitor/persona.ts — definePersona with Cloudflare integration scope (5 usage feeds) + Slack, 4 configurable thresholds
  • cloudflare-monitor/agent.ts — defineAgent with 2h cron sweep, fingerprint-deduped Slack alerts, memory persistence, inbox chat Q&A
  • cloudflare-monitor/README.md — wiring diagram, signal descriptions, data source table

Design decisions

  • No dead triggers: All 5 per-operation runtime error triggers (d1.query_failed, r2.operation_failed, etc.) dropped — no analytics dataset backs them. Only usage-threshold signals kept, per migration-lead direction.
  • API-first: Agent reads only the Relayfile VFS mounts materialized by Nango syncs — no Cloudflare API token in the agent.
  • Fingerprint dedup: Re-alerts only when signal values change, preventing Slack noise.
  • Follows neon-monitor pattern in the same repo.

Verification

  • tsc --noEmit passes clean
  • Self-review completed — no issues found

Closes Stream D (monitor half). Usage sync PR in cloud repo follows.

Summary by cubic

Adds a Cloudflare usage monitor for the relayfile-cloud migration (Stream D) that reads VFS usage feeds and sends deduped Slack alerts when thresholds are exceeded. Fixes cron tick handling, wires inbox Q&A, and improves fingerprinting/logging for reliable, low-noise alerts.

  • New Features

    • cloudflare-monitor/persona.ts: Persona with Cloudflare VFS scopes and Slack; inputs for SLACK_CHANNEL and thresholds; inbox Q&A.
    • cloudflare-monitor/agent.ts: 2-hour cron scan (cron gate fixed via isCronTickEvent), fingerprint-deduped Slack alerts (now include R2 Class A ops/egress and Worker requests), logs feed-read failures, memory persistence; no Cloudflare API token required.
    • Thresholds (defaults): D1 read 1,000,000; D1 written 100,000; R2 storage 100 GB; R2 Class A ops 1,000,000; R2 egress 100 GB; Queue unacked 1,000; Queue retries 100; Worker error rate 5%. Slack messages truncate to top 5 with “+N more”.

  • Migration

    • Set SLACK_CHANNEL (required). Optionally override threshold inputs.
    • Ensure Nango syncs populate /cloudflare/{d1,r2,queues,workers}/usage/** mounts.
    • Deploy; cron 0 */2 * * * runs the scan and posts alerts.