feat(backend): (INFP-589) Collect more metrics and introduce 24h windowing in telemetry collection

Merged
SA
4mo ago
+3488-91

Why

Phase 1 telemetry only reports a point-in-time snapshot (counts, versions) and a 7-day rolling event tally bounded by Prefect's retention. There's no adoption signal (accounts, active branches) and no way to see what happened over a fixed, comparable window.

Adds adoption metrics and a 24h activity window, computed the same way the product counts nodes so the numbers are correct across branches. Does not change the collection schedule, opt-out behavior, or storage.

What changed

  • activity_24h: logins (+unique), checks started/passed/failed, artifacts created/updated, branches created/merged/deleted, and webhook success/failure, over the previous full UTC calendar day — a deterministic, half-open [start, end) window that tiles exactly with no gaps or overlap.
  • New counts: accounts.active, accounts.groups, branches.active.
  • database.node_count.corenode (all managed nodes) and database.node_count.user (user-defined-namespace nodes only); the existing raw total is unchanged.
  • Per-metric graceful degradation: one failing source reports null for just that field; a source that succeeds with nothing to count reports 0. The
    per-source seams in gather_anonymous_telemetry_data are plain function
    parameters defaulted to the real gatherers — no mocking, per the project's
    no-mock testing rule.
  • payload_format bumped to 20260628. All changes are additive.

Implementation notes, known gaps, and the temporal model (point-in-time vs.
rolling vs. windowed) are documented in
dev/knowledge/backend/telemetry.md,
including field-level semantics for activity_24h (e.g. logins counts
interactive sign-ins only — password/OIDC/OAuth2 — not per-request API-key
authentication). Notably, checks_started can currently exceed
checks_passed + checks_failed for validators with no real checks to run —
documented and covered by a unit test, not fixed here (pre-existing
event-emission gap, out of scope).

Example payload

Here is a representative daily payload with this PR's additions. New fields are marked // NEW; everything else is unchanged. null means the source failed, 0 means measured-and-empty.

{
  "kind": "community",
  "payload_format": "20260628",              // BUMPED (was "20250318")
  "checksum": "9f2c…(sha256 of the data object)",
  "data": {
    "deployment_id": "6f1c9b2e-3a4d-4c8e-9f0a-1b2c3d4e5f60",
    "execution_time": 2.41,
    "infrahub_version": "1.4.0",
    "infrahub_type": "community",
    "python_version": "3.14.3",
    "platform": "aarch64",

    "workers": { "total": 2, "active": 2 },

    "branches": {
      "total": 12,
      "active": 4                            // NEW — open, non-system branches (int | null)
    },

    "accounts": {                            // NEW object
      "active": 7,                           // int | null
      "groups": 3                            // int | null
    },

    "activity_24h": {                        // NEW object — previous full UTC day [00:00, 00:00)
      "logins": 19,                          // int | null
      "unique_logins": 6,
      "checks_started": 88,                  // denominator; NOT additive with passed/failed
      "checks_passed": 80,
      "checks_failed": 8,
      "artifacts_created": 14,
      "artifacts_updated": 31,
      "branches_created": 9,
      "branches_merged": 5,
      "branches_deleted": 4,
      "webhooks_fired_success": 41,          // COMPLETED webhook-process runs
      "webhooks_fired_failure": 2            // FAILED / CRASHED webhook-process runs
    },

    "features": {
      "CoreArtifact": 12,
      "CoreResourcePool": 3,
      "CoreRepository": 5,
      "CoreGroup": 21,
      "CoreProfile": 4,
      "CoreProposedChange": 37,
      "CoreObjectTemplate": 2,
      "CoreTransformation": 6,
      "CoreWebhook": 3
    },

    "schema_info": {
      "node_count": 84,
      "generic_count": 19,
      "last_update": "2026-06-27T14:22:10.512000+00:00"
    },

    "database": {
      "database_type": "neo4j-community",
      "relationship_count": {
        "total": 481902
        // … existing per-graph-label counts unchanged …
      },
      "node_count": {
        "total": 154233,                     // unchanged — raw vertex total
        // … existing per-graph-label counts unchanged …
        "corenode": 4821,                    // NEW — all managed nodes (int | null)
        "user": 3902                         // NEW — user-defined-namespace nodes (int | null)
      },
      "servers": [],                         // populated on Neo4j Enterprise only
      "system_info": {
        "memory_total": 17179869184,
        "memory_available": 8231473152,
        "processor_available": 10
      }
    },

    "prefect": {
      "events": {                            // unchanged — cumulative over Prefect's ~7-day event retention
        "infrahub.account.logged_in": 142,
        "infrahub.account.logged_out": 60,
        "infrahub.artifact.created": 33,
        "infrahub.artifact.updated": 88,
        "infrahub.branch.created": 25,
        "infrahub.branch.deleted": 14,
        "infrahub.branch.merged": 12,
        "infrahub.branch.migrated": 0,
        "infrahub.branch.rebased": 7,
        "infrahub.group.auto_created": 5,
        "infrahub.group.member_added": 18,
        "infrahub.group.member_removed": 3,
        "infrahub.node.created": 1204,
        "infrahub.node.deleted": 231,
        "infrahub.node.updated": 5892,
        "infrahub.proposed_change.approved": 22,
        "infrahub.proposed_change.merged": 19,
        "infrahub.proposed_change.rejected": 2,
        "infrahub.proposed_change.review_requested": 24,
        "infrahub.repository.update_commit": 41,
        "infrahub.validator.started": 233,
        "infrahub.validator.passed": 205,
        "infrahub.validator.failed": 18
        // … remaining event names omitted for brevity …
      },
      "automations": {
        "action_trigger_rule": 0,
        "builtin": 6,
        "webhook": 3,
        "computed_attr_jinja2": 4,
        "computed_attr_python": 1,
        "computed_attr_python_query": 0,
        "display_label_jinja2": 12,
        "human_friendly_id": 8,
        "profile": 4
      },
      "work_pools": [
        { "name": "infrahub-worker", "type": "infrahubasync", "total_workers": 2, "active_workers": 2 }
      ]
    }
  }
}

The three checks_* fields are not additive (checks_started is the denominator; derive pass/fail rate from it). webhooks_* is outcomes-only this phase. See contracts/telemetry-payload.md for the full field semantics.

How to review

  • Core logic: backend/infrahub/telemetry/window.py (windowing),
    task_manager.py (windowed Prefect event/flow-run counters),
    database.py (node counts), tasks.py (payload assembly, degradation).
  • dev/specs/telemetry-collection-infp-589/ has the full spec/plan trail;
    contracts/telemetry-payload.md is the payload contract.

How to test

uv run invoke backend.test-unit backend.test-component --pattern telemetry

Manual end-to-end validation steps (trigger a collection, cross-check each
field against GraphQL, validate the window with the ad-hoc probe) are in
dev/specs/telemetry-collection-infp-589/manual-test-runbook.md.

Impact & rollout

  • Backward compatibility: additive only; no fields removed or retyped.
    Consumers should tolerate unknown fields and null.
  • Config/env changes: none. Daily schedule and local snapshot storage unchanged.
  • Deployment notes: safe to deploy independently.

Checklist

  • Tests added/updated
  • Changelog entry added
  • External docs updated (not needed — docs/deploy-manage/run-observe/telemetry.mdx
    documents the storage/export mechanism, not individual payload fields)
  • Internal .md docs updated
  • I have reviewed AI generated content