feat(backend): (INFP-589) Collect more metrics and introduce 24h windowing in telemetry collection
Why
Phase 1 telemetry only reports a point-in-time snapshot (counts, versions) and a 7-day rolling event tally bounded by Prefect's retention. There's no adoption signal (accounts, active branches) and no way to see what happened over a fixed, comparable window.
Adds adoption metrics and a 24h activity window, computed the same way the product counts nodes so the numbers are correct across branches. Does not change the collection schedule, opt-out behavior, or storage.
What changed
activity_24h: logins (+unique), checks started/passed/failed, artifacts created/updated, branches created/merged/deleted, and webhook success/failure, over the previous full UTC calendar day — a deterministic, half-open[start, end)window that tiles exactly with no gaps or overlap.- New counts:
accounts.active,accounts.groups,branches.active. database.node_count.corenode(all managed nodes) anddatabase.node_count.user(user-defined-namespace nodes only); the existing rawtotalis unchanged.- Per-metric graceful degradation: one failing source reports
nullfor just that field; a source that succeeds with nothing to count reports0. The
per-source seams ingather_anonymous_telemetry_dataare plain function
parameters defaulted to the real gatherers — no mocking, per the project's
no-mock testing rule. payload_formatbumped to20260628. All changes are additive.
Implementation notes, known gaps, and the temporal model (point-in-time vs.
rolling vs. windowed) are documented in
dev/knowledge/backend/telemetry.md,
including field-level semantics for activity_24h (e.g. logins counts
interactive sign-ins only — password/OIDC/OAuth2 — not per-request API-key
authentication). Notably, checks_started can currently exceed
checks_passed + checks_failed for validators with no real checks to run —
documented and covered by a unit test, not fixed here (pre-existing
event-emission gap, out of scope).
Example payload
Here is a representative daily payload with this PR's additions. New fields are marked // NEW; everything else is unchanged. null means the source failed, 0 means measured-and-empty.
{
"kind": "community",
"payload_format": "20260628", // BUMPED (was "20250318")
"checksum": "9f2c…(sha256 of the data object)",
"data": {
"deployment_id": "6f1c9b2e-3a4d-4c8e-9f0a-1b2c3d4e5f60",
"execution_time": 2.41,
"infrahub_version": "1.4.0",
"infrahub_type": "community",
"python_version": "3.14.3",
"platform": "aarch64",
"workers": { "total": 2, "active": 2 },
"branches": {
"total": 12,
"active": 4 // NEW — open, non-system branches (int | null)
},
"accounts": { // NEW object
"active": 7, // int | null
"groups": 3 // int | null
},
"activity_24h": { // NEW object — previous full UTC day [00:00, 00:00)
"logins": 19, // int | null
"unique_logins": 6,
"checks_started": 88, // denominator; NOT additive with passed/failed
"checks_passed": 80,
"checks_failed": 8,
"artifacts_created": 14,
"artifacts_updated": 31,
"branches_created": 9,
"branches_merged": 5,
"branches_deleted": 4,
"webhooks_fired_success": 41, // COMPLETED webhook-process runs
"webhooks_fired_failure": 2 // FAILED / CRASHED webhook-process runs
},
"features": {
"CoreArtifact": 12,
"CoreResourcePool": 3,
"CoreRepository": 5,
"CoreGroup": 21,
"CoreProfile": 4,
"CoreProposedChange": 37,
"CoreObjectTemplate": 2,
"CoreTransformation": 6,
"CoreWebhook": 3
},
"schema_info": {
"node_count": 84,
"generic_count": 19,
"last_update": "2026-06-27T14:22:10.512000+00:00"
},
"database": {
"database_type": "neo4j-community",
"relationship_count": {
"total": 481902
// … existing per-graph-label counts unchanged …
},
"node_count": {
"total": 154233, // unchanged — raw vertex total
// … existing per-graph-label counts unchanged …
"corenode": 4821, // NEW — all managed nodes (int | null)
"user": 3902 // NEW — user-defined-namespace nodes (int | null)
},
"servers": [], // populated on Neo4j Enterprise only
"system_info": {
"memory_total": 17179869184,
"memory_available": 8231473152,
"processor_available": 10
}
},
"prefect": {
"events": { // unchanged — cumulative over Prefect's ~7-day event retention
"infrahub.account.logged_in": 142,
"infrahub.account.logged_out": 60,
"infrahub.artifact.created": 33,
"infrahub.artifact.updated": 88,
"infrahub.branch.created": 25,
"infrahub.branch.deleted": 14,
"infrahub.branch.merged": 12,
"infrahub.branch.migrated": 0,
"infrahub.branch.rebased": 7,
"infrahub.group.auto_created": 5,
"infrahub.group.member_added": 18,
"infrahub.group.member_removed": 3,
"infrahub.node.created": 1204,
"infrahub.node.deleted": 231,
"infrahub.node.updated": 5892,
"infrahub.proposed_change.approved": 22,
"infrahub.proposed_change.merged": 19,
"infrahub.proposed_change.rejected": 2,
"infrahub.proposed_change.review_requested": 24,
"infrahub.repository.update_commit": 41,
"infrahub.validator.started": 233,
"infrahub.validator.passed": 205,
"infrahub.validator.failed": 18
// … remaining event names omitted for brevity …
},
"automations": {
"action_trigger_rule": 0,
"builtin": 6,
"webhook": 3,
"computed_attr_jinja2": 4,
"computed_attr_python": 1,
"computed_attr_python_query": 0,
"display_label_jinja2": 12,
"human_friendly_id": 8,
"profile": 4
},
"work_pools": [
{ "name": "infrahub-worker", "type": "infrahubasync", "total_workers": 2, "active_workers": 2 }
]
}
}
}
The three checks_* fields are not additive (checks_started is the denominator; derive pass/fail rate from it). webhooks_* is outcomes-only this phase. See contracts/telemetry-payload.md for the full field semantics.
How to review
- Core logic:
backend/infrahub/telemetry/window.py(windowing),
task_manager.py(windowed Prefect event/flow-run counters),
database.py(node counts),tasks.py(payload assembly, degradation). dev/specs/telemetry-collection-infp-589/has the full spec/plan trail;
contracts/telemetry-payload.mdis the payload contract.
How to test
uv run invoke backend.test-unit backend.test-component --pattern telemetry
Manual end-to-end validation steps (trigger a collection, cross-check each
field against GraphQL, validate the window with the ad-hoc probe) are in
dev/specs/telemetry-collection-infp-589/manual-test-runbook.md.
Impact & rollout
- Backward compatibility: additive only; no fields removed or retyped.
Consumers should tolerate unknown fields andnull. - Config/env changes: none. Daily schedule and local snapshot storage unchanged.
- Deployment notes: safe to deploy independently.
Checklist
- Tests added/updated
- Changelog entry added
- External docs updated (not needed —
docs/deploy-manage/run-observe/telemetry.mdx
documents the storage/export mechanism, not individual payload fields) - Internal .md docs updated
- I have reviewed AI generated content