Superseded by #11289. The six INFP-507 PRs are consolidated there into one commit targeting develop, with the complete design, code-flow diagrams and contract-based review guide. This PR is closed to keep one active review; its branch and previous discussions are preserved.

Stacked PR 4/6 for INFP-507 (task history and activity log retention), in GitHub stack #10873 on develop. Review and merge bottom-up.

1. #10862: retention settings applied to Prefect
2. #10869: task history cleanup command, and the upgrade's task history hint
3. #10876: Activities page queries
4. This PR: guard for Prefect's event types on the full stack
5. #10878: infrahub tasks background-services
6. #10879: documentation

Why

The activity log retention (PR 1) keeps Infrahub events for activity_log and Prefect's own events for prefect_own_events. Prefect's per-type retention overrides match event names exactly, so a Prefect event type missing from PREFECT_EVENT_TYPES is kept for the whole activity log retention. With the activity log raised to a year, that type piles up for a year. A unit test already covers every built-in state name of the pinned Prefect; this PR checks a real workload on the full stack.

Closes part of INFP-507.

What changed

• New integration-docker test backend/tests/integration_docker/test_prefect_event_types.py (shard b). On the full stack with real task workers and Postgres it runs:

  • schema load, node changes, branch create, rebase and merge, and a proposed change
  • a flow that fails: a read-only repository on a ref that does not exist
  • a run cancelled through Prefect's API
  • a task worker restart during the proposed change pipeline

  It then reads SELECT DISTINCT event FROM events WHERE event NOT LIKE 'infrahub.%' and asserts that no stored type is missing from PREFECT_EVENT_TYPES. It also asserts that Failed and Cancelled run events were stored, so a quiet stack cannot pass.

• The workload stored 30 distinct Prefect event types, all already listed; PREFECT_EVENT_TYPES is unchanged. With prefect.worker.submitted-flow-run removed from the list, the test fails and names it.

• Spec set: the widest Activities window already comes from Prefect's events retention, which #10862 sets from activity_log, so no production change was needed for it; research.md and the task-manager API contract now say so.

• Startup warning. Neither test sees a prefect.* type that a later Prefect release emits outside the workload. Once Prefect's own startup has migrated the database, the task manager now reads each stored event type (one index lookup per type, so a year of events is not scanned) and logs a warning naming every prefect.* type missing from the list:

  Task manager retention: Infrahub does not list these Prefect event types, so they are kept for the activity log retention (7 days) instead of prefect_own_events: prefect.infrahub-test.unlisted
  

  A type shows up from the first start after Prefect stored an event of it. A failed read is logged and does not stop the task manager. Code: prefect_server/database.py::read_stored_event_types, prefect_server/retention.py::unlisted_prefect_event_types, the check after Prefect's startup in prefect_server/app.py::create_infrahub_prefect.

  • Tests: a unit test of the selection; component tests of the read and the warning on a SQLite database of their own (with the recursion removed, the read returns only the first type); a full-stack test that stores an unlisted type, restarts the task manager and expects the warning, which also runs the read on Postgres. Against an image without the warning it fails with 0 == 1.

How to test

uv run pytest backend/tests/integration_docker/test_prefect_event_types.py -n 0
uv run pytest backend/tests/unit/prefect_server/test_prefect_event_types.py
uv run pytest backend/tests/component/task_manager/test_stored_event_types.py -n 0

Validated locally against an image built from this branch (3 passed in 3 min 31 s with the restart test); /pre-ci.

Checklist

• Tests added/updated
• Changelog entry added (test-only; the setting's fragment is in #10862)
• External docs updated (PR 6 of this stack)
• Internal .md docs updated (PR 6 of this stack)
• I have reviewed AI generated content

🤖 Generated with Claude Code