Why

Finished task history currently grows indefinitely, while the activity log follows Prefect's seven-day retention. Operators need to keep useful activity for longer, bound task-history growth, and recover disk space on existing instances.

Enterprise availability and any edition restrictions remain undecided. The current implementation has no edition gate and preserves native Prefect overrides; confirm availability and entitlement enforcement before release.

What changed

• Three Infrahub retention settings: task_history defaults to 30 days, activity_log to 7 days, and prefect_own_events to 2 days. Each accepts durations from 1 to 365 days; Prefect's own-event retention cannot exceed activity-log retention. Existing native Prefect settings take precedence and keep Prefect's own validation; the 365-day bound applies to the Infrahub settings. Effective values are logged.
• Prefect's hourly cleanup now deletes expired finished runs and their history. Infrahub activity and Prefect's internal events can have different retention periods.
• infrahub tasks flush flow-runs [--rewrite] clears an existing task-history backlog using the task manager's effective retention. With --rewrite, it can recover unused disk space in task-history and event storage. The CLI follows a background job and reports progress and partial outcomes.
• infrahub upgrade prints task-history size, the old-run count and cleanup guidance. It does not run the operator cleanup; failure to read the size does not fail the upgrade.
• Activities queries use indexed resource IDs where their semantics allow it, read recent time windows first, and calculate a total only when selected. The page continues by timestamp and deduplicates event IDs at page boundaries.
• infrahub tasks background-services applies the same retention configuration when Prefect's background services run separately. Operator guides, configuration/CLI references and internal documentation cover the feature.

How retention works

Infrahub defines the retention policy; Prefect's background services enforce it. Task history and activity are kept independently, while Infrahub and Prefect events share the same storage.

flowchart TB
    CONFIG["Infrahub retention settings<br/>Task history · Activity log · Prefect events"]
    TYPES["Infrahub-maintained list<br/>of Prefect event types"]

    subgraph TM["Task manager"]
        APPLY["At startup<br/>Translate Infrahub settings into Prefect settings"]

        subgraph SERVICES["Prefect background services — every hour"]
            TASKS["Task-history cleanup<br/>Remove finished runs older than task_history<br/>together with their logs and artifacts"]
            EVENTS["Event cleanup<br/>Keep Infrahub events for activity_log<br/>Keep listed Prefect events for prefect_own_events"]
        end

        APPLY --> TASKS
        APPLY --> EVENTS
    end

    subgraph DB["Prefect database"]
        RUNS[("Task history<br/>Runs, states, logs and artifacts")]
        LOG[("Shared event storage<br/>Infrahub activity + Prefect internal events")]
    end

    CONFIG --> APPLY
    TYPES --> APPLY
    TASKS -->|"Delete expired task history"| RUNS
    EVENTS -->|"Delete expired events and related resources"| LOG

Increasing activity-log retention can preserve a year of Infrahub activity without also keeping a year of Prefect's internal events. The same configuration step runs before separately deployed background services start. Explicit Prefect overrides remain effective, including switches that disable cleanup.

When task history is deleted

task_history is measured from a run's end time. Both cleanup paths select top-level runs that are completed, failed, cancelled or crashed, with end_time < cleanup time − effective task_history. The default is 30 days. Deletion also removes their associated task runs, states, logs and artifacts.

• Automatically: while Prefect's background services are running, task-history cleanup is scheduled every hour by default. An expired run is deleted when a cleanup processes it, rather than at the exact instant it reaches the retention age.
• On demand: infrahub tasks flush flow-runs starts a background cleanup job immediately, using the same eligibility rules and the task manager's effective retention. This is useful for an existing backlog after an upgrade or retention reduction. --rewrite additionally requests disk-space recovery; it is not required to delete old history.
• During an upgrade: infrahub upgrade itself deletes no history; it prints the size, old-run count and cleanup guidance. Automatic cleanup can still run once the background services are active.

For example, with 30-day retention, a run that finished 31 days ago is eligible at the next cleanup; one that finished 29 days ago stays. A top-level run still running or pending is not selected just because it is old. If task-history deletion is disabled in Prefect, automatic deletion is off and the manual cleanup refuses to start.

How Activities queries flow

sequenceDiagram
    autonumber
    participant UI as Activities page
    participant API as GraphQL / PrefectEvent
    participant TM as Task-manager event endpoint
    participant DB as Prefect PostgreSQL

    UI->>API: Request events with filters and time boundary
    API->>API: Resolve branch IDs and construct event filters
    API->>API: Set include_total only if GraphQL selects count
    API->>TM: POST /infrahub/events/filter

    TM->>TM: Determine read window from effective retention
    TM->>DB: SET LOCAL plan_cache_mode = force_custom_plan

    loop Read older windows until page is full or range exhausted
        TM->>DB: Query matching events, newest first
        DB-->>TM: Matching rows
    end

    opt Count explicitly requested
        TM->>DB: Count matches across the full filter range
        DB-->>TM: Total
    end

    TM-->>API: Events and optional total
    API-->>UI: GraphQL event objects
    UI->>UI: Deduplicate IDs at inclusive page boundaries
    Note over UI,API: Load more uses the oldest displayed timestamp as until.<br/>Ascending timelines continue with since.

The two-day default makes existing Prefect internal events older than two days eligible for cleanup, including heartbeats. Infrahub activity remains at seven days and task history at thirty days by default.

Companion changes: Helm #100 applies the settings to separate services with or without a sizing preset; backup #179 adapts the operator wrapper to the new CLI contract, including colored help output.

The specs highlights

Review focus	Contract and review guidance	Implementation model	Strongest evidence to inspect
Configuration and deployment	Configuration contract, FR-001–005 (edition decision pending): do validation, precedence and effective settings agree in the task manager and separate background services?	Settings → translation → startup. Validated policy becomes native Prefect settings; both startup paths apply the same translation and preserve native overrides.	Settings/startup tests and separate-services test.
Task-history deletion	Record lifecycle and cleanup model, FR-006–010 and FR-028. When: hourly automatic cleanup, or an operator starting infrahub tasks flush flow-runs. What: finished top-level runs whose end time is older than the effective retention (30 days by default), with their associated history. Verify both paths use these rules, keep unfinished top-level runs, and preserve already completed deletions if a later batch fails. The upgrade itself only prints guidance.	Two cleanup paths, one eligibility rule. Prefect owns hourly deletion; TaskHistoryCleanup coordinates bounded TaskHistoryTables transactions for operator cleanup.	Cleanup equivalence and concurrency tests.
Cleanup API and job lifecycle	Task-manager API contract. Three new internal routes: POST /infrahub/task-history/cleanup starts the job; GET /infrahub/task-history/cleanup/{id} reports progress; GET /infrahub/task-history/size supplies the upgrade's size and old-run count. These are served under /api on the task-manager service. Review the server-owned cutoff, one-job lock across replicas, restart handling and internal-network trust assumption. The existing POST /infrahub/events/filter route is extended, not newly introduced.	Routes → job runner → lock. CleanupJob holds progress, CleanupJobs owns execution, and CleanupLock prevents overlapping operator jobs across replicas. Status is in memory; committed deletions persist.	Route tests, client tests, PostgreSQL command tests.
Operator commands and upgrade	CLI contract, FR-009–014: do exit codes, removed options and partial-result reporting match the contract, and does the upgrade remain a bounded, best-effort hint?	CLI client and upgrade hint. run_task_history_cleanup starts and follows jobs; task_history_size_hint only reads size/count. The backup wrapper adapts to the worker's CLI contract.	Flush command tests and upgrade-hint tests.
Event retention	Configuration contract, FR-016–018 and FR-027: are Infrahub events kept for their retention, related resources removed with events, and new Prefect event types detected?	Event-type registry → overrides → Prefect cleanup. PREFECT_EVENT_TYPES identifies internal events, including heartbeats, for the two-day default. Infrahub activity keeps seven days by default.	Event-retention test and full-stack event-type guard.
Activities results and paging	Filter and pagination mapping, FR-019–026 and FR-029: are filter results and branch-name semantics correct, counts conditional, and page boundaries handled?	Filter construction → query execution → continuation. Resolve branch IDs, query expanding time windows, count only when requested, then continue by timestamp and deduplicate IDs. Read-window selection checks whether event cleanup is enabled.	Filter equivalence, window equivalence, and pagination tests.

Validation of the retention corrections

• 228 focused Python tests passed, including disabled event-persistence read windows, two-day defaults and the 365-day boundary.
• All four event-type Docker integration tests passed locally and in CI. The restart fixture identifies abandoned pending claims while still checking their stored events.
• YAML, Ruff, ty, mypy and documentation Markdown checks passed; generated documentation was refreshed. Vale is unavailable locally.
• The companion Helm change passed twelve render scenarios, including separate services without presets, all five sizing presets, OpenShift variants and custom overrides. No live-cluster cleanup was run.

Decisions and evidence still needing attention

• Pagination boundary: a full page of events with exactly the same timestamp cannot advance. Assess this against FR-023 rather than treating the no-skips requirement as fully satisfied.
• Primary-node filtering: some event types hold the primary node only in a label. Those cases keep label filtering; the ID-based optimization does not cover every request.
• Trust boundary: cleanup routes follow the existing internal task-manager API model and add no authentication. The API contract records this deviation for review.