feat(regeneration): safe-fallback escalation and direct-merge cascade for selective post-merge regeneration

Draft
PO
4mo ago
+2043-323

Summary

Selective post-merge regeneration (IFC-2704) for artifact and generator definitions: the merge
follow-up regenerates only what a merge changed when the change signal is trustworthy, and falls
back to full regeneration otherwise. This branch adds the safe-fallback escalation, the
generator → artifact cascade on both direct and proposed-change merges, the isolated artifact
member/limit filter, and consolidated test fixtures.

Changes

AreaBehavior
Untrusted-signal fallbackA definition with no computed fingerprint, or a missing/incomplete dependency closure, regenerates every member of its repository instead of narrowing. The escalation is repository-scoped: a trusted definition is still regenerated when a sibling in the same repository is untrusted.
Generator → artifact cascadeA generator dispatched by the follow-up runs after the merge diff was captured, so its output is absent from that diff. Every generator is awaited, then the artifacts are regenerated — on direct and proposed-change merges alike, so no consuming artifact is left stale.
Artifact member/limit filterThe member/limit selection is isolated so the limit boundary is covered directly.
Test fixturesShared SDK diff-summary factories across the suites; the WorkflowRecorder holds one ordered call log with per-kind views derived from it.
DocsChangelog fragment and a performance-scenarios document recording which merges take the selective path and the conditions that gate it.

Working scenarios

The backend unit suite passes. The proposed-change merge cascade is covered by the unit, component,
and fast integration selective-regen suites. Merge behavior was also validated locally against the
testcontainers integration scenarios below (asserting rendered artifact content after a real merge);
those slow integration tests land separately.

ScenarioResult
Generator → artifact cascade (direct and proposed-change merge)✅ artifacts reflect the generator's output after the merge
Generator opt-out (execute_after_merge disabled)✅ the generator is left untouched
Updated transform on main✅ renders against main's current transform, not the branch's stale copy
New definition on main✅ picked up and regenerated by the branch merge
Diverged main (main gained a member)✅ the branch member regenerates against the merged state
Member narrowing✅ only the changed member regenerates; an irrelevant-kind change regenerates nothing; deleting a member does not break the path; renders the post-merge state
Content composition refresh❌ known limitation, out of scope: a composite artifact that inlines an upstream artifact is not refreshed after a direct merge that changes the upstream

How to review

Suggested order:

  1. Spec & docs — dev/specs/ifc-2704-incremental-merge-regen/ (performance-scenarios, research, tasks) + changelog fragment.
  2. Change-signal predicates & impact — backend/infrahub/core/regeneration/ (models, predicates, impact, definitions).
  3. Selective selection — backend/infrahub/core/merge/selective_regen/ (gate → definition_selector/{base, artifact_selector, generator_selector} → fallbacks → orchestrator).
  4. Merge follow-up dispatch — regeneration_dispatcher.py, post_merge.py, core/branch/tasks.py.
  5. Artifact member/limit filter — backend/infrahub/git/models.py, git/tasks.py.
  6. Tests — unit (tests/unit/core/regeneration/, tests/unit/core/merge/selective_regen/, tests/unit/git/) → component (tests/component/proposed_change/) → integration (tests/integration/proposed_change/).

Notes

  • Draft: the testcontainers integration tests covering the scenarios above are held out of this PR for now.
  • The follow-up now behaves identically on direct and proposed-change merges, so the merge-kind flag that was threaded through the dispatch chain has been removed — the cascade decision depends only on whether any generator was selected.

Deferred to #10026 — artifact regeneration after a generator cascade

Whenever an after-merge generator is selected, the follow-up currently regenerates every artifact on the branch (the blanket TRIGGER_ARTIFACT_DEFINITION_GENERATE), on both merge kinds. This is deliberate over-execution: the generator's output lands after the merge diff was captured, so the consuming artifacts cannot be identified from that diff. Narrowing this to only the artifacts that actually consume the generator's output — and sequencing it correctly in the error/fallback path — is out of scope here and tracked in https://github.com/opsmill/infrahub/pull/10026:

  • Precision. Any selected generator forces a full artifact fan-out; enabling the feature no longer guarantees narrowed work when after-merge generators are in play.
  • Fallback sequencing. If an awaited generator run fails, dispatch falls back to the blanket full regeneration, which submits the generator and artifact triggers concurrently (no sequencing) — the same behavior as every other fallback path. The happy-path sequencing guarantee (artifacts after generator completion) does not hold on that error path.