feat(regeneration): target artifacts from an after-merge generator's output

Draft
PO
4mo ago
+582-149

Summary

Refines the post-merge generator → artifact cascade: instead of regenerating every artifact once
a generator runs, regenerate only the artifacts its output affects.

Change

A generator dispatched by the follow-up runs after the merge diff was captured, so its writes are
absent from that diff. Once every generator has run, this diffs the target branch over the window
they ran in and selects only the artifacts that read the changed data (reusing the artifact half of
the merge selector). Full regeneration stays the fallback if that output can't be captured or
selected, so no artifact is left stale. Direct and proposed-change merges take the same path.

How tested

LevelCoverage
Unittargeted dispatch, generator-before-capture sequencing, blanket fallback
Componentreal selector + graph, stubbed generator output → only the affected artifact
Integrationreal follow-up flow, empty capture → merge-diff selection stands
Backend unit suitepasses

Live validation (full battery, demo stack, SELECTIVE_EXECUTION_AFTER_MERGE=true)

Driven over the API against a live demo stack, asserting rendered artifact content (not workflow calls):

#ScenarioVerdictObservation
1Single-member narrowing✅diff=1 → rendered 1/3, others untouched
2Multi-member✅diff=2 → rendered 2/3, third untouched
4Generator → artifact cascade✅after-merge generator writes description, artifact re-renders
5Wide signal (query changed)✅regenerate_all_members=True, rendered 4/4
6Arista mtu (demo-edge)✅startup-config updated to new mtu; oc-interfaces correctly excluded (does not read mtu)
9Diverged main (main gains a member)✅branch member refreshed, no fallback
—Empty diff✅0 generator / 0 artifact — no wasted work
7Composition cascade (artifact inlines another)🔴 gapcomposite permanently inlines the stale section

Finding from the battery:

  • Composition cascade gap (open). An artifact inlining another's output (composite ← section) isn't
    re-triggered when the upstream refreshes — both dispatched in parallel, no ordering — so the composite
    keeps the stale section. Documented by test_merge_composition_cascade.

A/B performance (demo stack, 250-member group, people-only)

Dataset: 250 TestingPerson in a single people group; each member feeds 1 generator
(person_description_generator, writes description) and 3 artifacts — Section config,
Composite config (inlines the section), and the generator-fed Person cascade — so a full
regeneration touches ~250 generators + ~750 artifacts.

Rename one member and merge; wall-clock from merge to regeneration drain, median of 3 reps:

ModeMedianRegen flows dispatched
OFF (full regeneration)~302 s~1001 (250 generators + 750 artifacts)
ON (selective)~27 s~6 (the single impacted member)

~11× faster wall-clock — a conservative floor, since the ON time is dominated by the drain-detection
window; the dispatched regeneration work itself drops ~167× (1001 → 6 flows).

How to review

  1. backend/infrahub/core/merge/selective_regen/generator_diff_capturer.py
  2. select_artifacts in selective_regen/orchestrator.py
  3. regeneration_dispatcher.py — _dispatch_plan + _artifacts_from_generator_output
  4. _build_post_merge_regeneration_dispatcher in core/branch/tasks.py
  5. Tests — unit → component → integration
  6. (Optional) run live: add a group member, merge, watch for Targeted N artifact definition(s)

🤖 Generated with Claude Code

https://claude.ai/code/session_012crFmdtPR3Qm2ZCDK8e2Wk