run fewer constraint validators

Merged
AJ
4mo ago
+258-106

Why

Merging, rebasing, or running proposed-change checks on a branch that only
changed data used to select every constraint validator against every kind in
the schema, even kinds the change never touched. That work grows with the size
of the schema, not the size of the change, and dominates merge/rebase time on
large schemas.

Goal: scope constraint validation to the kinds a change can actually
affect, so a data-only branch validates only the kinds whose data changed plus
the few kinds that transitively depend on them.

Non-goals: no change to what any validator checks, nor to the set of
validators that exist. This only changes which validators are selected to run
for a given diff.

Closes #9246

What changed

Behavioral changes:

  • A branch with only data changes no longer triggers constraint validators for
    unrelated kinds. Node-level property constraints are scoped to the kinds
    present in the data diff, the generics they inherit from, and any peer whose
    uniqueness constraint reads an attribute of a changed kind (e.g. a
    owner__name uniqueness on Car when Person data changes).
  • Schema-transition-only checkers (node.generate_profile.update,
    node.inherit_from.update) are no longer run from the data-diff path. A data
    change cannot violate them — they only fire when the schema property itself
    changes — so they stay covered by the schema-diff path and are dropped from
    the data path.
  • Internal schema kinds (SchemaNode/SchemaGeneric/SchemaAttribute/
    SchemaRelationship) are validated when they appear in the diff (i.e. when a
    schema is edited on the branch) rather than being appended unconditionally.

Implementation notes:

  • New triggered_by_data_change flag on ConstraintCheckerInterface
    (defaults True), set False on NodeGenerateProfileChecker and
    NodeInheritFromChecker. The determiner consults CONSTRAINT_VALIDATOR_MAP
    and skips any constraint whose checker is not data-triggered — the
    classification lives with the checker that embodies it, and a new checker is
    forced to answer the question rather than silently defaulting into the data
    path.
  • ConstraintValidatorDeterminer computes an "impacted kinds" set (diff kinds +
    inherited generics + peers whose uniqueness reads them) and scopes node-level
    property constraints to it instead of iterating every kind.
  • Peer-uniqueness detection uses parse_schema_path for a structured view of
    each constraint element instead of string partitioning.
  • A kind that is absent from the schema (e.g. a kind whose schema the branch
    also deletes) is skipped instead of raising.

What stayed the same: the validators themselves are unchanged; schema-diff
constraint selection is unchanged; every caller still unions data-diff
constraints with schema-diff constraints, so selection is only ever narrowed
for data changes and never for schema changes.

How to review

Suggested order:

  1. backend/infrahub/core/validators/determiner.py — the scoping logic
    (_get_impacted_kinds, _get_property_constraints_for_impacted_kinds, and
    the triggered_by_data_change filter in the per-schema/per-field helpers).
  2. interface.py + node/generate_profile.py + node/inherit_from.py — the
    new flag and where it is set False.
  3. test_determiner.py — each scenario asserts the exact constraint set the
    determiner returns.

The risk to scrutinize is under-selection: narrowing which validators run at
merge/rebase/PC-check time must not drop a validator a change could actually
violate. The integration test test_schema_duplicate_merge (duplicate schema →
SchemaNode uniqueness violation at merge) and the exact-set unit tests guard
this.

Impact & rollout

  • Backward compatibility: no API or schema changes; validation is narrowed
    only for data-only diffs, never for schema diffs.
  • Performance: the motivating win — fewer validators dispatched on data-only
    branch merges, rebases, and proposed-change checks; the count no longer scales
    with total schema size.
  • Config/env changes: none.
  • Deployment notes: safe to deploy.

Checklist

  • Tests added/updated
  • Changelog entry added
  • External docs updated (n/a — internal validation behavior)
  • Internal .md docs updated (internal knowledge and AI code tools knowledge)
  • I have reviewed AI generated content

Summary by cubic

Scopes schema integrity validators to only the kinds impacted by data or schema changes and skips schema-only checks on data diffs. Fixes schema diffing and first-time init so identical branches start with an empty core-schema diff; addresses IFC-2604 by cutting full-DB uniqueness scans and speeding up PC pipelines.

  • Bug Fixes
    • Scope node-level constraints to changed kinds, their generics, and peers referenced by uniqueness paths; skip kinds absent from the schema.
    • Add triggered_by_data_change on checkers and mark node.generate_profile.update and node.inherit_from.update as schema-only so data-only PCs don’t run them.
    • Use parse_schema_path to detect peer-attribute uniqueness checks and include those schemas only when needed.
    • Compare id-less schema elements by value; remove PYTEST_RUNNING shortcut; preserve empty optional text fields; load deprecated_models on first init; add tests for empty core-schema diff, reduced validator set, and hierarchy edge diffs selecting both endpoints.