scope uniqueness constraints to affected objects
Why
Uniqueness validation scanned the entire population of a kind on every data change. Merging, rebasing, or proposing a change that touched a handful of nodes of a large kind still ran a population-wide scan — slow on big kinds (hundreds of thousands of nodes) and it shipped large duplicate result sets back to Python.
Goal: scope a data-triggered uniqueness check to only the nodes the diff actually affects, so the work is proportional to the size of the change rather than the size of the kind.
Non-goals: this does not change node-save-time uniqueness (grouped_uniqueness), does not touch other constraint validators, and does not change the uniqueness error surfaced to users.
Note: uniqueness constraints do not support attribute of relationships (e.g. "device__name"). some parts of the code support this kind of uniqueness constraint, but not all of it and it is still blocked when creating/updating a schema.
Planned follow-ups
- refactor
MergeConstraintValidatorconstruction.schema_branchshould probably be an argument for this class and its dependents, not a dependency - check on integration/functional-level testing for uniqueness constraint validation and add more if necessary
- issue for supporting attributes on relationships in uniqueness constraints
Part of IFC-2796 (epic IFC-2706).
What changed
Behavioral
- Data-triggered uniqueness validation (merge, rebase, proposed-change integrity) now validates only the changed/affected nodes via a batched, index-anchored targeted query instead of a full-population scan.
- A uniqueness check is scoped to exactly the nodes whose changed field participates in the constraint — changing one node's unique field no longer drags in every other changed node of the kind.
- A newly added or broadened constraint (schema-diff origin, with no affected-node set) still runs the full-population scan, as before.
- Cross-kind constraints that read a peer's attribute (e.g.
owner__name) fall back to full-population validation, which supports them; the targeted query does not. Again, this type of uniqueness constraint cannot be added to schemas currently. - Uniqueness error messages are unchanged — attribute values render identically to the full-population path regardless of their type.
Implementation notes
- A
node_uuidscarrier is threaded end to end:Nonemeans full scan, a list means validate exactly those nodes. It rides from the diff summary through the determiner, the constraint info, and the validator request into the checker. NodeDiffFieldSummary/NodeDiffIndexnow keep the correlation between each changed field and the nodes that changed it (per-field UUID maps), so scoping targets a specific field's nodes rather than every changed node of the kind.- New here (constructor-injected, wired by
build_constraint_validator_determiner):UniquenessConstraintScoperdecides, per kind, whether uniqueness must run and which nodes it affects (None→ full scan); theConstraintValidatorDeterminerupdate threads the affected set through; constraint dedup/merge helpers collapse a node's uniqueness check into its generic's where the generic already covers it. UniquenessCheckernow runs the targeted query for a scoped set, and falls back to the full-population scan when a constraint reads a peer attribute.
What stayed the same
- The full-population path and the node-save-time uniqueness checker (
grouped_uniqueness) are untouched. - No GraphQL/API contract changes, no database schema or migration changes.
Suggested review order
core/diff/model/path.py,core/diff/query/field_summary.py,core/validators/node_diff_index.py— how per-field changed-node UUIDs are captured.core/validators/uniqueness/scope.py+core/validators/determiner.py— how the affected-node set (orNone) is decided.core/validators/uniqueness/checker.py— running the targeted query for a scoped set vs the full-population fallback.proposed_change/tasks.py,merge/constraints.py,branch/tasks.py— the three call sites that thread the affected set through.- Tests.
How to review
Focus on the scope decision (UniquenessConstraintScoper._compute_scope) and the checker's targeted-vs-full-population branch (UniquenessChecker.check / _supports_targeted) — those govern correctness. Mechanical: the attribute_node_uuids conversions across the test files.
Impact & rollout
- Backward compatibility: no breaking changes; uniqueness error messages are identical to before.
- Performance: the headline win. On a large local dataset (≈500k-node kind) the full-population scan ran 8+ minutes before OOM-killing Neo4j; the targeted query validates ~9,400 changed nodes of that kind in ~12s and ~23,500 in ~29s, and all implicated kinds complete in tens of seconds. Work is now bounded by the change, not the population.
- Config/env changes: none.
- Deployment notes: safe to deploy; no coordinated release or migration required.
Checklist
- Tests added/updated
- Changelog entry added
- External docs updated (no user-facing surface change)
- Internal .md docs updated (internal knowledge and AI code tools knowledge)
- I have reviewed AI generated content
Summary by cubic
Scope node-level uniqueness validation to only the nodes that changed each unique field (and their dependents), using diff-derived IDs and batched, index-backed queries. Full scans now run only when a constraint is added/broadened or when peer-attribute elements are present, cutting worst-case merge and schema-update time (IFC-2796).
-
Bug Fixes
- Fall back to full-population validation for peer-attribute paths (e.g., owner__name) unsupported by the targeted query.
- Reset scoper cache on determiner re-init to prevent stale scopes between diffs.
- Stringify targeted values in error paths to match full-scan messages.
-
Refactors
- Batch checks per constraint group via
TargetedUniquenessValidationQuery, paging over changed nodes with one read-only session per change. UniquenessCheckerreads branch/schema from the validator request; dependency builder and m018 migration updated.- Carry
node_uuidsthroughSchemaUpdateConstraintInfoand validator requests across proposed-change, merge, rebase, and migrations; merge/dedup constraints and standardize onConstraintIdentifier.NODE_UNIQUENESS_CONSTRAINTS_UPDATE. - Consolidate chunking into
infrahub.utilities.chunks.chunkedand replace local helpers. - Refactor
UniquenessConstraintScoperfor clarity; trim docstrings.
- Batch checks per constraint group via