fix: remove generated template and profile schemas with their node (closes #10049)
Why
Loading a schema change onto a branch that removes a node with generate_template: true (via state: absent) leaves the branch permanently unable to converge: workers reloading the branch schema keep the orphaned Template<Kind> entry, crash with SchemaNotFoundError during process(), and never recover — the UI shows "Loading schemas…" forever, GraphQL on non-initiating workers misses fields from that and later loads, and every subsequent load on the branch re-triggers the failure. Reproduced on 1.10.1, 1.10.5, and current stable.
Goal: removing a node with generated kinds converges on every worker, and the branch stays usable.
Non-goals: the wider no-snapshot/no-rollback design of SchemaManager.load_schema (a mid-process() failure still leaves the in-memory registry half-updated) is a separate robustness issue and is not addressed here.
Closes #10049
What changed
Behavioral changes:
- Removing a node or generic from a branch schema now removes the
Profile<Kind>andTemplate<Kind>generated from it at the same time, so a worker refreshing the branch rebuilds a consistent schema instead of failing on every refresh. - A schema that somehow already holds a generated template with no backing node reprocesses cleanly and drops the orphan, rather than raising.
- An object template generated for a generic is now removed with that generic. It never was before, so a deleted or renamed generic left its
Template<Kind>in the generics map permanently — and in the branch hash.
Implementation notes:
The production change is 20 lines in backend/infrahub/core/schema/schema_branch.py, in two layers:
SchemaBranch.delete()cascades to the generated kinds (_delete_generated_kinds). This enforces the invariant at the source: a kind leaving the schema map takes its generated kinds with it, so the orphaned state never exists. The template lookup checksself.templatesand falls back toself.generics, becausegenerate_object_template_from_nodereturns aGenericSchemafor a generic andset()routes it into the generics map._generate_weight_templatesskips a template whose backing node is missing.generate_weight()runs beforemanage_object_template_schemas()inprocess_pre_validation, so this keepsprocess()total for a registry that already holds an orphan. It mirrors the guard already present in_generate_generics_templates_weight.
An earlier revision of this branch shipped layer 2 alone, after layer 1 was reverted for breaking test_exhaustive_handles_same_uuid_vertices_after_kind_migration in CI. That failure was a test-harness artifact, not a production signal, so layer 1 is back.
Keeping both layers is deliberate. Layer 1 prevents the orphan; layer 2 means a reader that meets one anyway does not crash. Making each reader tolerant instead does not scale — _generate_generics_templates_weight already needed the same guard.
What stayed the same: no schema-format or API changes, process_pre_validation() ordering untouched, profiles and templates remain excluded from the branch hash.
Suggested review order
backend/infrahub/core/schema/schema_branch.py— the whole production change.backend/tests/unit/core/schema/test_schema_branch.py— the cheapest expression of both invariants.backend/tests/component/core/schema/schema_branch/test_process_idempotency.py— the save/reload behaviour.backend/tests/component/graph_traversal/test_path_traversal_query.py— two lines, per above.
How to review
The tests are the part worth scrutinising. Each layer of the fix has exactly one dedicated guard, verified by removing each layer and re-running:
| Code state | What fails |
|---|---|
| both layers present | everything passes |
| cascade removed | the two TestDeleteRemovesGeneratedKinds cascade tests (2 failed, 9 passed); the round-trip tests still pass, since process() prunes the orphan |
| skip removed | TestProcessWithOrphanedTemplate (1 failed, 10 passed) |
| both removed | TestSchemaBranchDbRoundtrip::test_removing_node_drops_its_generated_kinds — SchemaNotFoundError: Unable to find the schema 'TestGadget' at manager.py:1123 |
TestSchemaBranchDbRoundtrip saves the base schema to the database once for the class; each test then applies only its own change, on its own branch, writing a single kind. Two disjoint standalone nodes (TestGadget removed, TestDoodad flipped) keep the tests independent of execution order.
A functional test added earlier on this branch (backend/tests/functional/schemas/test_load_remove_template_node.py) was removed. Functional tests use a single shared registry (and SchemaManager), so this would not be capable for reproducing the bug. Several component tests that compare SchemaBranchs before and after updates have been added instead.
Unable to reproduce the generate_template flip causing a schema update crash
The issue states that flipping generate_template: true → false without deleting the node triggers the same failure, and concludes the trigger is "removal of the generated template, not node deletion per se". I was unable to reproduce this either in tests or against a local Infrahub app. Three independent checks:
_generate_weight_templatesresolvestemplate.name, which is the node kind. A flag flip leaves the node in place, so the lookup succeeds andmanage_object_template_schemasdrops the template normally.- On a live stack the flip converged cleanly on a clean instance while
state: absentfailed on that same instance. An earlier flip run on a contaminated instance did time out, but zero error lines named the flip branch and all 24 named the earlierstate: absentbranch — the sync check is global, so one wedged branch keeps it red. The issue's own steps run the removal first, which is the likely origin of the observation. - At
infrahub-v1.10.1,_generate_weight_templates,delete(), and the removal handling inmanager.pyare materially identical tostable, so the flip could not have failed that way there either.
test_disabling_generate_template_drops_template covers the flip anyway, since the issue lists it under Expected Behavior — but it passes with both layers removed, so it is a convergence guard, not a regression test for this bug.
Testing
Verified end to end on a local deployment with 1 API worker and 2 task workers, same schemas and steps against both versions:
| Scenario | stable @ c918af76e | this branch |
|---|---|---|
Remove a generate_template node (state: absent) on a branch | "Schema is still not in sync"; both workers looping SchemaNotFoundError: Unable to find the schema 'DemoDoohickey' | Schema updated on all workers |
| Subsequent additive load on that same branch | n/a — branch already wedged | Schema updated on all workers |
Flip generate_template: true → false on a branch | converges (never the bug) | Schema updated on all workers |
SchemaNotFoundError in task-worker logs | 18 branch-tagged failures | 0 |
Impact & rollout
- Backward compatibility: none broken. No API or schema-format changes. A worker whose registry is already poisoned self-heals on its next schema refresh.
- Performance: negligible — two dict lookups per delete, and a
try/exceptper template during weight generation. The component test consolidation cut that file's database work from three full save + reset + core-model-registration cycles to one (70.8s for the file). - Config/env changes: none.
- Deployment notes: safe to deploy. Clean backport candidate for the 1.10.x line — the touched code is unchanged since 1.10.1.
Checklist
- Tests added/updated
- Changelog entry added (
changelog/10049.fixed.md) - External docs updated (if user-facing or ops-facing change) — N/A
- Internal .md docs updated (internal knowledge and AI code tools knowledge) — N/A
- I have reviewed AI generated content — pending human review
🤖 Generated with Claude Code