feat: separate user-facing schema (write/read) from internal model [INFP-234]
Why
The user-facing schema was a direct dump of Infrahub's internal model: it advertised internal-only fields (e.g. inherited) as settable and described constrained fields (e.g. attribute kind) as free-form strings. Users — and especially LLM agents — produced technically-invalid schemas as a result.
Goal: generate a distinct write contract (exactly what a user may submit) and read contract (adds visible-but-not-settable fields) from the single source of truth, validate submissions against the write contract at the load boundary, and let clients validate offline via the SDK.
Non-goals (this PR): retiring the SDK's hand-written schema models, publishing the write contract in the REST OpenAPI, and extracting schema code generation off the attribute definition model — all three documented as follow-ups in dev/specs/archive/002-user-facing-schema/opsmill-implement-followups.md.
Closes INFP-234
What changed
Behavioral — POST /api/schema/load now validates every submitted node, generic, and extension against the user-facing write contract:
- Invalid settable values are rejected with a field-level error naming the dotted path and the offending value: a constrained field outside its allowed set (attribute
kind, relationshipkind/cardinality, …), a value outside its allowed range or pattern, or a missing required field. Attributekindand the other bounded fields are now published enums rather than free-form strings, so a kind that does not exist is caught at the boundary instead of failing later. - Read-only, internal, and unknown fields are tolerated and dropped, at every nesting level — a schema read back from
GET /api/schema, an export, or an older/hand-edited payload still loads without stripping anything. This is a change of direction taken during review: the first implementation rejected those fields, which made the read-back → edit → load round trip a breaking workflow for no benefit. Requirement FR-003 inspec.mdstill states the original reject rule and has not been rewritten. versionis required on the write contract. The server already required it; the generated SDK model did not, so offline validation used to pass payloads the server rejected.- Schemas can be validated offline with only the SDK installed, and the offline verdict reproduces the server's.
Implementation
- New
Visibilityaxis on schema field definitions; every field classified write/read/internal from the single sourceinternal.py. - Generator emits write/read model families into the SDK (dedicated
generate_schema_sdk.j2), driven bySdkSchemaGeneratorintasks/backend.py; internal models unchanged (byte-identical output). - The load boundary is a
model_validator(mode="before")onSchemaLoadAPIdelegating to the SDK'svalidate_schema— one implementation, so server and SDK cannot drift. The generated write models areextra="ignore", so pydantic itself drops non-write fields; there is no hand-rolled projection step. - Reserved-namespace and reserved-suffix checks unified behind
SchemaRoot.validate_reserved_names(). - Incidental: generator no longer breaks on worktree paths containing shell/regex metacharacters.
Unchanged: GET /api/schema response shape; stored-schema read-back; no database or migration changes.
Suggested review order
backend/infrahub/core/schema/definitions/internal.py— theVisibilityaxis + per-field classification.tasks/backend.py+backend/templates/generate_schema_sdk.j2— write/read generation into the SDK.backend/infrahub/api/schema.py— themodel_validator(mode="before")boundary gate.- Tests, then generated SDK models (mechanical).
How to review
Focus on the boundary gate and the visibility classification. Companion SDK PR (land first): https://github.com/opsmill/infrahub-sdk-python/pull/1135.
How to test
uv run pytest backend/tests/unit/api/test_schema_load_contract.py
uv run pytest backend/tests/unit/core/schema/test_generated_visibility.py
uv run pytest backend/tests/functional/api/test_load_schema.py # requires Neo4j testcontainers
uv run pytest backend/tests/component/api/test_40_schema.py
Full functional load suite: 17 passed locally (Neo4j testcontainers).
Impact & rollout
- Backward compatibility: a payload is rejected only for an invalid settable value — an out-of-enum or out-of-range value, or a missing required field. Payloads carrying derived or internal fields (
inherited,used_by,hierarchy, derivedkind, …) keep loading, so theGET→ edit →POSTround trip needs no stripping. Changelog fragments and an upgrade note are included. - Performance: negligible — schema load is infrequent and not a hot path.
- Config/env changes: none.
- Deployment notes: requires a coordinated SDK release; the submodule pointer references the SDK branch, so land the companion SDK PR first.
Checklist
- Tests added/updated
- Changelog entry added
- External docs updated (upgrade note)
- Internal .md docs updated (dev/knowledge)
- I have reviewed AI generated content
🤖 Generated with Claude Code
Summary by cubic
Separates the user-facing schema into SDK-generated write/read models, wires /api/schema and /api/schema/{load,check} to those contracts, and serves the generated read models directly. /api/schema/load drops read‑only/internal/unknown fields, requires version, and validates constrained values with dotted, field‑level errors. Addresses INFP‑234.
-
New Features
- Generated
infrahub_sdkmodels:InfrahubSchemaWrite/InfrahubSchemaRead; discriminatedAttributeunion; typedcomputed_attributeand dropdownchoices; shared enums; read models computekindand includehash; profile/template read models; write‑onlyextensions. - API/OpenAPI:
GET /api/schemareturns the generated read models (NodeSchemaRead,GenericSchemaRead,ProfileSchemaRead,TemplateSchemaRead);POST /api/schema/{load,check}declaresInfrahubSchemaWriteand validates viainfrahub_sdk.schema.validate_schema()(extras are dropped). OpenAPI and frontend REST types regenerated; CI guards generated SDK schema/protocol drift. - Internals: added
Visibilityon field definitions to drive write/read generation; removed the hand‑rolled write projection; unified reserved‑name checks undervalidate_reserved_names(); ADR 0010 recorded; corrected code‑generation docs.
- Generated
-
Migration
- You can submit schemas that include read/internal/unknown fields (e.g., a read‑back/export); the server strips them, then enforces the write contract.
versionis now required. - Validate offline with
infrahub_sdk.schema.validate_schema(); server and SDK give the same verdicts. - The deprecated attribute kind
Stringis removed from SDK read models and OpenAPI; useText.
- You can submit schemas that include read/internal/unknown fields (e.g., a read‑back/export); the server strips them, then enforces the write contract.