feat(python): add preserveUnknownFields option

2Y
4h ago
+635-36

Why

Pydantic drops fields a model does not declare. Python SDKs therefore lose undeclared fields in nested JSON request and response bodies: they are not sent to the API, and they disappear from response models, so a response cannot be sent back unchanged.

What changed

Add python.preserveUnknownFields (default false). When enabled:

  • generated models and error bodies allow extra fields, including schemas with additionalProperties: false; explicitly typed additional properties still validate;
  • each model serializer reads every declared field from the key Pydantic emitted (alias or field name, including the alias-keyed output a nested model is serialized from again) and removes it, then copies the remaining keys, the unknown fields, back into the output. With the option disabled the serializer reads fields as before;
  • marshal_json keeps nested nulls and excludes only a top-level None; optional fields left unset are still omitted;
  • UNSET ignores extra fields;
  • generated test assertions compare only the fields their examples declare, through a drop_unexpected_fields test helper, because mock responses can carry more. Declared additional properties keep their exact key set, and their values are compared the same way.

Known limitations

  • When a payload contains both a field's API name and an undeclared key equal to its Python name (displayName and display_name), API-name dumps (model_dump(by_alias=True), the request body) keep both keys, but Python-name dumps (the default model_dump()) show the undeclared value in place of the declared one. Plain Pydantic behaves the same with extra="allow" and populate_by_name=True.
  • Additional properties assigned under a key the model already emits replace the declared value in the output.
  • Flattened operation arguments accept only declared fields, so undeclared top-level request fields need flattenRequests: false; nested objects keep theirs. The option description states this.
  • Unknown fields are carried in JSON bodies only; form, multipart, query and header serialization use declared fields.
  • As in #149, a dump that combines exclude_none=True with include, exclude, exclude_unset or exclude_defaults can still write excluded required fields back as None.

types/basemodel.py becomes a template (basemodel.py.stmpl), so the review SDK copy now goes through the Python formatter. Apart from that formatting, the review SDK gains only preserveUnknownFields: false in gen.yaml.

Test coverage

  • New unknown-fields uber fragment: an echo operation whose body nests objects, arrays, maps, discriminated and plain unions, allOf, open/typed/closed additional properties (including additional properties typed as an object schema), empty objects, open enums and nullable fields, plus a flattened variant and an error response.
  • New test ID unknown-fields-json-round-trip, skipped for other targets. The tertiary TypeScript overlay removes the echo operation because its inline success and error bodies get the same generated name in that configuration.
  • primary (option disabled): unknown fields are still dropped.
  • secondary (enabled, flattenRequests: false): request/response round trips for dicts and models, response-only and error-body extras, dump filters, typed additional-properties validation, top-level None handling.
  • quaternary (enabled, flattenRequests: true): nested unknown fields through flattened arguments, sync and async; drop_unexpected_fields ignores unknown fields inside typed additional-property values but still reports extra map entries and changed values.

Stack

  1. #145: unrecognized enum serialization
  2. #149: honour include/exclude filters in model dumps
  3. This PR: preserveUnknownFields
  4. #147: modelSerializationMethods

Testing

  • TARGET=primary make test-pythonv2 (option disabled): 723 passed.
  • TARGET=secondary make test-pythonv2: 330 passed, 1 skipped.
  • TARGET=quaternary make test-pythonv2: 256 passed, 1 skipped. The typed additional-properties helper test fails with the previous drop_unexpected_fields.
  • TARGET=review make build-pythonv2: integrity validation passes.
  • prettier passes; make check-template-pythonv2 was run on the top of the stack (#147), which contains these templates.
  • git diff --check passes.
  • TARGET=tertiary make test-pythonv2 (option disabled): 214 passed, 1 skipped.

Public-safety check

  • This change contains no credentials, customer documents, private repository URLs, private filesystem paths, or unredacted private logs.
  • Title, body, comments, and commit messages name no customers or customer-derived identifiers, private paths or trackers, or workflow provenance, and are understandable without private context (.claude/skills/public-repo-communication/SKILL.md).
  • Generated fixtures and review SDK changes are public-safe.
  • I reviewed git diff --check.