fix(python): honour include/exclude filters in model dumps

2Y
4h ago
+630-183

Why

Every generated Python model with optional or nullable fields has a wrap model_serializer. It rebuilds the output field by field from Pydantic's own dump and writes required fields, and nullable fields that were explicitly set, even when Pydantic left them out. A field that the caller filtered out therefore came back as None:

model = shared.FilteredFields(display_name="name", required_nullable=None)
model.model_dump(exclude={"display_name"})
# before: {'displayName': None, 'requiredNullable': None, 'kind': 'filtered', 'nullDefault': None, 'withDefault': 'fallback'}
# after:  {'requiredNullable': None, 'kind': 'filtered', 'nullDefault': None, 'withDefault': 'fallback'}

The same happened with include, exclude_unset and exclude_defaults, for required fields, const fields and nullable fields with a null default, including in nested models.

The write-back is needed in one case: request bodies are dumped with exclude_none=True, and must still send null for required fields and explicitly set nullable fields.

What changed

  • Every generated wrap serializer takes Pydantic's info and skips a declared field that is absent from Pydantic's dump under both its API name and its Python name, unless the dump uses exclude_none. Fields that Pydantic filters out of a dump stay out, except when the dump uses exclude_none. Both spellings are checked because a nested model's serializer also runs on its own earlier output, which is keyed by API name.
  • The read expression and the additional-properties handling are unchanged, so JSON request bodies are unchanged: marshal_json always dumps with exclude_none=True, where the new skip never runs, and it is the only dump the SDK runtime performs.
  • Fields whose API name is the empty string. Multipart file models declare their content field with alias="". The serializer computed the API name as f.alias or n, so it looked for content while Pydantic's by-alias output holds "". This was already broken: the value was lost and written back as None ({'content': None, 'fileName': 'a', ...}), and with the skip above the field would have disappeared entirely. The API name is now f.alias if f.alias is not None else n, so the value is kept under "" in model_dump(), model_dump(by_alias=True) and marshal_json. Multipart requests are serialized from field metadata, not from these dumps, and the multipart upload tests pass unchanged.
  • Review SDK: every wrap serializer gains the info parameter, the two skip lines and the new API-name line; gen.lock changes.

Known limitations

  • A dump that combines exclude_none=True with include, exclude, exclude_unset or exclude_defaults can still write excluded required fields back as None. The serializer cannot tell a field dropped for being None from one dropped by another filter, and request bodies rely on the former.
  • A field excluded from the dump is still written when an additional property uses the field's other spelling (display_name next to displayName).

Test coverage

New model-dump-filters primary fragment: FilteredFields (required renamed field, required and optional nullable fields, a defaulted field, a nullable field with a null default, a const field, a nested copy and an array of copies), FilteredFieldsWithAdditionalProperties, and an error response whose body nests FilteredFields.

test_model_dump_filters_additional.py covers:

  • include, exclude, exclude_unset and exclude_defaults, top level and nested;
  • the exclude_none limitation above;
  • default model_dump() / model_dump_json() of nested models and TypeAdapter(List[...]) of model lists keeping API names;
  • request bodies: explicit nulls (top level and nested), a required nullable field, an omitted null additional property, an additional property under a field's Python name, and a top-level None body;
  • error-data models, which have no wrap serializer: plain Pydantic filtering, and exclude honoured by the nested model;
  • a field with an empty API name (BinaryString.content) keeping its value, and staying out when excluded.

Stack

  1. #145: unrecognized enum serialization
  2. This PR: honour include/exclude filters in model dumps
  3. #146: preserveUnknownFields
  4. #147: modelSerializationMethods

Testing

  • TARGET=primary make test-pythonv2: 696 passed (17 new), including the multipart upload tests.
  • TARGET=secondary make test-pythonv2: 264 passed, 1 skipped.
  • TARGET=tertiary make test-pythonv2: 214 passed, 1 skipped.
  • TARGET=quaternary make test-pythonv2: 247 passed, 1 skipped.
  • With the serializer of the base branch restored in the generated primary SDK, the 7 tests for the changed behaviour fail and the 10 others pass, including every request-body case with identical output.
  • TARGET=review make build-pythonv2: integrity validation passes; the diff contains only the info parameter, the two skip lines and the API-name line in each of the 66 wrap serializers, plus gen.lock.
  • git diff --check passes. No TypeScript changed.

Public-safety check

  • This change contains no credentials, customer documents, private repository URLs, private filesystem paths, or unredacted private logs.
  • Title, body, comments, and commit messages name no customers or customer-derived identifiers, private paths or trackers, or workflow provenance, and are understandable without private context (.claude/skills/public-repo-communication/SKILL.md).
  • Generated fixtures and review SDK changes are public-safe.
  • I reviewed git diff --check.