Add source folder variable to Python Client

Merged
PM
17d ago
+74-21

Currently the Python client generator generates a "flat layout", where the generated Python module is directly below the project root:

$ tree -d ../src8
├── docs
├── test
└── weitblick_client
    ├── api
    └── models

This is a problem for editable installs as tools like uv have to generate a complicated "finder" to load the modules:

$ cat .venv/lib/python3.14/site-packages/__editable__.weitblick_client-1.2.0.pth 
import __editable___weitblick_client_1_2_0_finder; __editable___weitblick_client_1_2_0_finder.install()
$ wc -l .venv/lib/python3.14/site-packages/__editable___weitblick_client_1_2_0_finder.py 
85 .venv/lib/python3.14/site-packages/__editable___weitblick_client_1_2_0_finder.py

Several tools like mypy and ty fail to use it as they implement their own "import mechanism":

$ ty check
error[unresolved-import]: Cannot resolve imported module `weitblick_client`

This can be mitigated by switching to the "src layout", where all files live in the sub-directory "src".

$ tree -d ../src8
├── docs
├── src
│   └── weitblick_client
│       ├── api
│       └── models
└── test

Tools like uv then only have to place the absolute path in a .pth file, which also works with mypy and ty:

$ .venv/lib/python3.14/site-packages/__editable__.weitblick_client-1.2.0.pth
/home/pmhahn/PYTHON/weitblick-client-api/api/src

Similar to https://github.com/OpenAPITools/openapi-generator/pull/12119 but in the reverse add a new configuration option "sourceFolder" to make the base folder configurable. For backward compatibility it is empty by default, which generates the previous "flat layout". Set it to "src" to generate the "src layout".

@cbornet @tomplus @krjakbrjak @fa0311 Please have a look. My Java coding experience is ~15y old and I have never used Mustard.

PR checklist

  • Read the contribution guidelines.
  • Run the following to build the project and update samples:
    ./mvnw clean package || exit
    ./bin/generate-samples.sh ./bin/configs/*.yaml || exit
    ./bin/utils/export_docs_generators.sh || exit
    
    (For Windows users, please run the script in WSL)
    Commit all changed files.
    This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
    These must match the expectations made by your contribution.
    You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
    IMPORTANT: Do NOT purge/delete any folders/files (e.g. tests) when regenerating the samples as manually written tests may be removed.
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Running the test fails for me because of some C++ errors:

[ERROR] Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 12.35 s <<< FAILURE! -- in org.openapitools.codegen.cppboostbeast.Oas31ExactRuntimeTest
[ERROR] org.openapitools.codegen.cppboostbeast.Oas31ExactRuntimeTest.generatedClientHonorsCompositionAndParameterWireSemantics -- Time elapsed: 7.950 s <<< FAILURE!
java.lang.AssertionError: 
Generated C++ runtime objects must compile cleanly:
compile-0.log:
/home/phahn/REPOS/MISC/openapi-generator/modules/openapi-generator/src/test/resources/3_1/cpp-boost-beast-client/oas31-generated-runtime-regression.cpp: In member function ‘virtual org::openapitools::client::api::HttpResponseData {anonymous}::FakeHttpClient::executeStream(const std::string&, const std::string&, const std::string&, const std::map<std::__cxx11::basic_string<char>, std::__cxx11::basic_string<char> >&, org::openapitools::client::api::SseEventCallback, const org::openapitools::client::api::SseStreamOptions&)’:
/home/phahn/REPOS/MISC/openapi-generator/modules/openapi-generator/src/test/resources/3_1/cpp-boost-beast-client/oas31-generated-runtime-regression.cpp:95:64: error: missing initializer for member ‘org::openapitools::client::api::HttpResponseData::body’ [-Werror=missing-field-initializers]
   95 |             return {boost::beast::http::status::bad_request, {}};
      |                                                                ^
cc1plus: all warnings being treated as errors
 expected [true] but found [false]
	at org.testng.Assert.fail(Assert.java:111)
	at org.testng.Assert.failNotEquals(Assert.java:1578)
	at org.testng.Assert.assertTrue(Assert.java:57)
	at org.openapitools.codegen.cppboostbeast.Oas31ExactRuntimeTest.compileGeneratedRuntime(Oas31ExactRuntimeTest.java:704)
	at org.openapitools.codegen.cppboostbeast.Oas31ExactRuntimeTest.generatedClientHonorsCompositionAndParameterWireSemantics(Oas31ExactRuntimeTest.java:228)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:141)
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:686)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:230)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:63)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:992)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:203)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:154)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:134)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at org.testng.TestRunner.privateRun(TestRunner.java:739)
	at org.testng.TestRunner.run(TestRunner.java:614)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:421)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:413)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:373)
	at org.testng.SuiteRunner.run(SuiteRunner.java:312)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1274)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1208)
	at org.testng.TestNG.runSuites(TestNG.java:1112)
	at org.testng.TestNG.run(TestNG.java:1079)

[ERROR] org.openapitools.codegen.cppboostbeast.Oas31ExactRuntimeTest.generatedClientPreservesAdditionalPropertiesWhenEnabled -- Time elapsed: 4.398 s <<< FAILURE!
java.lang.AssertionError: 
Generated C++ runtime objects must compile cleanly:
compile-0.log:
/home/phahn/REPOS/MISC/openapi-generator/modules/openapi-generator/src/test/resources/3_1/cpp-boost-beast-client/preserve-additional-properties-runtime-regression.cpp: In function ‘int main()’:
/home/phahn/REPOS/MISC/openapi-generator/modules/openapi-generator/src/test/resources/3_1/cpp-boost-beast-client/preserve-additional-properties-runtime-regression.cpp:50:32: error: possibly dangling reference to a temporary [-Werror=dangling-reference]
   50 |     const boost::json::object& conflictOutput = extraFields.toJsonValue().as_object();
      |                                ^~~~~~~~~~~~~~
/home/phahn/REPOS/MISC/openapi-generator/modules/openapi-generator/src/test/resources/3_1/cpp-boost-beast-client/preserve-additional-properties-runtime-regression.cpp:50:84: note: the temporary was destroyed at the end of the full expression ‘org::openapitools::client::model::ExtraFields::toJsonValue() const().boost::json::value::as_object()’
   50 |     const boost::json::object& conflictOutput = extraFields.toJsonValue().as_object();
      |                                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
cc1plus: all warnings being treated as errors
 expected [true] but found [false]
	at org.testng.Assert.fail(Assert.java:111)
	at org.testng.Assert.failNotEquals(Assert.java:1578)
	at org.testng.Assert.assertTrue(Assert.java:57)
	at org.openapitools.codegen.cppboostbeast.Oas31ExactRuntimeTest.compileGeneratedRuntime(Oas31ExactRuntimeTest.java:704)
	at org.openapitools.codegen.cppboostbeast.Oas31ExactRuntimeTest.generatedClientPreservesAdditionalPropertiesWhenEnabled(Oas31ExactRuntimeTest.java:286)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:141)
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:686)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:230)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:63)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:992)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:203)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:154)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:134)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at org.testng.TestRunner.privateRun(TestRunner.java:739)
	at org.testng.TestRunner.run(TestRunner.java:614)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:421)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:413)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:373)
	at org.testng.SuiteRunner.run(SuiteRunner.java:312)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1274)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1208)
	at org.testng.TestNG.runSuites(TestNG.java:1112)
	at org.testng.TestNG.run(TestNG.java:1079)

As the same tests fail on master () and I have not changed anything related to those tests, I'm ignoring the failures and post this PR anyway. CONTRIBUTING.md does not even provide any hint that libboost-json (and more) is needed. I'm using Ubuntu 24.04 LTS on amd64.


Summary by cubic

Adds a sourceFolder option to the Python client generator so generated code can use the src layout instead of the flat layout.

The flat layout (default, when sourceFolder is empty) generates the Python module directly under the project root, which breaks editable installs in tools like uv, mypy, and ty. Setting sourceFolder to src places all generated source under src/, which those tools handle correctly. setup.py is updated to use package_dir and find_packages(where=...) when sourceFolder is set.

  • New CLI/config option sourceFolder, empty by default for backward compatibility.
  • setup.mustache emits package_dir and a where argument for find_packages when the option is set.
  • pyproject.mustache uses sourceFolder in mypy targets and Poetry packaging when set.
  • Sample config and docs updated to document the option.