feat: validate Arazzo workflows against their source descriptions
Why
openapi arazzo validate only checks that a document is well formed on its own. A workflow can call an operationId that doesn't exist in the OpenAPI document it points at, pass a query parameter to an operation as a path parameter, skip a required request body, or read $steps.fetch.outputs.petId when fetch never defines petId. Validation passes, and these mistakes only show up when the workflow runs.
This is the second Arazzo slice. It is stacked on #305 (Arazzo 1.1 support), so the diff here only covers source-aware validation. I'll retarget it to main once #305 merges.
What changed
- Source loading (library, opt-in). New
arazzo.LoadSourceDescriptionsloads the OpenAPI and Arazzo documents referenced bysourceDescriptions. Relative URLs resolve against$self, or the document's location. It supports a virtual FS and a custom HTTP client, and the default client has a 30s timeout. A source that can't be loaded is a warning, since a source being temporarily unavailable doesn't make the document invalid. AsyncAPI sources get a hint and are skipped. A declaredtypethat doesn't match the loaded document is an error. - Source checks (library, opt-in). New
(*Arazzo).ValidateSourceschecks workflows against the loaded sources:operationId,operationPath(including webhooks and 3.2additionalOperations), remoteworkflowIdand remotedependsOntargets exist.- Step parameters match the operation's parameters by name and
in. Path-item parameters are included, and header names are compared case-insensitively. An unknownpathparameter is an error, other locations are warnings. - Required parameters are supplied by the step or the workflow. A missing required
path/query/querystringparameter is an error,header/cookieis a warning. A missing required request body is an error. - Parameters passed to workflows in other Arazzo documents match their
inputs.
- Reference checks in
Validate(always on).$steps,$workflowsand$inputsexpressions anywhere in a workflow, including payloads and criteria, must reference steps, outputs and inputs that exist. Parameters passed to workflows in the same document (stepworkflowId,gotoactions, component actions) must match the target workflow'sinputs. - CLI.
arazzo validatechecks sources by default, and--skip-sourcesopts out (e.g. offline). Output now shows each finding's severity, and only errors fail the command.
Review notes
- Behaviour change for
Validate. The reference checks run as part of normal validation, so documents that used to pass can now report errors. That's the intent, since these references can't resolve at runtime. But it's the part most likely to surprise library users, so it's worth a look. The checks stay conservative where inputs can't be known statically: inputs schemas that use$ref,allOf/anyOf/oneOf,patternPropertiesoradditionalProperties(other thanfalse) are not checked, and dotted names such as$inputs.user.namematch on any declared prefix. - Three of the third-party stress-test documents (BNPL, Online Store, Adyen) and two spec pass fixtures have genuine dangling references, such as an
$inputstypo and outputs that were never declared. I added them to the existing ignore and deviation lists instead of changing upstream fixtures. - The severities are a judgement call. A bad path parameter can't work at runtime, but an unknown header or query parameter may be deliberate (gateways, undocumented params), so those are warnings. Happy to tighten or loosen these.
cmd/openapi/go.modis repinned to this branch's commit, following the AGENTS.md workflow. It will need repinning again after any further root-module change.- Out of scope: AsyncAPI sources (structural checks only, as agreed), and request body schema validation against the operation's media types.
Testing
- Added
sources_test.go,sourcevalidation_test.go,expressionreferences_test.go,workflowinputs_test.goand a CLIvalidate_test.go. They use new fixtures underarazzo/testdata/sources/: a local OpenAPI 3.2 petstore, a shared Arazzo document, and valid and invalid workflows. A mock HTTP client andfstest.MapFScover the remote and file loading paths. - Added an
Example_validatingSourcesexample, which also generates the newarazzo/README.mdsection. .mise-tasks/test-clinow runs the localsourcesfixtures, with source checks on and off. The existing fixtures that point atexample.comsources use--skip-sources, so CI doesn't make network calls.mise cipassed locally (format, lint, tests, build, CLI integration tests).- Local patch coverage: 370 of 373 changed blocks are covered. The uncovered ones are
runValidate, which only callscmdutil.Die, and a defensive parse-failure branch that can't be reached once YAML detection succeeds.
Summary by cubic
Makes Arazzo validation source-aware and adds lint, graph, bootstrap, and explore commands for Arazzo documents.
openapi arazzo validate now loads the OpenAPI and Arazzo documents referenced by sourceDescriptions and checks workflows against them, and Validate verifies $steps, $workflows, and $inputs expressions resolve within the document. New arazzo lint runs semantic, style, and security rules (unreachable steps, dependency cycles, unused components and inputs/outputs, missing success criteria and retry delays), arazzo graph renders workflows as Mermaid or Graphviz diagrams, arazzo bootstrap generates starter workflows from an OpenAPI document, and arazzo explore is an interactive terminal browser for workflows.
Behavior changes
- Reference checks run as part of normal
Validatecalls, so documents that previously passed may now report errors. Checks stay conservative where inputs can't be known statically (e.g.$refor composed schemas), and recheck aliased nodes reused across expression scopes. - The CLI validates sources by default;
--skip-sourcesskips loading them, useful for offline validation. A source that can't be loaded or decoded is a warning, and AsyncAPI sources get a hint and are skipped. - Source loading is opt-in in the library and supports virtual filesystems and custom HTTP clients.
- A source declared without a
typeis treated as ambiguous forchannelPathchecks; only explicitly non-AsyncAPI types are errors. - Missing
path/queryparameters and request bodies are errors; unknownheader/queryparameters and missingheader/cookieparameters are warnings.