fix(examples): skip on missing credentials instead of passing vacuously
Summary
The example tests passed in CI while exercising nothing. This makes the result honest.
Every example guarded missing credentials with print(...); return. CI has no STACKONE_* or OPENAI_* secrets, and setup-nix runs uv sync --all-extras --locked, so all seven examples imported cleanly, printed a hint, returned, and passed.
Reproduced on main with no credentials set:
$ uv sync --all-extras
$ env -u STACKONE_API_KEY -u STACKONE_ACCOUNT_ID -u OPENAI_API_KEY uv run pytest examples
8 passed in 4.45s
That is worse than having no example tests, because it reads as coverage.
Changes
test_examples.py checks credentials before exec_module and skips. A skipped test is honest; a vacuously passing one is not. The two credential classes skip for different stated reasons:
| Class | Examples | Reason |
|---|---|---|
| StackOne only | auth_management, search_tools | Can eventually run against a mock once STACKONE_BASE_URL is honoured (#197) |
| Live LLM | openai, langchain, langgraph, crewai, pydantic_ai | Needs OPENAI_API_KEY; cannot run against a mock StackOne server |
The examples' own guards now raise rather than return. A guard that returns early is the mechanism that made the vacuous pass possible โ if the skip logic is ever weakened, raising fails loudly instead of silently going green. This also matches the pattern documented in .claude/rules/examples-standards.md.
openai_integration and crewai_integration additionally gained the OPENAI_API_KEY guard the other three LLM examples already had. Both construct an LLM client at runtime but neither checked for the key, so they would have raised from deep inside the client rather than at the top.
search_tools.search_and_execute keeps its print-and-return guard: it skips one section of an otherwise-working example, not the whole example.
On STACKONE_BASE_URL
No example passes base_url โ all construct StackOneToolSet() bare. So once #197 lands (SDK constructor reads STACKONE_BASE_URL), all seven honour it for free. No change was needed here, and none was made, to avoid duplicating that PR.
Verification
Before / after, same command, no credentials:
before: 8 passed in 4.45s
after: 1 passed, 7 skipped in 1.20s
Skip reasons are distinct per class โ with StackOne creds set but no OPENAI_API_KEY:
SKIPPED examples/test_examples.py:83: Skipping langchain_integration.py: needs a live LLM
(OPENAI_API_KEY), so it cannot run against a mock StackOne server
Guards confirmed to raise rather than return silently:
auth_management: raises ValueError(STACKONE_API_KEY is not set. Export it or add it to .env)
search_tools: raises ValueError(STACKONE_API_KEY is not set. Export it or add it to .env)
openai_integration: raises ValueError(STACKONE_API_KEY is not set. Export it or add it to .env)
Full suite with --all-extras: 298 passed, 27 skipped, 0 failures. ruff check clean, ruff format --check clean.
Note
These tests still cannot verify SDK behaviour in CI โ they now skip instead of lying about it. Actually exercising them requires pointing STACKONE_BASE_URL at a mock, which needs #197 plus a CI job that starts one. This PR makes the gap visible rather than hidden.
๐ค Generated with Claude Code
Summary by cubic
Fixes example tests passing vacuously in CI when credentials are missing โ they now skip honestly instead of reporting false coverage.
test_examples.pynow skips tests when StackOne or OpenAI credentials are missing, with distinct skip reasons for the two credential classes.- Examples' credential guards now raise
ValueErrorinstead of printing and returning early. openai_integrationandcrewai_integrationgained theOPENAI_API_KEYguard the other LLM examples already had.search_tools.search_and_executekeeps its print-and-return guard because it skips only one section of an otherwise-working example.