ci: Run agent integration tests in their own job

Merged
AW
1mo ago
+10-3

Proposed Changes

The pipeline integration job is one of the two slowest jobs, so it sets the time of the Integration gate. Its slowest test class, TestAgentBuildIntegration in tests/integration/agents, takes about 100 s. The suite uses loadscope, which keeps a class on one worker, so the other workers wait for that class.

         suite:
+          - name: agents
+            paths: "tests/integration/agents"
           - name: pipeline
-            paths: "tests/integration/agents tests/integration/ingestion ..."
+            paths: "tests/integration/ingestion tests/integration/embedding ..."

The agent tests move to their own job with their own Neo4j. No class is split across workers, because these tests share one graph and rely on loadscope. The LLM_* secrets already apply to every suite.

How did you test it?

Per-test times from the JUnit results of a recent run: the agents directory takes 130 s of the 316 s of test time in pipeline, and pipeline ran 195 s on 4 workers. I expect agents near 170 s and the new pipeline near 120 s per job. The E2E (llm) job then sets the gate time at about 230 to 270 s, down from about 315 s. This PR's own run shows the real numbers. zizmor reports no findings on the workflow.

Notes for the reviewer

  • This adds one runner job per run.
  • The E2E (llm) job is limited the same way: two LLM tests share one class. This PR does not change it.
  • The coverage and result artifacts use the suite name, so the new job uploads integration-test-results-agents and a new Codecov name.

Checklist

  • I have updated the related issue with new insights and changes
  • I added unit tests and updated the docstrings
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:
  • I documented my code

Summary by cubic

Moves the agent integration tests out of the pipeline suite into their own agents job, so TestAgentBuildIntegration — which loadscope keeps on a single worker — no longer holds up the rest of the pipeline workers.

Notes

  • The new agents job uploads integration-test-results-agents and a separate Codecov entry.
  • Adds one runner job per run; E2E (llm) is unchanged.