chore: source operations gateways are thread safe
Description
Source-operation gateways (SourceOperations subclasses) are now thread safe: one gateway instance may be called from several threads at once. The capability check runner relies on this, because the next PR (#16259) runs a run's checks in parallel on one shared gateway. The contract is in the SourceOperations docstring and in the connectors feature map, with three patterns: lock lazy client builds, use one client per thread for SDKs that are not thread safe, and refresh tokens under a lock.
Audit of every gateway:
| Gateway | Finding | Change |
|---|---|---|
| GitHub | PyGithub keeps each request's URL and headers on one shared connection object, so two threads could send each other's request. | One Github client per thread. |
| SharePoint | The office365 ClientContext queues pending queries, and its cache could hand a thread the context of another site. | One REST context per thread. |
| Confluence | probe_site changed the shared client's timeout while other threads used it. | Probe on a new client, then cache it. |
| HubSpot | Client and rate limiter were built without a lock. The shared rate_limit_builder was not thread safe: once parallel calls pushed the count past the limit, its == check never triggered again. | Locked builds. rate_limit_builder locks its call history and uses >=. This also fixes Drupal Wiki, ClickUp, Braintrust and Celery, which share the helper. |
| Linear | With a static credentials provider, two threads could use the same OAuth refresh token. | Refresh under a gateway lock. |
| Google Drive, Jira, Slack, Notion, YouTrack, OneDrive, Outlook, Microsoft Graph | Already safe: locked builds, per-thread Google clients, thread-safe SDK clients, or a harmless duplicate build. | None. |
How Has This Been Tested?
- One race test per fixed gateway, using threads and barriers. Each test fails on the old code, except GitHub's, which uses the new per-thread client store.
tests/unit/onyx/connectorsandtests/unit/ee/onyx/external_permissions: 4855 passed.
Additional Options
- [Optional] Please cherry-pick this PR to the latest release version.
- [Optional] Override Linear Check
🤖 Generated with Claude Code
Summary by cubic
Makes SourceOperations gateways safe to call from several threads at once, so the capability check runner can share one gateway across parallel checks. The contract now lives in the SourceOperations docstring and the connectors feature map.
- GitHub keeps one
PyGithubclient per thread, since a shared connection object could send each thread's request over another thread's connection. - SharePoint keeps one office365 REST context per thread, since a context queues its pending queries on itself.
- Confluence
probe_siteprobes on a new client, leaving the cached client's connection settings untouched for other threads. - HubSpot builds its client and rate limiter under a double-checked lock;
rate_limit_builderlocks its call history and sleeps while>= max_calls, which also fixes Drupal Wiki, ClickUp, Braintrust, and Celery. - Linear refreshes an OAuth token under a gateway lock, covering static credential providers whose rotation lock is a no-op.
- Added one race test per fixed gateway plus one for the rate limiter; all 4855 unit tests in the connector and external-permission suites pass.
Summary by CodeRabbit
-
Bug Fixes
- Improved connector reliability during concurrent operations.
- Prevented excess requests during simultaneous rate-limit checks.
- Prevented duplicate OAuth token refreshes and authentication conflicts.
- Isolated SharePoint and GitHub client usage across concurrent operations.
- Prevented concurrent Confluence probes from interfering with active operations.
- Ensured shared HubSpot resources initialize safely.
-
Documentation
- Added guidance requiring connector operations to support safe concurrent use.