fix(integrations): backfill existing connections into the table #7317 reads
Summary
#7317 moved every integration lookup from integration_connections to the new connections table and shipped without moving the existing rows. It reached production on 19 September at 20:56 UTC with connections empty, so all 2,393 live integrations read as disconnected: 1,658 Linear, 682 Slack, 28 Google, 23 Notion, 1 Sentry, 1 Teams. Linear webhook deliveries logging No active connections went from 2% of requests (old deployment, just before) to 92% (new deployment). The handler answers 200, so Linear does not retry them.
This adds the backfill, a manual workflow to run it, and fixes the one reader #7317 left on the old table.
What it does
legacyConnectionRow(packages/trpc/src/lib/connectors/legacy.ts) maps an old row to the row a fresh connect would write.- Keeps the id:
automation_eventsdedupes on it, and Linear, Gmail and Calendar build event and resource keys from it. A new id would make past deliveries look new. - Moves old
configtostateunchanged: LinearnewTasksTeamId(1,379 rows), Google watch channels and sync tokens, the TeamsclientStatethat live Graph subscriptions echo back, SentryinstallationUuid. - Seals both tokens. The new table stores ciphertext and
decryptSecretthrows on plaintext. - Takes scope and auth method from the connector definitions; lowercases Google addresses, which the Gmail push lookup matches exactly.
- User-scoped rows need an
external_user_idthe old table never held. It comes fromuser_identitieswhere the old connect flow recorded one (Google 28 of 28, Notion 23 of 23, Linear 293, Slack 112), otherwise from the Superset user id, the fallback the Slack callback already uses. No reader uses that column for these connectors and a reconnect replaces it.
- Keeps the id:
packages/trpc/scripts/backfill-connections.ts: dry run unless--apply. Skips rows already copied and owners who have reconnected since, so a lookup never finds two live rows. Refuses to run withoutSECRETS_ENCRYPTION_KEY. Reads every written token back throughdecryptSecretand exits 1 if any differs. Inserts directly rather than throughupsertConnection, which rejects an external account already connected to another organization; 120 Linear workspaces legitimately are..github/workflows/backfill-connections.yml: manual dispatch,productionenvironment, dry run unlessapplyis ticked. The encryption key exists only as a GitHub secret, so this is where it has to run.- Slack
thread-sessionsnow resolves the team throughaccountConnection. It still read the old table and would have stopped resolving any workspace connected after #7317.
Testing
12 unit tests on the mapping. Rehearsed on a Neon branch of production (2,865 old rows, connections empty, a throwaway key):
| Check | Result |
|---|---|
| Dry run | Counts match production exactly, nothing written |
| Apply | 2,865 of 2,865 written, 0 conflicts, 0 tokens failing to read back |
| Apply again | 0 to copy, 2,865 already copied |
| SQL comparison with the old table | 0 differences in id, org, user, connector, state, account id and label, token expiry, refresh token presence; 0 plaintext tokens; 0 user-scoped rows without an external user id; 0 lookup ambiguities |
Production lookup and decrypt code over every row (connectionById, accountConnections, userConnection / orgConnection, connectionSecrets) | 2,865 checked, 2,393 live, no failures |
| Same check after corrupting two rows on purpose | Reports both |
| Delete 23 rows, run again | Copies exactly those 23 |
Clean checkout, no .env, frozen install, only the workflow's variables, bash -e | Runs; exits 1 without the key |
One known difference: created_at, updated_at and disconnected_at lose sub-millisecond precision passing through JavaScript dates. token_expires_at is exact. Nothing reads those at that precision.
Rollout
- Merge. Deploy Production is disabled, so this deploys nothing by itself.
- Run Backfill Connections with
applyunchecked and compare its counts with the table above. - Run it again with
applychecked. - Deploy main, which carries the
thread-sessionsfix, and re-enable Deploy Production.
Events dropped between the cutover and the backfill are not recoverable from Linear's side.
Summary by cubic
Backfills existing integration connections into the new connections table and fixes the Slack thread-sessions reader that still used the old table.
Backfill script and workflow
- Adds
packages/trpc/scripts/backfill-connections.tswith a dry run by default and a manual GitHub Actions workflow to run it against production. - Maps each legacy row to the new schema: keeps the id, moves
configtostate, seals tokens, and derives the external user id fromuser_identitiesor falls back to the Superset user id. - Skips rows already copied and owners who reconnected since, and verifies every token decrypts after writing.
Slack reader fix
thread-sessionsnow resolves the team throughaccountConnectioninstead of the oldintegration_connectionstable.
Summary by CodeRabbit
-
New Features
- Added a controlled connection migration process with dry-run reporting and optional application of changes.
- Preserved connection details, authentication data, and account associations during migration, including provider-specific handling.
-
Bug Fixes
- Improved Slack thread follow-up behavior by consistently resolving active team connections through the shared connection system.
-
Tests
- Added coverage for connection migration, credential protection, identity matching, provider-specific data, and Slack connection handling.