perf(service): keep repository reaches and tokens per instance
What was asked
Reading an agent's pull request on GitHub should stop reaching the repository and minting a repository token over and over. Every read went through agentPullRequestReach → GitHubApp.repositoryPullRequestToken, which runs four serial GitHub round trips before any pull request read: the installation read, a listing token mint, the listing, and the repository token mint. On top of that, an overview read whose kept published answer had lapsed did all of it twice.
Measured
This is the new route test in coding-agents-app.test.ts, with the scripted GitHub (githubReaching) and a one-off 100 ms delay on every scripted request (the delay is not committed). "Cold" means the first overview read, with nothing kept. "Warm" means the next read, which gets the kept published answer.
| GitHub requests | of them reach and mint | installation reads / listing mints / listings / repo mints | time at 100 ms per request | |
|---|---|---|---|---|
| before, cold | 18 | 8 | 2 / 2 / 2 / 2 | ~1570 ms |
| before, warm | 10 | 4 | 1 / 1 / 1 / 1 | ~730 ms |
| after, cold | 14 | 4 | 1 / 1 / 1 / 1 | ~1165 ms |
| after, warm | 6 | 0 | 0 / 0 / 0 / 0 | ~320 ms |
The four reach-and-mint requests run one after another, so each one adds a full GitHub round trip before the tab's first read. After this change, a warm read spends nothing before reading the pull request. The image route and the changes route get the same saving on each request.
What changed (packages/service/src/github/github-app.ts)
- What the GitHub App keeps per function instance: the service now holds two Effect
Caches (requireServicesAt: "lookup", aswithPublishedCachedoes). It is built withLayer.effect, so each function instance gets its own caches, and each test layer gets fresh ones.- Reach: whether a workspace's installation reaches a repository, keyed by (workspace, repository as asked), for 5 minutes. Only a reach that succeeded is kept. A failure, or a repository the installation doesn't reach, is never kept.
- A kept reach is used only while the store still links its installation to the workspace as active. That costs one store read per ask, and it means a reinstall, a disconnect, or a removal reported by webhook takes effect at once rather than when the 5 minutes run out.
- Repository token: keyed by (workspace, installation, repository, permissions). It is kept until 10 minutes before GitHub's
expires_at, and never longer than the token's one-hour lifetime. A failed mint is never kept. A token that is already inside that 10-minute window is not kept.
- Security model unchanged: a token is still minted for exactly one repository, with the same permissions as today. Every key includes the workspace, so nothing kept crosses workspaces. The caller's own access check (
agentWorkspace) still runs on every request, outside the caches. - Write tokens stay fresh: a write token is handed to a sandbox for its whole run, so it is still minted fresh each time, with its full hour. Only the reach in front of it comes from the cache.
- Member-facing reads stay fresh:
repositoryReach,workspaceRepositoryandrepositoryCiReadableanswer a member whether a repository is reached and why not, so they still ask GitHub every time. - "Don't reach twice within one overview request": this is met by the caches rather than by changing
overviewEndpoint. On a cold read, the published lookup's reach fills both caches, and the overview's ownagentPullRequestReachgets the cached token without any GitHub request (8 → 4 above). I left the overview's endpoint, wire and renderer alone, because another agent is making the Overview load progressively on top of #3221.
Tests
coding-agents-app.test.ts: two overview reads make one installation read, one listing and one repository mint between them (main makes 3 of each). A read an hour later mints one new token (main mints 2). Fails on main.github-app.test.ts: repeat asks for the pull-request token mint once. Write tokens mint on every ask. Another workspace asking for the same repository gets its own answer (none), not the first workspace's kept one. Fails on main.github-app.test.ts: if the account reinstalls after a token was kept, the next ask mints through the newly linked installation. Fails without the link check.github-app.test.ts: after a refused mint, a repository not yet reached, or a token GitHub expires within the margin, the next ask goes to GitHub again. This guards the "never keep a failure" rule.
pnpm check:affected passes: lint, discarded-effect, types, service and web tests, build.
Summary by cubic
Keeps the GitHub App's repository reaches and tokens per function instance, so reading an agent's pull request no longer reaches the repository and mints a token on every request. Previously each read ran four serial GitHub round trips — the installation read, a listing token mint, the listing, and the repository token mint — and an overview read whose kept published answer had lapsed did all of it twice.
Successful reaches are now kept for five minutes under their workspace, and tokens until ten minutes before GitHub expires them, keyed by workspace, installation, repository, and permissions.
Caching rules
- Only successful reaches and tokens are kept; failures and unreached repositories are never kept.
- Write tokens are still minted fresh for each sandbox run, and member-facing reach reads still ask GitHub every time.
- Tokens remain per-repository with unchanged permissions, and nothing cached crosses workspaces.
- A kept reach is used only while the store still links its installation to the workspace as active; the links are read after the reach, since a fresh reach mends a stale installation status before it answers.
Made with Luke