Trial P-39 (pass): each App's grants are read from GitHub and the data is held to them; each key lives in the environment of the one job that holds it
Closes #26.
The trial
P-39, a new row in §6, batch 1: "Does what GitHub granted each Factory App, its permissions, its events and the repos it reaches, agree with config/apps.json?" Pass, as ruling 3 states it: for each App, app_id is the data's; the permissions and their levels are exactly the data's, no more and no fewer, and the same registered and installed; events is empty; the selection is selected; and the repos are exactly installed_on.
Verdict: pass, for all three Apps, on two fixtures:
trials/P-39.grants.json(2026-10-18):GET /apps/{slug}for each App, andGET /orgs/combsmith/installations, kept whole, read through the Maintainer's login. Nothing was minted.trials/P-39.reach.json(2026-10-19T03:24:52Z):GET /installation/repositoriesfor each installation. The Maintainer rantools/trial_p39_reach.shat the Maintainer's own terminal. Each token was narrowed tometadata: read, used once and revoked, and each revocation returned 204. The session built the fixture from the script's output directory alone.
| App | app_id | Installation | Selection | Reaches |
|---|---|---|---|---|
| combsmith-sf-controller | 5364130 | 172571513 | selected | software-factory, software-factory-canary |
| combsmith-sf-pr-review | 5364144 | 172571064 | selected | software-factory, software-factory-canary |
| combsmith-sf-reader | 5364151 | 172572220 | selected | software-factory, software-factory-canary |
Each list fits in one page (no link header). Oversight posted its verdict on #26.
What was removed
- P-39.grants.json: each request's
authorizationheader, and each installation'saccess_tokens_urlin the installations list (17 entries). - P-39.reach.json: each request's
authorizationheader. The redaction found no credential in the response. It also dropped ten response headers per exchange that a fixture may not keep (x-github-api-version-selected,access-control-*,strict-transport-security,x-frame-options,x-content-type-options,x-xss-protection,referrer-policy,content-security-policy,x-github-edge-region). Like the grants fixture,removedlists only where credentials were. A second redaction pass over the drafted fixture removed nothing.
Public exposure
trials/P-39.grants.json names every App installed on the org, third-party Apps included, with their permissions. Its x-oauth-scopes header shows the scopes of the Maintainer's gh login. Ruling 2 accepted keeping the installations read whole, and six other fixtures already carry that header.
Nothing was made in the canary, so step 8 had nothing to undo. No rule of the fake asks P-39, so nothing is grounded in sim/grounding.py.
The environments and keys (ruling 6)
All are on combsmith/software-factory. Only names, dates and branch policies were read.
| Environment | Secret | Created | Deployment branches | Admin bypass | Required reviewer |
|---|---|---|---|---|---|
factory-poller | CONTROLLER_APP_KEY | 2026-10-19T03:39:48Z | main (branch) | off | none |
factory-apply | PR_REVIEW_APP_KEY | 2026-10-19T03:39:58Z | main (branch) | off | none |
factory-run | READER_APP_KEY | 2026-10-19T03:40:06Z | main (branch) | off | none |
Each environment holds one secret. There is no repository-level secret. The Maintainer stored each key himself, and only after its App passed.
What changes
config/apps.json: each App gets itsapp_idand akeyof{environment, secret}, given by name, never by value. Schema:schemas/apps.schema.json.tools/check_apps.py: holds each App's permissions and levels to what P-39 recorded for the installation of itsapp_id, and checks that installation belongs to the App so named. With no such record, it trusts nothing. It holds each App's key to the environment of the job that holds it, by a fixed map: the Controller's tofactory-poller, the reader's tofactory-run, and every Role's tofactory-apply. Several Roles' keys may sharefactory-apply, as §4.9 givesapplyeach Role's App key. Each secret is named for its App. Swapping the Controller's and the reader's keys is refused. There are six new mutants, and the tests intests/test_tools.py::Appskill them.schemas/trial.schema.json:repois the canary, the orgcombsmith(a trial about the org and its Apps), or null (a vendor's CLI, from P-13). The counter-example isexamples/invalid/trial.neither-the-canary-nor-the-org.json.tools/check_trials.pypairs the org with what was asked, as it pairs null with a subscription:repoiscombsmithexactly when every call is of an App (/apps/…), the org (/orgs/combsmith/…) or an installation (/installation/…). The test covers both directions, and two mutants back it.read:facts.queuenow waits on #185 (P-2, the merge queue), per ruling 7, and no longer on this issue.tools/trial_p39_reach.sh, changed after the run: OUT is resolved, with links and..followed, before it is held to be outside the repo, and a revocation that does not return 204 warns that the token may be live for up to an hour and must be revoked by hand. The fixture records the run, not this guard.- §6 adds the P-39 row with its verdict, and §6.1 puts P-39 in batch 1. The skill and
trials/AGENTS.mdsay the same. - Decision record 0035: "An App's grants are read from GitHub and the data is held to them; each key lives in the environment of the one job that holds it." It decides one row of decision 0033 otherwise: a fixture's
repomay also be the org, for a trial whose every call is of the org or its Apps.
Protected edits for the Maintainer's review (write 4): §6's P-39 row, decision 0035 and config/apps.json.
Merge order
The record takes the next free number at enqueue, when this PR is clean and green. This PR enqueues before #161, so its record is 0035, the next after main's 0034. #161 takes the next free number when it enqueues. If another record reaches main first, this one is renamed to the next free number before it is enqueued.
Verify
make verifypasses in full on a cleangit archiveexport, withPATH=/usr/bin:/binandSIM_SEEDS=40.- Mutants were swept by hand on each target file this branch changes (
tools/check_apps.py,tools/check_trials.py). Each is killed by a test that passes without it.
🤖 Generated with Claude Code
Summary by CodeRabbit
- New Features
- Platform trials can now cover organization-level questions and GitHub Apps, in addition to canary-repository trials.
- App configuration checks now verify that registered permissions and repository access match GitHub, and that each app key is assigned to its designated job environment.
- Documentation
- Trial guidance now includes organization and App trials, with updated examples and decision criteria.