feat(standards): code scanning merge protection on every default branch (CodeQL, medium_or_higher)
What
Makes code scanning merge protection on every default branch an org standard: a code_scanning ruleset rule for CodeQL with security_alerts_threshold: medium_or_higher and alerts_threshold: errors.
Why: the required CodeQL status check only says the analysis ran. GitHub's code scanning check fails only on high or critical security alerts, so a pull request that adds a medium security alert merges silently. This rule blocks that merge.
| Part | Change |
|---|---|
| Doc | standards/github-settings.md § Code scanning merge protection (All Repositories): the values, why, the floor rule, what feeds it, what a repo with no CodeQL language does, and GitHub's limits. Pointers added in ci-standards.md §2, advanced-security.md (audit check 5), standards/rulesets/README.md and AGENTS.md. The AGENTS.md note this replaces said google-app-scripts had a protect-branches ruleset with code scanning. It doesn't any more (live, it has only code-quality and pr-quality), so that note was out of date. |
| Ruleset data | standards/rulesets/code-quality.json gains the rule beside its required status checks, so apply-rulesets.sh applies it along with the rest of code-quality. |
| Never lowered | New scripts/lib/code-scanning.sh holds the one ordering of GitHub's threshold values (none < critical < high_or_higher < medium_or_higher < all; none < errors < errors_and_warnings < all). Before a PUT, apply-rulesets.sh reads the live ruleset and keeps any stricter threshold. If it cannot read the live ruleset it does not PUT, so it can never lower a stricter setting. --dry-run reports when it would keep one. |
| Sweep | compliance-audit.sh gets check_code_scanning_rule, called from check_rulesets. It reads GET repos/{o}/{r}/rules/branches/{default}, which lists the rules from every active ruleset, and compares the strictest CodeQL entry with the floor in code-quality.json. Findings: code-scanning-rule-missing, code-scanning-rule-weak-security_alerts_threshold, code-scanning-rule-weak-alerts_threshold, plus fail-closed -unreadable (the repo is also marked inconclusive) and -source-invalid. All of them use category rulesets, so they route to dev-lead:hands-off like every other ruleset finding. A stricter repo (all) passes. |
| Tests | test/scripts/compliance-audit/code-scanning-rule.bats (19 tests): at the floor, stricter, the rule living in the repo's own ruleset, layered rulesets, pagination, missing, another tool only, each weaker field, an unknown value, unreadable, not-an-array, source invalid, wiring and routing. test/scripts/apply-rulesets/apply-rulesets.bats gains 9 tests: the codified rule, create, keeping a stricter live value for each field, raising a weaker one, no live rule, unreadable live means no PUT, dry-run, and the pure payload function. apply-rulesets-tests.yml now also triggers on scripts/lib/code-scanning.sh. |
Nothing here writes any live setting. The rule reaches a repo only when the audit or the applier next runs after merge.
What the next runs will change (read live on 2026-10-20, without writing)
| Repo | Effective rule on main today | Next audit (Fri 12:57 UTC) | Next apply-repo-settings self-heal (Mon 06:17 UTC) |
|---|---|---|---|
| software-factory | CodeQL all / errors (own main ruleset) | passes, no finding | not reached (no stub) |
| .github | none | code-scanning-rule-missing | adds the rule to code-quality |
| empower-personal-dashboard | none | code-scanning-rule-missing | adds the rule |
| markets | none | code-scanning-rule-missing | adds the rule |
| bmad-bgreat-suite | none | code-scanning-rule-missing | adds the rule |
| TalkTerm | none | code-scanning-rule-missing | adds the rule |
| .github-private, incubator, repo-template, ContentTwin, broodly, google-app-scripts, broodminder-data | none | code-scanning-rule-missing | not reached (no apply-repo-settings.yml stub): needs apply-rulesets.sh --repo combsmith/<repo> code-quality with an admin token, per the runbook |
| software-factory-canary | none (own main ruleset, no code-quality) | code-scanning-rule-missing, next to the ruleset findings it already gets | not reached |
So the audit files 13 new Compliance: code-scanning-rule-missing issues, one per repo, and 0 weak-threshold issues: no repo has a weaker rule today. The self-heal then fixes 5 of them. Every repo has CodeQL working, so the rule won't block merges because a tool is missing: 12 use default setup configured, and .github and .github-private run the advanced codeql.yml. Every repo reports at least the actions language.
combsmith/software-factory and #1352
software-factory asked in combsmith/.github#1352 to be left out of the compliance audit, the scorecard and the fleet monitor. That issue is still open, and the audit has no opt-out mechanism yet. This PR doesn't add one and doesn't depend on one:
- The new check reads rules from every ruleset, so software-factory's own
mainruleset (CodeQL,all/errors, applied 2026-10-20 from its issue #95 and recorded as a departure in its decision 0014) passes. The check files nothing there. - The applier can't lower that repo. It has no
code-qualityruleset and noapply-repo-settings.ymlstub, so nothing reaches it. Even an explicitapply-rulesets.sh --repo combsmith/software-factorywould only createcode-qualitybeside its ownmainruleset. Rulesets add together, so itsallwould still be enforced. If that repo ever moves its rule intocode-quality, the floor logic keepsall. - When #1352 lands, this check is skipped along with everything else for that repo. Nothing here needs changing for that.
- software-factory-canary isn't named in #1352, so it will get the finding. If it should be exempt too, add it there.
Merge queues: verified against GitHub's docs
GitHub's merge protection page says merge protection with rulesets does not apply to "Merge queue groups", or to "Dependabot pull requests analyzed by default setup". It is "not related to status checks". The rules reference says it blocks a merge when a required tool finds an alert at the configured severity, when its analysis is still in progress, or when it "is not configured for the repository".
So in the repos with a merge queue (.github-private, software-factory), the rule is checked on the pull request, not on the merge_group run. The group is gated only by required status checks, which for CodeQL means the advanced codeql.yml's CodeQL check. Not verified: whether GitHub checks the rule when a pull request is added to the queue, or only on a direct merge. The docs don't say, and I ran no trial. The standard says this.
Checks run locally
shellcheck --severity=warning -x scripts/*.sh scripts/lib/*.sh: cleanbats test/scripts/apply-rulesets/ test/scripts/compliance-audit/: all pass (bash 5)markdownlint-cli2 "**/*.md": 0 errorstest/scripts/apply-repo-settings/reusable-contract.batsfails locally only becauseyqisn't installed here. It's unrelated and doesn't touch these files.
🤖 Generated with Claude Code