feat(#1894): answer "why isn't this PR approved?" from the real pr-review gate chain (+ why-not-approved agent)

Merged
DO
1mo ago
+601-20

Summary

Rebuilt (7dd295e, 103ee82) at the maintainer's request. The first version re-implemented every approval gate in scripts/lib/approval-diagnostic.sh (~550 lines). That copy drifted from the real gates, and every pr-review cycle found new divergences: advisory-bot comments were handled differently, an UNSUPPORTED regex the gate doesn't have, no check-run participation, a skipped fail-closed thread path, and an approval shortcut. This version makes drift impossible by running the gates themselves.

  • Diagnose mode in scripts/review-one-pr.sh (PR_REVIEW_DIAGNOSE=true).
    • Runs the real gate chain on the live PR, in order: CI → advisory bots → maintainer comments → maintainer threads → already-reviewed-at-head → escalation hold / cycle cap → automation budget.
    • Stops before any model runs.
    • Read-only (DRY_RUN; the budget is checked read-only) and never forced (FORCE_REVIEW and FORCE_RE_REVIEW are off), so it reports what a normal run would do.
    • The answer is the run's existing emit_verdict line (decision, reason, would_change, #1552). When nothing holds the PR it ends with proceed / gates-clear.
  • Every pr-review run now writes its verdict to the step summary, so "why not approved?" is answered where an operator looks (AC #1).
  • scripts/pr-approval-diagnostic.sh <pr-url> [--json] wraps a diagnose run. It adds the PR facts (review decision, approvals at the head, hold labels, merge state) and renders a Markdown or JSON report.
  • agents/why-not-approved.md, the "skill" you asked about. It's an org-wide custom agent profile, invocable from GitHub.com, VS Code and JetBrains, that runs the diagnostic and explains each reason. .claude/skills/ is gitignored in this repo as a vendor projection, so agents/ is the shared home. AGENTS.md points Claude sessions at the script and the profile.
  • Kept from the first version: the advisory gate now names the bots that drop out of the reduced denominator (AC #2), and the docstrings.

Refs #1894. This PR does not cover every acceptance criterion; the issue stays open for AC #4, #5 and #6.

Problem

From #1894: most open PRs never received an approving review, and nothing reported which gate was withholding approval or what would satisfy it.

Risk

Low to medium.

  • Diagnose mode is opt-in and only reachable through the new script. Normal runs never set it.
  • The step-summary line is best-effort and only appends text.
  • scripts/lib/approval-diagnostic.sh and its review-one-pr.sh hook are removed. On main they never existed, so the net change to review-one-pr.sh is 36 added lines.

Test plan

  • tests/test_pr_approval_diagnostic.bats: 11 tests, all pass. Every review-one-pr.sh case asserts no GitHub write and no engine run.
    • Diagnose mode: gates-clear, ci-failing (with what clears it), ci-pending without polling, FORCE_REVIEW in the environment ignored, human-escalated, already-reviewed-at-head.
    • Step-summary line.
    • Wrapper: Markdown report, --json, URL validation.
    • Agent profile: frontmatter, and every reason in its table is one review-one-pr.sh actually emits.
  • Four of these tests fail against main's review-one-pr.sh.
  • Every bats suite that loads review-one-pr.sh or advisory-review-gate.sh passes.
  • shellcheck --severity=warning, scripts/dev-lead-lint.sh, check-model-pins and the validate-agent-profiles step are all clean.

Rollback

Revert this PR. There are no non-revertible side effects.

Monitoring

  • Each pr-review run's step summary shows a decision (reason). Changes when: … line per PR.
  • bash scripts/pr-approval-diagnostic.sh <pr-url> on any stuck PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_019K6UsD7r25pAcRjJL3G6Kx