fix: treat Claude quota refusals as non-fatal in PR review health report

DO
2d ago
+159-8

Summary

This PR fixes issue #2446 (DEGRADED health status for daily-pr-review-health.yml) by handling Claude quota refusals gracefully:

  • scripts/pr_review_health.sh: Added llm_quota_refused() function to detect quota/rate-limit refusals and treat them as non-fatal. The script appends a "Log analysis skipped" section to the report and exits 0, allowing deterministic report sections to still be published.
  • tests/pr_review_health.bats: Added comprehensive tests covering quota refusals on stdout/stderr, other failures (should still fail), and successful analyses that merely discuss rate limits (should not trigger a false-positive quota detection).
  • .github/workflows/lint.yml: Added pr_review_health.bats to the bats test suite so quota handling is exercised in CI.

Problem

The daily PR review health workflow (daily-pr-review-health.yml) failed in 50% of recent runs (1 of 2). Root cause: the workflow invokes Claude for log analysis. When Claude quota is exhausted, the CLI returns a refusal message and non-zero exit. The health script treated ANY non-zero exit as a fatal failure, causing the entire health report (including critical deterministic sections) to not be published.

Fix: Recognize quota/rate-limit refusals by pattern matching, log them as a warning, add a "Log analysis skipped" note to the report, and exit 0 so the deterministic health metrics are still visible.

Risk

Medium — changes GitHub Actions workflow behavior exercised only post-merge. The risk is bounded:

  • Quota refusals are narrowly detected (grep pattern matches only Claude-specific wording like "hit your limit", HTTP 429/529, etc.).
  • Other Claude failures still exit 1 and fail the step (no regression).
  • Successful analyses are unaffected.

Test plan

Verified via:

  • shellcheck scripts/pr_review_health.sh — no warnings
  • bats tests/pr_review_health.bats — all 7 tests pass locally, suite runs in CI via lint.yml

Rollback

Revert this PR. No non-revertible side effects (no tags, migrations, or external state changes).

Monitoring

Watch daily-pr-review-health.yml runs in the Actions tab. On the next quota exhaustion, the workflow should:

  1. Emit a ::warning:: annotation mentioning "quota"
  2. Publish a report with deterministic sections (latency, outcome mix, sweep hit-rate) and a "Log analysis skipped" section
  3. Exit 0 (not fail the job)

Interaction contract

N/A — no agentic roles changed.

Checklist

  • Shellcheck passed: scripts/pr_review_health.sh (no warnings)
  • Bats tests: tests/pr_review_health.bats — all 7 tests pass
  • Linked issue: Closes #2446

Closes #2446