feat(harness-audit): GitHub + deploy-target categories, big-phase Stop hook

Closed
KN
5mo ago
+428-353

Summary

Extends the deterministic /harness-audit engine with two new category families and a phase-boundary nudge hook.

New categories

  • GitHub Integration (always-on, 0-10): workflows present, PR template, issue templates dir, CODEOWNERS, Dependabot/Renovate config.
  • Per-provider Integration (conditional, 0-10 each): vercel / netlify / cloudflare / fly. Only scored when a provider marker file is detected. Checks: config file, deploy script in package.json, .env.example documents provider keys, GitHub workflow references the provider.

JSON output contract changes

  • max_score now computed dynamically (no longer hardcoded 70).
  • New fields: applicable_categories[], category_count, rubric_version: "2026-05-19".

Big-phase Stop hook

Registers big-phase-nudge.js in hooks/hooks.json. Fires when any of:

  • = 5 distinct files touched

  • = 200 non-lockfile lines added

  • Dependency manifest modified (package.json / Cargo.toml / pyproject.toml / go.mod)

Lockfile guard: if > 80% of added lines are in lockfiles, suppress. Honors CLAUDE_QUIET=1 and ECC_PHASE_NUDGE_OFF=1. Aggregates metrics only — never logs raw transcript content.

Test plan

  • node tests/scripts/harness-audit.test.js passes locally.
  • node scripts/harness-audit.js repo --format json on a Vercel-marker repo shows the Vercel category and a dynamic max_score.
  • Same script on a non-Vercel repo omits the Vercel category but still scores GitHub Integration.
  • JSON output contains applicable_categories[], category_count, and rubric_version.

Summary by cubic

Adds a GitHub Integration category and conditional deploy-target categories to the deterministic harness-audit, plus dynamic scoring and a big‑phase Stop hook to nudge before large changes.

  • New Features

    • GitHub Integration (0–10): checks for workflows, PR template, issue templates, CODEOWNERS, Dependabot/Renovate.
    • Deploy-target categories (0–10 each): Vercel, Netlify, Cloudflare, Fly. Only scored when a marker is present (vercel.json, netlify.toml, wrangler.toml/jsonc, fly.toml). Checks config, package.json scripts, .env.example keys, and workflow usage.
    • JSON output: dynamic max_score, plus applicable_categories[], category_count, and rubric_version: "2026-05-19".
    • Big‑phase Stop hook: nudges when ≥5 files, ≥200 non‑lockfile lines, or a dependency manifest changes. Suppressed if >80% of added lines are lockfiles. Respects CLAUDE_QUIET=1 and ECC_PHASE_NUDGE_OFF=1. Docs updated.
  • Migration

    • Do not assume a fixed total. Read max_score and use applicable_categories/category_count to understand active categories.
    • To activate provider scoring, commit the marker file, reference the provider in scripts/workflows, and document env keys in .env.example.
    • The Stop hook emits a single‑line nudge on large phases. Disable with ECC_PHASE_NUDGE_OFF=1 or quiet with CLAUDE_QUIET=1.

Summary by CodeRabbit

Release Notes

  • New Features

    • Expanded audit scoring to include GitHub Integration and deployment provider categories (Vercel, Netlify, Cloudflare, Fly).
    • Enhanced audit reports with applicable categories count, maximum score, top actions, and recommended skills.
  • Bug Fixes

    • Cleared inactive hook configurations.
  • Tests

    • Added comprehensive coverage for new audit categories and scoring behavior.

Review Change Stack