feat(harness-audit): GitHub + deploy-target categories, big-phase Stop hook
Closed
KN
5mo ago+428+428+012345678901234567890123456789-353-353-012345678901234567890123456789
Summary
Extends the deterministic /harness-audit engine with two new category families and a phase-boundary nudge hook.
New categories
- GitHub Integration (always-on, 0-10): workflows present, PR template, issue templates dir, CODEOWNERS, Dependabot/Renovate config.
- Per-provider Integration (conditional, 0-10 each): vercel / netlify / cloudflare / fly. Only scored when a provider marker file is detected. Checks: config file, deploy script in
package.json,.env.exampledocuments provider keys, GitHub workflow references the provider.
JSON output contract changes
max_scorenow computed dynamically (no longer hardcoded 70).- New fields:
applicable_categories[],category_count,rubric_version: "2026-05-19".
Big-phase Stop hook
Registers big-phase-nudge.js in hooks/hooks.json. Fires when any of:
-
= 5 distinct files touched
-
= 200 non-lockfile lines added
- Dependency manifest modified (
package.json/Cargo.toml/pyproject.toml/go.mod)
Lockfile guard: if > 80% of added lines are in lockfiles, suppress. Honors CLAUDE_QUIET=1 and ECC_PHASE_NUDGE_OFF=1. Aggregates metrics only — never logs raw transcript content.
Test plan
-
node tests/scripts/harness-audit.test.jspasses locally. -
node scripts/harness-audit.js repo --format jsonon a Vercel-marker repo shows the Vercel category and a dynamicmax_score. - Same script on a non-Vercel repo omits the Vercel category but still scores GitHub Integration.
- JSON output contains
applicable_categories[],category_count, andrubric_version.
Summary by cubic
Adds a GitHub Integration category and conditional deploy-target categories to the deterministic harness-audit, plus dynamic scoring and a big‑phase Stop hook to nudge before large changes.
-
New Features
- GitHub Integration (0–10): checks for workflows, PR template, issue templates, CODEOWNERS, Dependabot/Renovate.
- Deploy-target categories (0–10 each): Vercel, Netlify, Cloudflare, Fly. Only scored when a marker is present (
vercel.json,netlify.toml,wrangler.toml/jsonc,fly.toml). Checks config,package.jsonscripts,.env.examplekeys, and workflow usage. - JSON output: dynamic
max_score, plusapplicable_categories[],category_count, andrubric_version: "2026-05-19". - Big‑phase Stop hook: nudges when ≥5 files, ≥200 non‑lockfile lines, or a dependency manifest changes. Suppressed if >80% of added lines are lockfiles. Respects
CLAUDE_QUIET=1andECC_PHASE_NUDGE_OFF=1. Docs updated.
-
Migration
- Do not assume a fixed total. Read
max_scoreand useapplicable_categories/category_countto understand active categories. - To activate provider scoring, commit the marker file, reference the provider in scripts/workflows, and document env keys in
.env.example. - The Stop hook emits a single‑line nudge on large phases. Disable with
ECC_PHASE_NUDGE_OFF=1or quiet withCLAUDE_QUIET=1.
- Do not assume a fixed total. Read
Summary by CodeRabbit
Release Notes
-
New Features
- Expanded audit scoring to include GitHub Integration and deployment provider categories (Vercel, Netlify, Cloudflare, Fly).
- Enhanced audit reports with applicable categories count, maximum score, top actions, and recommended skills.
-
Bug Fixes
- Cleared inactive hook configurations.
-
Tests
- Added comprehensive coverage for new audit categories and scoring behavior.