#576 Playbook Agent: grounded confidence engine (5/8)

• depends on #572
• depends on #573
• depends on #574
• depends on #575

Summary

Grounded confidence scoring where the LLM can only reduce confidence — never inflate it.

Part 5 of 8 — depends on #575 (hard rule engine).

Formula

C_final = C_ground * (1 - alpha * u_llm)

C_ground components (objective, query-backed)

Component	What it measures
Evidence completeness	Required vs optional query coverage
Signal strength	Data volume and investigation depth
Signal agreement	Supporting vs contradicting evidence ratio
Data quality	Query success rate, empty results, edge cases

Each weighted by playbook-configured values (must sum to ~1.0). The LLM's u_llm is discounted by alpha (typically 0.35-0.5).

confidenceEngine.ts

• computeConfidence(): full formula with all 4 components
• Configurable penalties for critical missing evidence, contradictions, failures
• Outputs raw 0-1 score and bucketed 1-5 score

confidenceEngine.test.ts — 6 tests

• High confidence when all evidence present and agrees
• Reduces confidence for critical missing evidence
• Penalizes contradictory signals
• Proves LLM can only reduce (same C_ground, higher u_llm = lower C_final)
• Handles query failures gracefully
• Valid 1-5 buckets across full u_llm range

Test plan

• npm test -- --testPathPattern=confidenceEngine passes (6 tests)

Depends on #575 Hard rule engine. Merge #572, #573, #574, #575 first.

Summary by CodeRabbit

• New Features

  • Introduced playbook-driven investigation framework with validated playbook loading and semantic-versioned references
  • Added rich playbook schema/types, a query catalog for reusable evidence queries, a deterministic hard-rule engine, and a grounded confidence scoring system (0–1 + 1–5 buckets)

• Tests

  • Comprehensive test coverage for playbook parsing/validation, query catalog, hard-rule evaluation, and confidence computation behaviors

Part of #659