Enforce independently measured statement coverage

Merged
UN
15d ago
+1018-45

The coverage gate declared statements: 100 but never measured or enforced statements. This change measures genuine AST statements independently and rejects insufficient or missing statement coverage at the declared threshold.

PM item: https://github.com/unbraind/pm-ado/blob/main/.agents/pm/issues/pm-ado-5w3m.toon

Problem and acceptance scenario

Before editing, built and packed the package, installed the archive into a real scratch pm tracker, and exercised the shipped offline ado validate command. It returned the revision-asserting-write diagnostic without an Azure call. A source containing if (flag) { return 42; } return 7; on one line then passed the gate after calling only the true path, despite statements: 100.

The regression retains lines=100 and functions=100 while setting branches=0 to isolate statement enforcement. Two of three AST statements now fail the statement gate; executing both paths reaches three of three and passes. The exact measured threshold passes; a fraction above it fails. The rebuilt archive also passed offline validation in a fresh real tracker, and its partial-statement fixture exited 1.

Design and before/after

  • Preserve Node V8 line/branch/function flags and LCOV source-presence reconciliation.
  • Add pinned Istanbul instrumentation and coverage-map dependencies for a separate statement pass over the same enumerated sources. Native type stripping preserves TypeScript source locations; AST counters distinguish multiple statements on one executable line.
  • Generate a preload that substitutes only configured source files. Propagate it to child Node processes so launcher behavior is measured, and merge test-worker counters while excluding unrelated source records.
  • Validate statement maps and integer counters, require every source in the statement report, and reject absent/corrupt reports even at a zero threshold. Clear statement reports before measurement so a stale report cannot satisfy a silent runner. Preserve existing Node options and propagate runner startup, suite and signal failures.
  • Compare raw covered/total counts without rounding. Retain the statement summary and merged counter maps as ignored runtime artifacts.
  • Update README and coverage/certification documentation. Preserve all thresholds, the four-source inventory, package behavior and owner release-approval guard.

Before: statements: 100 could pass without any statement report.
After: the partial fixture reports 2/3 statements and fails; complete execution passes at 3/3. The package measures 306/306 statements (100%).

Behavioral revert proof

Removed only the statement measurement call and its failure-status check from runCoverageGate; kept implementation, imports and tests intact. Ran:

node --test --test-name-pattern='statement threshold rejects|statement report fails closed' test/coverage-gate.test.ts

Both tests loaded and failed assertions with actual gate status 0 versus expected 1, including the malformed-report case. Restored enforcement: both passed. This proves the missing behavior without relying on a test-load failure.

Validation

Verified on committed head 119b225ec6f84bf8c53ee6c953dba0980ddc6f1a:

  • npm run release:check: PASS; 129/129 tests in each coverage pass, zero skips.
  • bun run release:check: PASS; 129/129 tests in each coverage pass, zero skips.
  • Independent statements: 306/306 (100%); Node V8 lines, branches and functions: 100% each at unchanged thresholds.
  • pm test pm-ado-5w3m --run --progress: focused linked suite PASS, 30/30. Result recording is disabled by project policy; verified results are recorded in item comments.
  • npm run changelog:full: completed after the final pm writes; changelog checks passed.
  • Docstrings: 38/38 declarations. Production audit: zero vulnerabilities. Pack dry run, provenance attestation and version-derived changelog date checks passed.
  • Real scratch tracker built-archive offline acceptance passed before and after; no pm SDK mocks. No pm SDK bugs found. The CLI correctly refused an oversized local-directory source scan; packed installation succeeded.

Coverage scope remains index.ts, scripts/coverage-gate.ts, scripts/prepare-merge-driver.ts and scripts/docstring-gate.ts. This is the configured four-source inventory, not every operational script. Remote CI/review and live Azure integration are separate from these local results.

The item claim is released and the item is unclosed for orchestrator verification. No release, tag, publish, Azure write, approval-variable change or merge was performed.

Summary by Sourcery

Enforce the declared statement coverage gate with independent AST instrumentation while retaining existing coverage and release safeguards.

New Features:

  • Enforce the configured statement coverage threshold using an independent AST-based measurement across the required source inventory.

Bug Fixes:

  • Prevent missing, stale, malformed, incomplete, or invalid statement reports from passing the coverage gate.
  • Ensure statement coverage accounts for worker execution and ignores unrelated source records.

Enhancements:

  • Preserve existing Node V8 coverage checks while adding exact, non-rounded statement threshold comparisons and runner failure propagation.

Build:

  • Add pinned Istanbul instrumentation, coverage, and TypeScript type dependencies.

Documentation:

  • Document independent statement coverage, regression evidence, measured scope, and updated certification results.

Tests:

  • Add coverage-gate tests for partial statements, exact thresholds, worker merging, invalid or missing reports, stale data, runner failures, configuration validation, and Node option preservation.

Summary by cubic

Enforces the statements: 100 coverage gate, which was declared in configuration but never actually measured. The gate previously passed sources with unexecuted statements on fully covered lines; it now measures statements independently with Istanbul AST instrumentation and rejects insufficient or missing coverage.

  • Requires a valid statement map for every configured source; missing, stale, malformed, or invalid reports fail even at a zero threshold.
  • Preserves the existing Node V8 line/branch/function checks, merges test-worker counters, and compares statement counts without rounding.
  • Adds pinned Istanbul dev dependencies, regression and fail-closed tests, and docs/statement-coverage.md documenting the new measurement.