fix(latency): report incomplete browser engine coverage

Closed
TE
7d ago
+248-42

Fixes manaflow-ai/cmuxterm-hq#5179.

The latency scoreboard used to catch every browser launch error as an unstructured warning and continue. A partial Chromium/WebKit sweep could therefore look complete, including the universal “every action fits one frame” line.

This change:

  • records the selected engine set before launching browsers;
  • preserves each launch failure as { engine, error, stage: "launch" };
  • writes expected/measured engine coverage and launch failures to JSON and the GitHub summary;
  • suppresses the universal success line when engine coverage is incomplete; and
  • keeps the nonblocking --ci scoreboard and explicit --engine filtering unchanged.

The launch-selection helper has focused tests for complete coverage, one-engine failure, both-engine failure, and explicit one-engine selection. No latency samples, budgets, or runtime exit policy changed.

Performance

No perf numbers: this changes latency evidence reporting only; it does not alter measured latency, budgets, or runtime performance.

Validation will run through the managed Linux webview test lane.

🤖 Generated with Claude Code
Agent session: agent session


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Fixes manaflow-ai/cmuxterm-hq#5179.

The latency scoreboard previously caught browser launch errors as unstructured warnings and continued, so a partial Chromium/WebKit sweep could look complete and even print "every action fits one frame."

This change records the selected engine set before launching, keeps each launch failure as structured { engine, error, stage } data, and writes expected/measured engine coverage plus launch failures to the JSON output and GitHub summary. The universal success line is suppressed when coverage is incomplete, replaced by a measured/expected engine count. The run now exits non-zero when engine launch coverage is incomplete (unless --no-fail), matching the existing action-budget exit. Nonblocking --ci scoreboard behavior and explicit --engine filtering (including an invalid filter) are unchanged.

The new launch-selection helper has focused tests for complete coverage, one-engine failure, both-engine failure, and explicit one-engine selection. No latency samples or budgets changed.