Add conformance replay gate to CI

Merged
BN
4mo ago
+738-8

Summary

  • Fix runner-watch conformance replay ID correlation for multi-job broker flows
  • Add committed v2.335.1 replayable golden fixtures for CI
  • Add a pull_request/push conformance workflow that starts aksh and runs runner-watch conform

Verification

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets
  • cargo test --workspace --quiet (107 passed)
  • cargo run -p runner-watch -- conform --runner v2.335.1 --aksh-url http://127.0.0.1:80 --skip-cargo-test locally: all 11 scenarios matched

Note: local conformance used 127.0.0.1:80 because this dev machine has a pfctl loopback redirect that breaks direct 127.0.0.1:9090; the CI workflow uses direct 127.0.0.1:9090 on GitHub-hosted Linux.


Summary by cubic

Add a CI conformance gate that replays official runner v2.335.1 goldens against aksh and fails on HTTP endpoint/status regressions. Also fix multi‑job broker ID correlation in runner-watch and clarify the product parity target and conformance scope in docs.

  • New Features

    • Add conformance workflow (push/PR) that builds aksh-runner-server and runner-watch, starts aksh, runs runner-watch conform, and uploads reports.
    • Commit replayable goldens under .runner-watch/golden/v2.335.1; ignore heavy .mitm and .bin capture files.
    • Gate checks endpoint coverage and status-code parity via pure Rust HTTP replay.
  • Bug Fixes

    • Correctly correlate broker job IDs across multi-job flows by syncing official and aksh IDs by delivery order; includes unit tests.