Harden GL context-loss recovery and gate dashboard reliability in CI

Merged
AL
4mo ago
+575-117

Summary

  • Context loss now quiesces every source of deferred GPU work — draw RAF, wheel-zoom RAF and pending zoom, view animation, view/re-bin timers — and bumps the request sequence so pre-loss kernel/worker replies cannot mutate the rebuilt context
  • Streamed append payloads stay canonical while the context is down; restore rebuilds all GPU state from the latest payload
  • A failed restore no longer throws from the event handler: it reports context_restore_failed, stays explicitly failed, and ignores later restore events on the detached canvas
  • Recovery state is observable via root.dataset.fcContextState and context_lost/context_restored/context_restore_failed chart events
  • The dependency-free render smoke forces three loss/restore cycles with queued deferred work, asserts quiescence and pixel-identical rebuilds each cycle, and verifies post-restore interaction
  • CI gains a dashboard reliability smoke: bench_dashboard.py 10/20/50 must produce a loss-free, fully nonblank 10-chart row within catastrophic budgets, enforced by verify_benchmark_report.py
  • Interaction and visual budgets are pinned in the verifier so a benchmark change cannot silently relax its own gate
  • Timing regressions beyond 2x stay advisory; beyond 4x fail hard
  • Visual y-axis checks keep asset rows strict while tolerating Chromium glyph-antialias differences on generated charts
  • Reformatted examples/demo.ipynb (AST-identical) so repo-wide ruff format --check passes and no longer stops CI before browser tests

Validation

  • cargo build --release
  • pytest -q: 888 passed (8 new tests, including negative paths for every new gate)
  • ruff check ., ruff format --check . (now clean repo-wide), ty check
  • scripts/render_smoke_nonumpy.py: 3 recovery cycles, all pixel hashes identical, quiescence and post-restore zoom verified
  • scripts/visual_regression_smoke.py: 9 charts, 18 assets, 2 chrome shells OK
  • CI gate end-to-end: dashboard 10/20/50 run → 10-chart row complete (0 losses), 20/50 partial with metrics retained → verify_benchmark_report --kind dashboard-browser passes
  • Notebook change verified formatting-only: all 16 code cells AST-identical, markdown unchanged

Notes for reviewers

The 10-chart smoke gate includes hard timing budgets (render_ms ≤ 5 s). Software-GL render is ~1.8 s on a fast Mac; GitHub's SwiftShader runners are typically 2–4× slower, so watch the first CI runs — the loss-free/nonblank gate is the load-bearing part and is timing-independent, so the budgets can be raised without weakening it if they prove noisy.

🤖 Generated with Claude Code