#168 fix(replay): port the engine to browser-use 0.13's CDP actor API (deterministic replay was 100% dead)

Problem

The replay engine is written against Playwright's API, but runs on browser-use 0.13's CDP actor surface, which has none of it. Verified with the pinned version (browser-use==0.13.4):

Called by the engine	On CDP actor Page/Element
page.locator / locator.wait_for	❌ doesn't exist
page.wait_for_selector / wait_for_load_state	❌ doesn't exist
page.check / page.uncheck	❌ doesn't exist
page.query_selector_all / el.is_visible() / el.text_content()	❌ doesn't exist
locator.click(force=True)	❌ no force= kwarg
locator.select_option(label=...)	❌ values only — option label text is invisible to it
locator.press	❌ Element has no press
page.screenshot(path=...)	❌ returns base64, no path=
evaluate treated as returning dict/bool	❌ always returns a string (JSON-stringified)

Because nearly every call site wraps these in broad except blocks, the failures are silent: get_best_element_handle exhausts all fallbacks and raises for every selector, so any recorded workflow with a cssSelector step fails on its first element interaction. Radio/checkbox steps only 'succeed' when the state already matches; form-validation detection always returns 'no errors' (rejected submits are reported SUCCESS); XPath fallback never fires; the multi-strategy finder reads node.text/node.aria_label etc. that don't exist on the slotted EnhancedDOMTreeNode, so no semantic strategy can ever match.

Fix (reviewable commit-by-commit)

1. test(contract) — an import-time API-contract test (workflows/tests/test_browser_use_contract.py, no browser launch):
   • asserts every Page/Element/Browser attribute the engine relies on exists with a compatible signature (fails at CI time on a breaking browser-use upgrade, instead of silently at replay time)
   • a static scan asserting no Playwright-only API call exists outside the compat layer — it failed with 35 real call sites before the port and is green after; it also prevents this bug class from regressing
2. fix(replay) deterministic path — new workflow_use/compat/cdp.py (single module that knows the Playwright↔CDP differences: polling wait_for_element, visibility via geometry+computed style, XPath resolution via a temporary marker attribute, JSON-decoding of evaluate string returns, readyState-based load waits, base64 screenshot-to-file, checkbox state setter, select-by-visible-text with proper input/change events) + ports controller/utils.py, controller/service.py, workflow/service.py. Also:
   • DISABLED_DEFAULT_ACTIONS listed pre-0.2 action names, so only 4 of 27 matched and 16 agent-only builtins stayed registered while custom click/input/scroll silently overwrote builtins — replaced with excluding all builtins by their real names + explicit registrations
   • go_back/go_forward steps built an empty ActionModel(), which Tools.act no-ops while reporting success — now real CDP navigations
   • dropped Playwright-only :has-text()/:visible selectors (invalid CSS that throws in querySelectorAll)
3. fix(replay) semantic executor / validation / element finder — same treatment for semantic_executor.py (11 page.check sites, 5 wait_for_selector sites, select-by-visible-text, AI extraction called ainvoke(str) and read .content while the API takes a message list and exposes .completion), validation_utils.py (rewritten as one page-evaluate pass), and element_finder.py (reads the real node sources: attributes dict, ax_node, get_all_children_text()).

Verification

cd workflows && uv run pytest tests/test_browser_use_contract.py -q   # 14 passed

Related: #165 (recording-side findings), #166 (recording pipeline fixes — independent; both PRs apply cleanly in either order aside from trivial context overlap).

🤖 Generated with Claude Code

---

Summary by cubic

Ports the replay engine from Playwright APIs to the browser-use 0.13 CDP actor surface, restoring deterministic replay. Adds a CDP compat layer and contract tests so API breaks fail fast instead of silently at runtime.

• Bug Fixes

  • Replaced Playwright-only calls with CDP equivalents: selector waits, visibility checks, XPath resolution, evaluate return decoding, load-state waits, screenshots, key press, checkbox/radio handling, and select-by-visible-text.
  • Controller now excludes all built-in agent actions and registers deterministic actions explicitly; implements real back/forward navigation (fixes prior no-op success).
  • Element finder reads real EnhancedDOMTreeNode sources (attributes/AX/text, including nested {'attributes': {...}}), uses CDP-safe text/XPath strategies with per-lookup marker tokens, and a page-side text finder that scans rendered innerText and polls until timeout; removed :has-text()/:visible.
  • Validation reworked to collect visible error texts in one pass; prevents false “success” after rejected submits.
  • Error reporting uses get_url/get_title and writes base64 screenshots; XPath click path now decodes evaluate JSON before inspecting; checkbox state handling is centralized via cdp.set_checkbox_state.

• New Features

  • New workflow_use/compat/cdp.py shims: wait_for_element, visibility via geometry+style, XPath resolution with tokenized markers, readyState-based load waits, base64 screenshot-to-file, checkbox state setter, select-by-visible-text, and focused key press.
  • Import-time contract tests assert Page/Element/Browser API surface and statically block Playwright-only calls to guard browser-use upgrades.
  • Dev setup: add pytest for the contract test; remove pytest-asyncio and pytest config. Run the gate with: uv run pytest tests/test_browser_use_contract.py.