feat(cli): restore cursor overlay through page.evaluate

Merged
SH
2mo ago
+390-14

Summary

Restore Browse's visible cursor as a CLI-owned DOM overlay, without adding a cursor API to core Stagehand V4.

  • Keep the overlay implementation in one dedicated cursor-overlay.ts file.
  • Install it idempotently for the current document through page.evaluate(CURSOR_OVERLAY_SCRIPT) and for future navigations through page.addInitScript(...).
  • Retry installation on DOMContentLoaded when the init script runs before the document root exists.
  • Keep injection in the top frame and update the marker from coordinate input, including when input lands inside a child frame.
  • Treat visual position updates as best-effort so they cannot block or invalidate real mouse input.
  • Preserve the V3 browse cursor JSON response: { "cursor": "enabled" }.

Stack (#2872)

  1. #2833 — exact Browse V3 baseline import
  2. #2834 — Stagehand V4 runtime and standard command parity
  3. #2869 — CLI-owned cursor overlay
  4. #2849 — CLI-private CDP sidecar; V3 network parity
  5. #2835 — remove --return-xpath; supported V3 parity/release checkpoint
  6. #2838 — eval and packaging integration
  7. #2839 — managed Context names (fast-follow)
  8. #2701 — shared Functions core consumer (fast-follow)

Why this is separate

The cursor is a self-contained optional visual feature with different review concerns from the combined V4 runtime/command migration: injected DOM/CSS, idempotency, event handling, and screenshot behavior. Keeping it additive on #2834 lets this feature be reviewed or reverted without disturbing browser lifecycle or commands.

E2E Test Matrix

Review-feedback verification compared the exact prior head 6a9d6aa09 with fixed implementation head 1035fbbf5 through the built CLI and real Browserbase browsers. Final head 68f6fcb2c only expands automated coverage and does not change runtime code. Targets were the public example.com and example.org pages.

Command / flowObserved outputConfidence / sufficiency
Prior head: enable cursor, alternate 20 cross-origin navigations, inspect #__browse_cursor_overlay__ before any mouse inputOverlay count was 0 after 20/20 navigationsReproduces the DOM-readiness bug raised in review
Fixed head: repeat the same 20-navigation flowOverlay count was 1 after 20/20 navigations (0 misses)Proves the DOMContentLoaded retry restores the overlay after navigation in the real browser path
Prior head: replace the page's cursor-position callback with a throwing function, then run browse mouse click 200 200 against an oversized synthetic buttonCommand exited 1; the page's click state remained nullReproduces the visual-update failure blocking real mouse input
Fixed head: repeat the same forced overlay failure and clickCLI returned { "clicked": true }; page click state became "yes"Proves overlay rendering is best-effort while real input still executes
Built CLI: browse cursor{ "cursor": "enabled" } (prior head returned { "enabled": true })Confirms V3-compatible output for existing scripts
pnpm --filter browse lintPassed formatting, ESLint, and TypeScript checksStatic validation on the final head
pnpm --filter browse test:cli26 files / 393 tests passedFull Browse suite, including DOM readiness, safe styling, idempotency, top-frame isolation, cursor positioning/clamping, and all four coordinate input commands
browse stop after each live runCompleted successfullyCovers Browserbase session and daemon cleanup

The already-uploaded screenshot below remains representative visual proof of the same overlay behavior.

Browse V4 CLI cursor overlay surviving navigation and pointing inside an iframe

No LLM path or customer data was involved.