fix: offer transparent checkbox menu toggles

BT
11d ago
+64-2

Problem

Many sites open menus with a transparent checkbox laid over an icon (the CSS "checkbox hack"). Wikipedia's Vector skin uses it for the ☰ Main menu (#vector-main-menu-dropdown-checkbox, opacity: 0, 32×32 over the icon).

snapshot.js filters with checkVisibility({checkOpacity: true, ...}), so that checkbox is never offered. On the Wikipedia Main Page, the only on-screen route to menu links like About Wikipedia is missing from the action space. The other copy sits in the footer about 4,400 px down.

With the goal Open the page that explains what Wikipedia is (the About Wikipedia page). Do not use search., the agent returned BLOCKED after 9 actions in one run. In another it clicked Talk at p=0.38 and then blocked. The page offered 41 elements, none containing "about".

Change

  • snapshot.js: a transparent checkbox/radio is offered only when it is the elementFromPoint target at its own center. A transparent checkbox covered by other content is still excluded.
  • browser.py: the execution guard accepts the same controls. The existing elementFromPoint occlusion check still runs before input.
  • scripts/check_toggle_menu.py: a local-browser regression in the style of check_guards.py, with no model calls and no external sites. It covers:
    • the toggle is offered
    • a covered transparent checkbox is not offered
    • the toggle click executes
    • the link inside the opened menu is offered and clickable

Verification

  • scripts/check_toggle_menu.py fails on main with Transparent menu toggle should be offered; got ['Wait for the page to update'], and passes on this branch (4/4).
  • scripts/check_guards.py: 21/21 pass.
  • uv run pytest: 31 passed. uv run ruff check . is clean. node --check passes for static/app.js and snapshot.js. uv build succeeds.
  • Live run (headless Chromium 154): the About Wikipedia goal went from blocked to 3/3 done in 2 clicks (Main menu → About Wikipedia, p=1.00). The final URL in all three runs was https://en.wikipedia.org/wiki/Wikipedia:About.

Related observation (not in this PR)

I ran the agent against a dedicated headless Chromium in Docker. There, the owned background=True tab rendered about 2 frames per 500 ms (measured with requestAnimationFrame), and Emulation.setFocusEmulationEnabled did not change that. CSS transitions froze at currentTime: 0, so Wikipedia's menu stayed at opacity: 0 after opening. Target.activateTarget brought it to about 30 frames per 500 ms and fixed it. That only matters for headless/dedicated browsers, where there is no user tab to protect. I kept it out of this PR, but I'm happy to send it as an opt-in if it's useful.

Summary by cubic

Fixes the action space so transparent checkbox menu toggles (like Wikipedia's Vector skin menus) are offered and clickable. Previously the opacity check dropped them, so the only on-screen route to menu links like "About Wikipedia" was missing from the action space and agents ended in BLOCKED.

  • Offers a transparent checkbox or radio only when it is the hit-test target at its own center; checkboxes covered by other content stay excluded.
  • Updates the execution guard to accept the same controls; the existing occlusion check still runs before input.
  • Adds scripts/check_toggle_menu.py, a local-browser regression that fails on main and passes on this branch.