[fix]: `.count()` not working on xpaths with attribute predicates

Merged
SH
9mo ago
+499-213

Summary

  • Fix locator.count() returning 0 for XPath selectors containing attribute predicates (e.g. //img[@alt='Stagehand'])
  • Consolidate all four copies of the XPath step parser into a single shared module with unit tests

Root Cause

Stagehand has two ways to evaluate XPath expressions:

  1. Native browser XPath (document.evaluate()) — understands the full XPath spec, including attribute filters like [@alt='Stagehand'], but cannot see into shadow DOM
  2. Custom XPath parser — a hand-written DOM traversal that walks the composed tree (including shadow roots), but only understood simple patterns like //div or //div[2], not attribute predicates

The isVisible() code path (via resolveXPathMainWorld) tried native browser XPath first and fell back to the custom parser. But the count() code path (countXPathMatchesMainWorld) went straight to the custom parser. The custom parser's regex only recognized numeric positional indices ([1], [2]), so [@alt='Stagehand'] was treated as part of the tag name, matching nothing.

Fix

1. Extract shared parser (xpathParser.ts)

The XPath step parser was duplicated four times across counts.ts, selectors.ts, piercer.runtime.ts, and waitForSelector.ts. All four are consolidated into a single xpathParser.ts module exporting parseXPathSteps() and elementMatchesStep().

2. Add attribute predicate parsing

The parser now handles [@attr='value'] predicates (single and double quotes, multiple predicates, combined with positional indices). Attribute predicate XPaths work correctly including inside shadow DOM.

3. Fix forward slash splitting in predicates

The step splitter now tracks bracket depth so / inside attribute values (e.g. [@href='/api/endpoint']) no longer incorrectly splits the step. This was a pre-existing bug in three of the four copies; waitForSelector.ts had independently fixed it.

4. Document supported XPath subset

The parser docstring explicitly lists what's supported and what isn't, so future contributors know the scope.

Files changed

  • xpathParser.ts (new) — shared parser with parseXPathSteps() and elementMatchesStep()
  • counts.ts — uses shared parser (removed ~40 lines of inline parser)
  • selectors.ts — uses shared parser (removed ~40 lines of inline parser)
  • piercer.runtime.ts — uses shared parser (removed ~40 lines of inline parser)
  • waitForSelector.ts — uses shared parser (removed ~50 lines of inline parser + matching)
  • global.d.ts — updated docstring
  • xpath-parser.test.ts (new) — 25 vitest unit tests

Test plan

Unit tests (25 vitest tests in xpath-parser.test.ts)

  • Basic tag parsing and case normalization
  • Child vs descendant axes
  • Positional indices (including clamping)
  • Attribute predicates: single quotes, double quotes, multiple predicates, combined with index, hyphenated names, empty values
  • Multi-step paths with mixed predicates
  • Forward slashes inside attribute values ([@href='/api/endpoint'])
  • URL attribute values ([@data-url='http://example.com/path/to/page'])
  • Edge cases: empty input, xpath= prefix, whitespace
  • Element matching: tag name, wildcard, attribute filtering, multiple attributes, tag+attribute combination

Integration tests (verified manually)

  • page.locator("//img[@alt='Stagehand']").count() returns correct count (was 0, now 3) on https://www.stagehand.dev/
  • page.locator("//img[@alt='Stagehand']").isVisible() still returns true
  • Simple XPath //div returns correct count
  • CSS selectors unaffected
  • Non-existent XPath returns 0
  • //div on page with shadow DOM counts all elements (light + shadow = 6)
  • //div[@class='test-item'] on page with shadow DOM counts all elements (light + shadow = 5)

Fixes #1668

🤖 Generated with Claude Code