fix(tokenizer): drop unfinished tags after a closing tag name or slash at EOF
When the input ends inside a tag after a closing tag name (</b ) or after the / of an opening tag (<b /), the tokenizer falls through to the default branch of handleTrailingData and calls ontext(-1, end), because sectionStart was already reset to -1. The parser then emits a slice of whatever is left in its buffer, so the text depends on how the input was split into chunks:
import { Parser } from "htmlparser2";
const run = (...chunks) => { const out = []; const p = new Parser({ ontext: (t) => out.push(t) }); chunks.forEach((c) => p.write(c)); p.end(); return out; };
run("x</b "); // [ 'x', ' ' ]
run("x", "</b "); // [ 'x', '</b ' ]
run("<p>x</p "); // [ 'x', ' ' ] and the text has startIndex 8, endIndex 7
Expected is ['x'] in every case, the same as for x</b and x<b today: an unfinished tag at EOF is dropped.
The fix adds InSelfClosingTag and AfterClosingTagName to the list of in-tag states that handleTrailingData ignores.
The new tokenizer test fails on master with:
- Expected
+ Received
+ [
+ "ontext",
+ -1,
+ 5,
+ ],
With the fix, npm run test:vi (216 tests) and npm run lint pass. I also ran 200k random inputs through the parser once in one piece and once split at random points; before the fix the event streams differed on these EOF cases, after it there were no differences.
Summary by CodeRabbit
- Bug Fixes
- Incomplete self-closing tags and tags with an unfinished closing-tag name are ignored during tokenization instead of triggering partial tag callbacks. This behavior is consistent across different points where streamed input may be split, so incomplete markup does not produce a partial tag event.