fix: compile-check reads identifiers written with unicode escapes

Merged
HA
4d ago
+185-16

Summary

  • uloop compile-check now reads identifiers written with \u / \U escapes, so a dependent that writes a changed name only that way is compiled instead of left out by name narrowing.

User Impact

  • Before: name narrowing cut identifiers from the source text without decoding escapes, so \u0046oo() did not count as writing Foo; a change to Foo left that dependent out and its compile error was missed.
  • After: a well-formed escape counts as the character it spells, as it does for the compiler.

Changes

  • New identifier_escape.go: decodeUnicodeEscape accepts \uXXXX and \UXXXXXXXX only when every digit is hexadecimal and the value (gathered unsigned, so eight digits cannot overflow into a negative value) is neither a surrogate nor past the last code point - the same characters the compiler refuses in an identifier. runeAt answers ASCII first, since the scan asks for every byte.
  • scanIdentifiers reads through runeAt and decodes an identifier's text only when it holds an escape, then removes formatting characters as before. Identifiers without an escape are still taken from the text as it is, without a copy.
  • Escapes in string literals and comments are decoded too. The spelling of an escape itself (a name like u0041) is no longer read, but no name the compiler binds is lost; the existing scan test was updated from u0041 to A for that reason.
  • docs/compile-check.md and ADR 0017 drop the gap (ADR 0017 gets an amendment line).

Verification

  • cli/dispatcher: gofmt -l . empty, go vet ./..., golangci-lint run ./... 0 issues, go test ./... -count=1 pass.
  • New tests: TestScanIdentifiersDecodesUnicodeEscapes, TestScanIdentifiersLeavesAMalformedEscapeAlone, TestDecodeUnicodeEscapeRejectsASurrogate, TestDecodeUnicodeEscapeRejectsATruncatedOrOutOfRangeEscape, TestCompileUnitsCompilesADependentThatWritesTheChangedNameWithAnEscape. The tests build escapes with a helper (escapeOf) so the source never holds a literal backslash-u sequence.
  • BenchmarkScanIdentifiers: 12.6 ms before, 13.3 ms after.
  • Mutations (each reverted):
    • decodeUnicodeEscape always rejects → the scan, decode and scheduler escape tests fail.
    • accept a non-hexadecimal digit → ...LeavesAMalformedEscapeAlone fails.
    • accept a surrogate → ...RejectsASurrogate fails.
    • accept a truncated escape at the end of the file → ...RejectsATruncatedOrOutOfRangeEscape fails.
    • accept a value past the last code point → ...RejectsATruncatedOrOutOfRangeEscape fails.
    • gather the value in a rune again → ...RejectsATruncatedOrOutOfRangeEscape fails (\U80000000, \UFFFFFFFF).
    • reject everything from U+D800 up (drop the upper surrogate bound) → ...RejectsASurrogate (\uE000 accepted) and ...DecodesUnicodeEscapes (\uFF21oo) fail.
    • drop the lone-backslash guard → TestScanIdentifiersReadsAFileEndingInABackslash fails with an index-out-of-range panic.