perf(kernel): stop Python local-binding checks from rescanning files per candidate

Merged
BO
16d ago
+157-21

Indexing CPython on the fork took 48.8 s, up from 26.5 s at 48903f51 on the same host. The resolve loop's wait on pool workers grew from 1.7 s to 18.7 s. A native profile put most of that growth in python_locally_bound, which arrived with the upstream merge through #2248 (1f209fe3). To decide whether a Python name is bound locally, it compiled four regexes for the name and scanned the enclosing function and then the whole file. It did that again for every same-named definition in another file.

Change

  • python_locally_bound memoizes its answer by (file, line, name), and the file-wide scan by (file, name). Both memos live for one resolution run, like the resolver's other memos.
  • The four binding patterns moved into a PyBinding enum. Their regex text is unchanged. Each pattern has a regex-free filter (may_match) that admits a line only when the name is followed by a character the pattern needs, so the regex is compiled and run on far fewer lines. A memchr finder skips lines without the name.
  • A kernel test, may_match_admits_every_binding_match, runs every pattern over 27 line shapes and three names (including a non-ASCII one). It checks that each line a pattern matches also passes that pattern's filter. Removing the filter's ) skip for assignments makes it fail.

Results

Same host, Node, two interleaved runs each, daily build c62f88ee against this branch:

CorpusBeforeAfter
CPython49.5 s / 48.1 s, 6.45 GiB peak38.8 s / 40.2 s, 5.90 GiB peak
pretix10.2 s / 9.8 s9.6 s / 10.5 s

Both graphs are identical row for row: all node columns except updated_at, and edges compared as a multiset on source, target, kind, metadata, line, column and provenance. CPython has 167,884 nodes and 599,991 edges; pretix has 32,338 and 77,554. The fix recovers 9.3 s of the 22.3 s slowdown. The rest is spread across newer resolution work, with no single resolver step above 5% of the profile.

Checks

  • Full suite on the fix commit: 575 files, 7,342 passed, 39 skipped.
  • check:kernel-rules passed. __tests__/readme-sync.test.ts passed after the doc commits.
  • Kernel golden dumps were not re-baselined. The suite's golden test passed unchanged, and the graph comparison above shows no output change.
  • eval:precision was not run, because the change leaves the graph byte-identical on both Python corpora.
  • An independent review found no correctness problem in the filter. It flagged two doc claims and the changelog format, which were fixed.

README rows checked

  • Measured results: the index table (pretix and CPython rows) predates this fix. A note below the table gives the new CPython figures and says the table has not been re-run.
  • The "3–5% fewer edges on pretix and CPython" line is unchanged, since the graph is unchanged.
  • The name-resolution row of "What the fork adds" is unaffected.