perf(kernel): Kotlin projects index faster

Merged
BO
2d ago
+187-9

Kotlin projects index faster: a full index of ktor takes a median 10.78 s instead of 13.87 s (22% less) and JetBrains Exposed 7.90 s instead of 9.75 s (19% less), with 24% and 18% less CPU and identical graphs (Node 24, 16 vCPUs, three alternating runs per build), at 7% and 6% more peak memory.

Kotlin full index: ktor 10.78 s, was 13.87 s; Exposed 7.90 s, was 9.75 s. Lower is better. Variation criterion: the identical-build floor, at most 4.5% of the baseline here. Workload: codegraph init on ktor and JetBrains Exposed, Node 24, 16 vCPUs; before is fork d161ef74; median of 3 alternating runs per build, identical graphs. ktor wall: 13.9 s to 10.8 s, 22% lower; ktor CPU: 52.6 s to 39.9 s, 24% lower; Exposed wall: 9.75 s to 7.90 s, 19% lower; Exposed CPU: 36.2 s to 29.6 s, 18% lower; ktor peak RSS: 2,573 MiB to 2,756 MiB, 7.1% higher; Exposed peak RSS: 2,494 MiB to 2,633 MiB, 5.6% higher.

What was slow

When a Kotlin member is a candidate link, the resolver asks whether the declaration sits in a companion object. It answered by building the file's full block frames. Building them types every DSL lambda block (build { ... }) by finding the called function's declaration and parsing that file, and for each called name it compiled a fresh regular expression to find the call's receiver. Profiles of the resolver workers put this frame scan at 8.6 s of Exposed's 28.3 s and 13.6 s of ktor's 41.6 s, most of it tree-sitter parses and regex compiles.

The change

  • One shared receiver pattern. The receiver pattern captures the callee and compares it with the call name, instead of compiling one pattern per name (codegraph-kernel/src/resolve/kotlin_calls.rs, scan_kotlin_frames). Same matches: the callee group is the identifier the per-name pattern spelled out.
  • Companion check from declaration heads. Companion frames come from declaration heads (companion object), never from lambda typing, so in_kotlin_companion first reads a head-only frame scan that leaves DSL blocks untyped. It builds the full frames only when an untyped DSL block encloses the declaration and some Kotlin source in the project spells Companion.(: the only text from which a lambda type, directly or through an alias, can name Companion as its receiver. Neither ktor nor Exposed contains it, so neither builds full frames for this check. Other callers of the full frames are unchanged.

__tests__/kotlin-companion-frames.test.ts covers the head-only answer end to end: a companion member reached through a DSL-builder file, with and without a Companion.( lambda receiver elsewhere in the project. It fails when the companion check is forced false and passes on the base and this branch. Two attempts to reach the full-frame fallback through indexing (a local class and a property initializer inside the DSL block) did not reach it even on the base kernel, so a Rust unit test drives it directly: kotlin_calls::tests::companion_check_types_dsl_blocks_when_a_lambda_receiver_names_companion builds a resolver over a one-file project in which companionScope(block: Companion.() -> Unit) and builderScope(block: Builder.() -> Unit) each wrap a local function, and checks that only the first counts as inside a companion. It fails when the fallback answers false, when the Companion.( scan answers false, and when the fallback answers true.

The kernel's unit tests run outside Node, and a test that builds a KernelResolver refers to N-API functions that only Node provides, so the test binary did not link. codegraph-kernel/Cargo.toml now adds napi's dyn-symbols feature as a dev-dependency: test builds look the N-API functions up at run time, and normal builds keep the same napi features (cargo tree -e features --edges normal is unchanged; Cargo.lock is unchanged).

Results

CorpusMetricBefore (fork d161ef74)AfterChangeIdentical-build floorUpstream e8ec7fd8
ktorWall13.87 s (13.63–14.19)10.78 s (10.77–11.26)22% lower (1.29× faster)0.60 s7.80 s (7.76–7.84)
ktorCPU52.61 s (52.47–53.99)39.86 s (39.53–41.09)24% lower (1.32×)1.76 s31.50 s (31.41–31.62)
ktorPeak RSS2,573 MiB (2,564–2,651)2,756 MiB (2,719–2,762)7% higher54 MiB2,710 MiB (2,697–2,712)
ExposedWall9.75 s (9.71–9.87)7.90 s (7.81–7.94)19% lower (1.23× faster)0.39 s5.93 s (5.78–5.94)
ExposedCPU36.24 s (35.69–36.40)29.57 s (29.49–29.85)18% lower (1.23×)1.49 s21.02 s (20.74–21.36)
ExposedPeak RSS2,494 MiB (2,489–2,506)2,633 MiB (2,596–2,646)6% higher50 MiB2,153 MiB (2,149–2,158)

Medians with ranges. Every change in the table is larger than its floor and the before and after ranges do not overlap, the rule declared before the runs. The fork is still slower than upstream on both corpora; upstream's graphs differ from the fork's, so that column is context, not an equal-work comparison.

Workload. codegraph init from scratch on ktor (d08fd038) and JetBrains Exposed (023a6a3a), Node 24.22.0, a 16-vCPU WSL2 host. Three runs per build per corpus in rotated orders, plus three runs of a second copy of this branch's build as the identical-build control; the floor is max − min over those six runs. Wall is elapsed time, CPU is user + system, and peak RSS is GNU time's maximum resident set size (the largest single process). One attempt was rejected for other processes using more than two cores (ktor, before build, round 3) and re-run. The graph hash (nodes, edges, unresolved-reference states) of every after run equals every before run's: ktor 57e9c5136b5e, Exposed b332758c901b.

Where the time went

CPU sampling of the resolver workers, one run per build per corpus (perf record -e cpu-clock:u -F 499 -g, frame-pointer builds of the same sources, run under the same host lock right after the timing runs; attribution only, not timing samples). Times are inclusive CPU seconds summed over the six workers.

CorpusBuildWorker CPUBlock-frame scanof it, tree-sitter parsesof it, regex compilesAll regex compiles
ktorbefore41.6 s13.58 s5.08 s5.00 s8.11 s
ktorafter29.2 s2.88 s1.83 s02.48 s
Exposedbefore28.3 s8.59 s3.64 s3.04 s4.58 s
Exposedafter20.9 s2.55 s1.88 s01.35 s

Regex compiles elsewhere fall too (ktor 3.1 s to 2.5 s outside the frame scan): the per-name patterns no longer push other patterns out of the thread-local regex cache. The frame scan that remains serves the other callers of the full frames. The largest remaining Kotlin parse cost is kotlin_infix_site (2.9 s on ktor, 2.0 s on Exposed, unchanged).

Checks

On head 1ba0f684 (this branch merged with fork/consolidated at 0e6f7e30, which includes #517 and #518; the kernel was rebuilt on the merged tree):

  • npm run build:kernel (cargo clippy --lib with -D warnings, release build): exit 0.
  • cargo test --release in codegraph-kernel: 81 passed, 1 ignored, exit 0, including the new fallback test.
  • The new test under three mutations of in_kotlin_companion and the Companion.( scan (fallback forced false, scan forced false, fallback forced true): each fails it; restored, it passes.
  • UPDATE_GOLDEN=1 npx vitest run __tests__/kernel-golden-dumps.test.ts: 8 passed, no .dump diff.
  • npx vitest run __tests__/kotlin-companion-frames.test.ts __tests__/readme-sync.test.ts: 8 passed.
  • npx tsc --noEmit: exit 0. npm run check:agent-docs: exit 0.
  • Full suite npx vitest run: 747 files passed, 9,265 tests passed, 40 skipped, exit 0.
  • Earlier, on 7ef01159: npm run eval:precision on exposed and javalin, base and branch: exposed absent 2/2 held, present 6/6 held, 75,104 edges on both; javalin absent 1/1, present 3/3, 16,137 edges on both. Not re-run after the merge.

README rows checked

"About this fork": the name-resolution row and the Kotlin feature bullets (unchanged: no behavior change), and the 2026-10-15 mid-size comparison paragraph, which quotes ktor and Exposed. That paragraph now has one sentence saying this change cut a full index of ktor by 22% and of Exposed by 19% against the fork at d161ef74 just before it, that the figures are in the changelog and this PR, and that both still take longer than upstream e8ec7fd8 did in the same runs. An earlier wording named d161ef74 as if it were the measured build. The Measured results tables do not include a Kotlin corpus. No docs-site page quotes these figures.

Not covered

  • Peak memory rises 6–7% on both corpora and the cause is not yet explained; the change adds only a per-file memo of head frames and one boolean. These figures were measured before this branch merged #517 (one regex cache shared by the resolver workers), which lowers Kotlin indexing memory; the merged result was not re-measured.
  • kotlin_infix_site still parses whole files to classify infix calls (about 2.9 s of ktor's worker CPU), and the receiver-typing helpers still compile per-name patterns; those helpers are shared with Java and C# and were left for a separate change.
  • The six resolver workers each keep their own frame memos; sharing them across workers needs a process-wide cache with invalidation and was not attempted.

Summary by CodeRabbit

  • Performance
    • Kotlin indexing is faster for projects using companion-object calls inside DSL lambdas. Full-index times fell from 13.87 to 10.78 seconds for Ktor and from 9.75 to 7.90 seconds for Exposed, with identical results. In these comparisons, indexing remained slower than upstream.
  • Bug Fixes
    • Improved resolution of imported companion-object functions in Kotlin DSL lambdas.