feat: compile-check answers a method-body edit in a large assembly in seconds instead of a full recompile

HA
1h ago
+19683-1041

Summary

  • After a method-body edit in a large assembly, uloop compile-check checks only the changed files against the last successful full compile, instead of recompiling the whole assembly. On a real project this took a body-edit check from about 15 seconds to under 2.
  • Running it again on the same edit does not check the same files again: it reports the previous check's result, in about half a second on the same project.
  • An edit that keeps an assembly's API no longer drags its dependents along: when the rebuilt reference assembly is byte-identical to Unity's, or differs in bytes while showing dependents the same API, the dependents are skipped. On the same project, adding a private reference-type field to an assembly that 67 others reference went from 36–45 seconds and 68 compiles to 14–16 seconds and one compile.
  • When the API does change, the dependents no longer wait for each other: each one that has to compile starts as soon as the edited assembly has finished, assuming the dependents it references keep their API, and compiles again only when one of them did not. On the same project, adding a public property to that assembly went from 37–43 seconds to about 28.5 seconds for the same 68 compiles, with no assembly compiled twice.
  • Anything the check cannot vouch for still compiles in full, and the output says why. --full compiles in full for one run.

User Impact

  • Before: a one-line body edit in an assembly of about 3,000 files took 17–22 seconds, because csc recompiled and re-analyzed every file.
  • After: the same edit is checked in under 2 seconds, with the same errors and warnings as a full compile. A repeat run over the same edit, which agents do often, answers in about 0.5 seconds. The output lists which assemblies were checked this way, which rules only a full compile runs, and asks for --full before finishing.
  • Before: in a nullable-enabled assembly, adding a private reference-type field, a private method containing a lambda, or a private iterator changed the bytes of the reference assembly, and every assembly referencing it compiled again. After: those dependents are skipped. A change to the API itself (a public member added, removed, or changed) still compiles the direct dependents, as before.
  • The first run after updating compiles again every assembly it would otherwise have reused from a recorded result, because the record format changed.

What the check guarantees (ADR 0015)

It runs only when the last full compile of the assembly by compile-check succeeded, every input other than the source contents is unchanged, and every changed file keeps its declarations (only method or accessor bodies, non-constant initializers outside partial types, comments, or whitespace changed).

It guarantees:

  • The compiler and analyzer diagnostics of the changed files, with DiagnosticSuppressors and the response file's options (-nowarn, -warnaserror, -ruleset, -define, -nullable) applied.
  • For the unchanged files, the diagnostics of the last full compile.
  • A replayed check reports what checking the same files again would report: it is looked up only after every condition the check needs has been confirmed, and only for the same inputs, baseline, and worker build it was recorded with. FastChecks[].Reused marks it.

It does not guarantee the following, and the output asks for --full before finishing:

  • CS0067, CS0169, CS0414, and CS0649, and the rules of analyzers with Compilation, CompilationEnd, SemanticModel, or SymbolStart actions. FastChecks[].UncheckedCodes lists them.
  • Errors the compiler reports only while emitting, such as CS8178 and CS0656.
  • Source generators. It reads the generator output saved by the last full compile; GeneratorsNotRerun says so.

Fallback reasons

An assembly the check was tried on compiles in full and lists one of these in FastPathFallbacks:

  • no baseline: ..., which says why: the last full compile had errors, it did not record one, no full compile of this assembly is on record (also when an input changed while the last full compile ran), or it was recorded by another version of compile-check
  • baseline compile is no longer on disk
  • inputs other than source contents changed: ..., naming the kinds of input that changed (for example references)
  • inputs other than source contents could not be read
  • generator output of the baseline is gone
  • baseline record does not match its compile result
  • too many changed files (N, limit 30)
  • worker unavailable: ...
  • baseline was fingerprinted by another worker build
  • worker failed: ...
  • DeclarationChanged: ...
  • Unsupported: ... (for example an analyzer that threw)

It is not tried at all with --all or --full, for assemblies with fewer than 300 sources, for assemblies picked only because a reference changed, or without an input key. FastPathBaselineNotes says why a successful full compile left no baseline.

Changes

  • A C# worker, built once per Editor toolchain on the user's machine with the Editor's own dotnet and Roslyn (4.3.1 API only), that fingerprints declarations and checks changed files. NUnit tests pinned to Roslyn 4.3.1 run in CI. (#3315)
  • Go side that builds, caches, and calls the worker. It reads PE headers directly so the macOS ReadyToRun framework assemblies are referenced, and records only real build failures. (#3316)
  • The fast path in compile-check: baselines after successful full compiles, the check with its fallbacks, the output fields FastCheckedAssemblies, FastChecks, FastPathFallbacks, and FastPathBaselineNotes, the --full option, docs, and the compile-check skill. (#3317)
  • Fallback reasons that say what to do about them, and docs on when the first body edit is still compiled in full. (#3320)
  • Planning reads the project's directories once per run instead of three times, which took a no-change run from 1.65 s to 0.88 s and a fast-checked body edit from 2.66 s to 2.02 s on the same project, with identical results. (#3321)
  • Planning reads the directory tree, the .asmdef files, and each assembly's checks several at a time, and reads Unity's build log and the response files while the tree is walked. Which assemblies are selected and which error is reported do not change. On the same project, measured under heavy machine load with the old and new builds alternating, a no-change run went from 1.05 s to 0.43 s and a fast-checked body edit from 2.32 s to 1.69 s, with identical results. (#3322, #3323)
  • Local packages (file: dependencies) are walked in the order of their names in Packages/manifest.json, so when two of them declare the same assembly, or several are broken, every run picks the same one and reports the same error. Before, the order followed Go's random map order. (#3324)
  • Everything compile-check records per assembly (the compile result, the baseline, the saved generator output, and now the last changed-files check) is read and written only through one record store, in its own package with its contract at the boundary. No behavior change. (#3325, #3326, #3327)
  • The record of the last changed-files check, and its replay: when an assembly's input key, its baseline, and the worker build are the ones the previous check saw, the run reports that check without starting the worker, and FastChecks[].Reused says so. The replay stays among the fast-checked assemblies, so the --full reminder and UncheckedCodes still show. (#3328, #3329)
  • A result is recorded only when the input key read again right before recording still matches the one the run started from. csc and the worker read the sources while they run, so an edit made meanwhile would otherwise be recorded under the key of the bytes before it, and putting the file back would replay what the edited file reported. This applies to the changed-files check and to full compiles; the full-compile case dates from the first version of compile-check. (#3329, #3330)
  • An assembly blocked because one it references has errors keeps its outputs and records. When the assembly with errors comes back with the same reference assembly, the first successful run after the error reuses the dependents instead of compiling them again; a reference that comes back changed still compiles them. Assemblies skipped because their references kept Unity's public surface are still cleaned up. (#3331)
  • The reason after a full compile with errors says what brings the changed-files check back, and the skill and docs say that a using change counts as a declaration change and that, with -warnaserror, rules listed in UncheckedCodes can fail the --full run only. (#3332)
  • With ULOOP_DEBUG set, each run writes cli_compile_check_start and cli_compile_check_complete to the CLI VibeLog of the project, with what it compiled, reused, fast-checked, or blocked, why each fallback happened, how long each assembly took, and every reference-assembly comparison under surface_checks. compile-check never talks to the Editor, so nothing else recorded this. (#3333, #3341)
  • The summary says "1 assembly" and "1 error", and the skill says which full compiles drop the baseline (any full compile with errors, and a compile done only because a reference changed) and that a failure of the command prints no payload and writes its error to stderr. (#3334)
  • A dependent of a rebuilt assembly whose reference assembly came out byte-identical to Unity's keeps referencing Unity's copy. Its input key no longer changes with the location of that file, so its recorded result stays reusable. Before, such a dependent compiled again (about 10 s each on the project above) whenever the upstream reference moved between Unity's build and compile-check's output directory. (#3339)
  • When a rebuilt reference assembly differs from Unity's in bytes, the worker's compare-surface command decides whether dependents see the same API. It compares the metadata both reference assemblies expose, reading each nullable annotation by the value that applies to it, leaving out the compiler's embedded attribute types and unreferenced private nested types, and answers different for anything it cannot account for. same skips the dependents; different, a worker that cannot be built, and a failed or timed-out comparison compile them. The answer is recorded in <Assembly>.surface.json, keyed by the content of both files, so the runs until Unity builds again do not compare the same pair twice. (#3340, #3341)
  • Sources are handed to csc in the order of Unity's response file, with files added since the last Unity build appended in ordinal order. Unity 2022.3 writes many response files in an order that is not ordinal, and csc emits types in source order, so the earlier ordinal sort gave an unchanged assembly a reference assembly in other bytes than Unity's, and the comparison read the reordering as a change. The compile result format version is 2, and a baseline tied to a version 1 result is not used, so no output compiled in the old order is reused. (#3342)
  • Each recorded compile result carries the parts of its input key: the digest of the response file and sources without the project-reference lines (Shape), and each project reference with the identity of the file it resolved to (Unity's artifact or this check's output, by content). A record is kept while its shape and the references that have finished still match, and it is consulted before an assembly is started early. Records are format version 3, so the first run after updating compiles again what it would have reused. (#3343)
  • When an assembly's public API changes, the dependents that have to compile no longer wait for each other. A dependent starts as soon as every assembly it references that was selected for its own sources has finished, reading Unity's reference assembly for the dependents still compiling; when one of those turns out to have changed its API, the dependent is compiled again after they finish (at most twice per run), and when one fails it is blocked. A dependent whose recorded result may still apply once they finish waits for them instead, so a run over the same inputs still replays it. assumed_kept, compiled_again_after, discarded_ms, and waited_for_record in the VibeLog say which happened. ADR 0016 records the decision and its limits. (#3344)
  • Among the assemblies ready at the same time, one whose references have all finished starts before one started early, then the one with the most sources, and a tie goes to dependency order; finished assemblies are settled before new ones start, so the ones they release compete for the free jobs. Results and their order are unchanged. (#3345)
  • ADR 0015 records the decision, the guarantees, and when to revisit it.

Verification

  • Measured on a real project, in an assembly of 3,081 source files:
    • Body edit: fast-checked in 2.96 s, against 14.98 s for --full on the same state, with identical errors and warnings. A re-check after the last fixes took 2.59 s.
    • Injected type error: fast-checked in 3.70 s, against 9.87 s for --full. It reported CS0029 at the right line and column, and the diagnostics matched --full.
    • Adding a method fell back with DeclarationChanged and compiled the 18 dependents.
    • Replay, with the previous build alternating on the same output directory over three edits: the second run over an edit took 0.48 s (median) against 1.93 s for the run that checked it, and its JSON matched apart from FastChecks[].Reused, also with an injected type error. Recomputing the key before recording did not show above the noise. After a --full compile, an unchanged run reused its result, so the key check before recording holds on a real project.
  • Dependents, measured on the same project in a nullable-enabled assembly of 1,293 files that 67 assemblies reference directly, with the Editor closed and each edit reverted afterwards:
    • Adding two private reference-type fields compiled one assembly, in 16.3 s and 13.6 s; the comparison took about 0.9 s and answered same API. Before, the same edit compiled 68 assemblies in 45.2 s and 35.7 s.
    • A private method containing a lambda and a private iterator method changed the reference assembly's bytes, answered same API, and the dependents were skipped.
    • As the control, adding a public property answered changed and compiled the 67 dependents, in 42.5 s.
    • With the dependents started optimistically and the largest ready one first, the same public-property edit took 28.4 and 28.6 s for the same 68 compiles (37.4 and 43.4 s before), with no assembly compiled twice and the dependents' share down from 25–32 s to about 18.6 s; the optimistic start alone took 30.4 s. The second run over the same edit replayed all 68 in 1.3 s.
    • The runs between edits, with the file put back, compiled nothing. Before, each of them recompiled one dependent for about 10 s because the upstream reference had moved.
  • In the development project (Unity 2022.3), a body edit in an assembly whose response file is not in ordinal order now produces a reference assembly with Unity's sha256, and its six dependents are skipped; before this branch's last change, the comparison answered changed and they compiled.
  • With the Editor open on the same project and left unfocused, the same public-property edit took 30.8 s with identical results and Unity did not rebuild; the Editor's own CPU use accounts for the 2 s. After uloop compile had rebuilt the edit, compile-check compiled nothing (0.6 s). A run started while Unity was still building did not crash but reported one CS0006 (Metadata file ... .ref.dll could not be found) for reference assemblies Bee was rewriting and exited 1; the run after the build compiled nothing and carried no trace of it. Telling a Unity build in progress apart from a missing reference is a follow-up.
  • Real use: an agent wrote a small game (6 to 10 files) inside a 3,000-file assembly of a project with 561 assemblies, running compile-check after each edit, in three rounds (32, 37, and 36 runs). Across the rounds, --full on the same state never disagreed except for a rule listed in UncheckedCodes, which is the documented gap, and every run took the path the docs describe. The rounds found the issues fixed in #3331 to #3334; the third round found none of that kind. Per round, the changed-files check took 2.1 to 2.4 s (median), a replay 0.5 to 0.9 s, and a full compile of that assembly 11.5 to 16.5 s; the total was about 54 to 62 % of running --full every time. The project's Editor was closed throughout, Unity's response files did not change, and git status matched before and after.
  • The worker builds and runs with the toolchains of Unity 2022.3, 6000.3, and 6000.7 on macOS arm64.
  • Each pull request into the integration branch was verified locally (Go fmt, vet, lint, complexity, tests, coverage, file length, skill size, and the C# worker tests), had an independent review, and mutation checks on the guards that matter.