feat: compile-check no longer makes dependents wait for each other when an assembly's public API changes
Summary
- When an assembly's public API changes, compile-check no longer makes its dependents wait for each other before compiling. A dependent that has to compile starts as soon as the assemblies selected for their own sources have finished, assuming the other dependents it references keep Unity's public surface, and compiles again only if that assumption turns out wrong.
- Builds on #3343, which is merged.
User Impact
Before: on a project with 561 assemblies, adding a public member to an assembly that 67 others reference compiled all 68 in 35 to 43 seconds. The 67 dependents reference each other through 354 references up to ten levels deep, and each waited for its references to finish, although in every measured run only the edited assembly changed its reference assembly. The dependents' critical path, 31 to 37 seconds, was the wall time, and more --jobs did not shorten it.
After: a dependent that has to compile starts as soon as the edited assembly finishes. Simulating the measured unit times puts the critical path at 20 to 25 seconds; the floor is the edited assembly plus the largest dependent. Results are unchanged: every result reported is the compile against the right reference assemblies, after every reference finished.
A second run over the same inputs still replays every assembly instead of compiling it twice.
Changes
- A unit may start before every plan reference has a final state when it has to compile anyway (selected for its own sources, or a dependent one of whose finished references changed), every missing reference is a dependent rather than one selected for its own sources, and it is not blocked. The references still compiling are read from Unity's build (
KeptSurfaceReferences), never from an output an earlier run left. - A unit's final state is decided only once every reference has one:
- every assumption held: the early compile is its result;
- an assumed reference changed its surface: it compiles again against this run's output (at most twice per run);
- an assumed reference failed or was blocked: it is blocked and its result dropped.
The errors and the surface verdict of a dropped result release and block nothing.
- A unit whose record may still be replayed once those references finish (the predicate from #3343) waits instead, so a run over the same inputs replays it.
- Starting and settling alternate until neither moves, because a skip decided while starting units can settle a unit that already finished.
- A skipped dependent now counts as having kept Unity's surface explicitly. Its outputs and records are still removed as before.
- A plan that can never finish (references waiting on each other) is an error naming the units left, instead of an empty success.
- Every final state goes through one place that returns an error if a reference is still unfinished. A unit queued early and then blocked while it waits for a job is only taken out of the queue; the last of its references to finish offers it again, and it is decided then.
UnitResultgainsAssumedReferences,CompiledAgainAfter,DiscardedDurationandWaitedForRecord. The VibeLogunitsentries carry them asassumed_kept,compiled_again_after,discarded_msandwaited_for_record, each only when it applies. A unit replayed after waiting is left out ofunitslike every replayed unit, sowaited_for_recordmarks a wait for a record that then did not apply.- The JSON output is unchanged.
docs/compile-check.mdstep 5 anddocs/vibe-logs.mddescribe this; ADR 0016 records the decision.
Verification
-
22 new tests: 21 scheduler tests in
scheduler_optimistic_test.go, and one VibeLog test.- The first 20 were written before the implementation. On the base (with only the four
UnitResultfields added), 17 of them failed and the three guard tests passed, as they should:- a dependent waits for a reference selected for its own sources;
- a dependent with no changed reference does not start early;
- a blocked dependent does not start early.
- The last two were added after review:
- a blocked unit queued early is decided only once its references finish;
- a unit queued early is queued once when its last reference finishes while it waits for a job.
- Every existing test passes unchanged.
- The first 20 were written before the implementation. On the base (with only the four
-
The tests that need a dependent to start while a reference still compiles no longer rely on a 60 ms hold. The fake compile of that reference finishes only once the dependent has begun, or after 10 seconds, leaving the test's assertions to fail. Tests that check a unit does not start keep the hold, which a slow machine can only make pass.
-
Repeated runs:
go test -race -count=10 -run TestCompileUnits ./internal/compilecheck/passes;- 24 concurrent runs of the race-built test binary with
-test.cpu=1 -test.count=10pass: 240 runs, no failure. With the hold, 2 of 240 failed.
-
Mutations, each restored afterwards:
Mutation Fails Settle a finished unit before all its references are final 13 tests, through the error of the final-state check; with that check removed too, 9 tests, including the discarded verdict and discarded errors tests Never mark a unit stale compiled again (dependent and edited assembly), stale only once more, discarded verdict and errors, the record tests Ignore references selected for their own sources waits for a reference whose own sources changed, and 6 others Start dependents that have no changed reference does not start before a reference is known to have changed; skips the whole chain below an unchanged surface Leave assumed references out of KeptSurfaceReferencesstarts a dependent before the dependents it references finish (it read the stale output), and 2 others Keep the results of blocked units blocks a dependent that started before its reference failed; blocks one whose assumed reference was blocked Drop the error for a plan that cannot finish reports a plan it cannot finish Never wait for a record waits for an assumed reference when its record may still apply Settle only after collecting a completion settles a dependent whose assumed reference was skipped; blocks one whose assumed reference was blocked Decide a blocked unit queued early while a reference still compiles decides a blocked unit queued early only once its references finish (the final-state check reports it) Drop the guard against queuing a unit twice queues a unit once when its last reference finishes while it waits for a job (D compiled twice) Dropping the
blockedcondition from the early start is not a mutation here: the blocked check when a unit is taken from the queue stops the same unit, so nothing observable changes. The guard test pins that this check works on the early path. -
In
cli/dispatcher:gofmt -lis empty;go vet ./...andgolangci-lint run ./...(0 issues) pass;go test -racepasses oninternal/compilecheck/...andinternal/dispatcher/...;- statement coverage from
go test -coverprofile ./...is 94.055%, against 94.066% on the base branch measured the same way. That is 0.011 points lower, within the 0.1-point tolerance. The uncovered error branch of the final-state check cannot be reached while the scheduler works correctly.
-
scripts/check-code-complexity.shandscripts/check-file-length.shpass. -
Repository CI does not run on pull requests into this branch; the checks above were run locally. Wall-time measurements on a large project are not part of this PR.