feat: compile-check starts the largest ready assembly first so long compiles do not finish last

Merged
HA
7h ago
+109-5

Summary

  • When more assemblies are ready to compile than jobs are free, compile-check now starts the one with the most sources first, so the longest compiles no longer start last while the other jobs sit idle.
  • Builds on #3344, which is merged.

User Impact

Before: assemblies ready at the same time started in the order they became ready. A large dependent could start last and keep the run going alone after every other job had finished.

After: among the ready assemblies, one whose references have all finished starts before one started early (before its references finished). Within each group the one with the most sources goes first, and a tie goes to the one first in dependency order. On a project with 561 assemblies, how long an assembly takes to compile follows its source count closely (Spearman 0.72 to 0.81). Results and their reported order are unchanged.

Changes

  • The ready queue gives out units in this order:
    • a unit whose references are all final before one queued early. Otherwise a large unit queued early could overtake one of its own references still waiting in the queue, compile against Unity's file for it, and compile again once that reference changed its surface;
    • then the unit with the most sources;
    • then the unit first in the plan. Plan order, rather than the order units became ready, keeps the start order the same from run to run.
  • The scheduler now decides the final state of finished units before it starts new ones. The units a finished one releases then compete for the free jobs with those already waiting, instead of queuing behind whatever started first.
  • docs/compile-check.md step 5 says how ready assemblies are ordered.
  • The scheduler tests' fake compiler records the order its invocations start in.

Verification

  • Three new tests, each written first and failing before its change:

    • the ready unit with the most sources starts first, and results stay in plan order;
    • a tie in source count goes to plan order, even when the unit first in the plan became ready later. This test first passed on the old queue, because its order happened to equal plan order, so it was rebuilt so that the order units become ready differs from plan order;
    • a unit whose references are all final starts before a larger unit queued early that references it, which compiles once (added after review; before the fix it compiled twice, start order X D A D B).
  • Mutations, each restored afterwards:

    MutationFails
    Drop the final-references-first rulethe queued-early test
    Start units queued early firstthe queued-early test; queues a unit once when its last reference finishes while it waits for a job
    Ignore the source countall three new tests
    Start new units before settling finished onesthe tie test; the queued-early test
  • In cli/dispatcher:

    • gofmt -l is empty;
    • go vet ./... and golangci-lint run ./... (0 issues) pass;
    • go test -race passes on internal/compilecheck/... and internal/dispatcher/...;
    • go test -race -count=10 -run TestCompileUnits ./internal/compilecheck/ passes.
  • scripts/check-code-complexity.sh and scripts/check-file-length.sh pass.