[6/6] perf(control/pg): promote dependents and job waiters over their own rows
Stacked on #484, merge in order. Stack: 6/6, splitting #479. This is the tip of the stack: its tree is identical to #479's head (git diff --stat between them is empty).
Summary
Completions promote the dependents they unblocked, and the concurrency waiter a completion releases, over those jobs' own rows instead of loading the whole run graph (PgBackend::load_graph + Sweep::sweep + flush). Each path runs inside the same transaction and run lock as before.
What changed
- Dependents: each dependent
settle_job_targetedleftblockedwith no needs outstanding is decided on its own node plus a job-scopedNeedsView(ancestors followed through matrix base ids), with the samedependency_decisionsemantics. Skip/error settle the dependent throughsettle_unrunand feed the dependents that frees into the next pass of a work queue, so a skip cascade finishes in the completion that started it, including a job that fails at promotion (unservableruns-on, denied environment, failed gate), whose dependents previously waited for a later sweep. A reusable caller takes its JobSet gates (shared with the sweep); a deferred matrix goes to the expansion queue; everything else enterspromote_one(#5). Newcontrol/pg/dependents.rs;PgBackend::promote_unblockedis deleted.Sweepstays for submit, rerun, cancel, expansion and the reaper. - Job-level concurrency waiters: a waiter is promoted from its own rows (
dispatch.rs,rerun.rs), not the run graph. - Tests: the shared suite pins dependent promotion on both backends (dependents of a matrix base get merged outputs and wait for every leg, a failed leg skips dependents transitively,
if: always()after a failure,failure()reading an ancestor reached only through a matrix base, a reusable caller that depends on another job). A scenario for a job that fails at promotion skipping the jobs behind it in the same completion fails on the sweep-based promotion (that commit is red by design until the "promote the dependents a completion unblocked over their own rows" commit that follows it in this PR) and passes after. PG tests for waiter promotion from rows. - Docs:
internal/settle-redesign.mdand CHANGELOG describe completions as graph-free (NeedsView, the dependents work queue, skip cascade), correct the ancestor query to follow matched rows through matrix base ids, and drop the graph-backed caveat.
Measurements
Lab harness on a laptop (benchmarks/control-load profiles, 2 nodes, Postgres 18), 200-leg workloads. Changed arms are 3 clean rounds; baselines are 1 clean round (main and #378 + review fixes). Treat these as direction, not a capacity claim. The "This stack" arm is the full #479 head, i.e. this tip.
| Workload | Metric | #378 + review fixes | This stack |
|---|---|---|---|
200 legs in one job-level concurrency: group | graph loads / completion | 0.995 | 0 |
| settle lock hold, mean | 23.9 ms (main: 28.3) | 14.8 ms [12.8-16.0] | |
| 200 legs + always-run merge | merge-promoting completion | ~35 ms of graph work | ~2-6 ms of row work |
(The capped max-parallel: 20 numbers are in #5.) On the plain 200-leg workload, leg completions run the same statements as before. Mean hold over 3 rounds was 16.5 ms [14.6-19.4] vs 14.8 ms [13.4-16.2] for the baseline; ranges overlap and the difference tracks run-row lock contention, not added statements. Claims still load the graph on the runner claim path.
Not included
- Runner claim refactor, the
remaining_needsdecrement counter (it slowed completions under overlapping runs and is held back), and the post-completion reply read. - Workflow-level concurrency resume (
resume_held_run), submit, rerun, and expansion (claim_expansion/apply_expansion) still load the graph. - The lock-scope redesign: a lock-order audit found it needs a larger design and exposes 12 pre-existing ordering problems; not attempted here.
Test plan
On this PR's tip, against a disposable Postgres 17 (max_connections=2000, PRELOOP_PG_WRITERS=3 PRELOOP_PG_READERS=3, PRELOOP_TEST_POSTGRES_URL set):
cargo fmt --all --check: cleancargo clippy --workspace --all-targets -- -D warnings: cleancargo test --locked -p preloop-runner-server --lib -- control::: 367 passed, 0 failed, 0skipping:lines (same count as #479's head)just test-ci(fmt-check, clippy, zizmor,PROPTEST_CASES=8 cargo test --locked --workspace, control migrations,just conform): passed; the workspace run was 2811 passed, 0 failed, 3 ignored over 50 test binaries, 0skipping:lines,conform: PASS.git diff --stat split/6 pr/graph-free-scheduler: empty (identical trees,65a67a12fb90f7b2b8ab85091738b472d093bcb2).
The local engine CI run (ci.yml) was not run: the laptop engine's runner pool does not advertise the preloop-cpane label it requires, so jobs fail at enqueue. The checks on this PR are the first run of ci.yml for these commits; the earlier PRs in the stack (#480-#484) were verified with the control suite, fmt and clippy only, not just test-ci.
Stack
- #480
fix(pg): max-parallel stall, cohort waiter retry, nested-caller fold, non-terminal guard - #481
fix(pg): deferred reusable callers - #482
refactor(pg): remove the in-memory completion path - #483
refactor(pg): row-based enqueue target, needs view, settle hand-off (no behaviour change) - #484
perf(control/pg): max-parallel admission over rows - #485
perf(control/pg): dependents and job waiters over rows
Each PR's base is the previous PR's branch, so each shows only its own diff. Supersedes #479 (closed).