fix: queue cancellable send admission with plain lockers
Closes #176.
The cancellable locks in context_lock.go retried outside the sync.Mutex and sync.RWMutex wait queues. Once plain callers had put a mutex into starvation mode, a WriteDataContext or MTPTransferContext caller could be kept out until its deadline, although the lock was released many times in between. Five locks had this shape: the association's send lock and unscoped delivery lock, the Application Server's delivery lock, the transfer lock (an RWMutex taken by MTPTransferContext against routing and state writers) and the per-flow transfer lock.
All five now use github.com/gomaja/go-ctxsync, whose Mutex and RWMutex queue cancellable and plain callers together and hand off in starvation mode. context_lock.go and its tests are deleted.
Unchanged:
- A context that ends before admission reports
DataNotSentwith a cause matchingctx.Err(). - A context with no Done channel keeps the ordinary submission path.
CloseandAbortstill bypass the send lock, so they can release a blocked transport write.- DATA submission outcomes and their error mapping.
go-ctxsync is required at 23c9cb6, the first revision with the fix for gomaja/go-ctxsync#1, where cancelled waiters could take tens of seconds to return under the race detector with many Ps. go-ctxsync was already in the module graph through go-sctp; it is now a direct requirement. There is no API change.
Tests
TestWriteDataContextProgressUnderPlainSendTrafficandTestMTPTransferContextProgressUnderPlainMutationTrafficreproduce #176: sixteen plain callers on one P with 2 ms critical sections, and one cancellable call with a 250 ms deadline. With the previous locks, which main still has unchanged, both failed 50 of 50 runs with and without-race; with go-ctxsync both passed 50 of 50.TestWriteDataContextWaitersDoNotLeak(128 cancelled writers queued on a held send lock) passed 20 of 20 fresh processes under-racewith GOMAXPROCS=32, about 1 s each. With go-ctxsync de26fb4 it timed out after 30 s in 3 of 3.
Benchmarks
A 20-pair comparison against main (fresh processes, alternating order, CPUs 8-11, GOMAXPROCS=4) could not be decided on the shared machine it ran on: the baseline spread was 25% to 284% against a 10% limit. The paired ratio estimates were between 0.94 and 1.04, and allocations and bytes per operation were unchanged.
| Benchmark | Main median (ns/op) | Branch median (ns/op) | Paired ratio | 95% interval |
|---|---|---|---|---|
BenchmarkWriteDataContext/payload-128 | 131.3 | 127.8 | 0.987 | 0.913–1.067 |
BenchmarkWriteDataContext/payload-4096 | 983.5 | 954.5 | 0.995 | 0.844–1.173 |
BenchmarkMTPTransferContext/payload-128 | 1418.0 | 1461.0 | 1.032 | 0.879–1.212 |
BenchmarkMTPTransferContext/payload-4096 | 2544.5 | 2697.5 | 1.011 | 0.863–1.185 |
BenchmarkWriteData/payload-128 | 130.6 | 131.6 | 1.029 | 0.917–1.155 |
BenchmarkWriteData/payload-4096 | 969.5 | 964.4 | 1.040 | 0.948–1.141 |
BenchmarkMTPTransfer/payload-128 | 1473.5 | 1422.0 | 0.989 | 0.936–1.046 |
BenchmarkMTPTransfer/payload-4096 | 2621.0 | 2577.0 | 0.943 | 0.849–1.047 |
Local validation
The 21 context-send tests passed 210 runs with -count=10 under -race and GOMAXPROCS=32 in Docker on 35ab41b6178e. The full local suite passed on the same head (54 steps, including the Linux SCTP race suite in Docker and the cross-compiles); non-zero steps: none.
Summary by cubic
Fixes #176: cancellable senders could be starved by plain locker traffic. The custom lock in context_lock.go retried outside the sync.Mutex and sync.RWMutex wait queues, so once plain callers put a mutex into starvation mode, a WriteDataContext or MTPTransferContext caller could be kept out until its deadline even though the lock was released many times between. All five barriers (association send and unscoped delivery locks, Application Server delivery lock, ASP transfer RWMutex, per-flow transfer lock) now use github.com/gomaja/go-ctxsync, which queues cancellable and plain callers together and hands off in starvation mode. context_lock.go and its tests are deleted.
Unchanged behavior: a context that ends before admission reports DataNotSent with a cause matching ctx.Err(); a context with no Done channel keeps the ordinary submission path; Close and Abort still bypass the send lock; DATA submission outcomes and error mapping are preserved. Reproduction tests (sixteen plain callers on one P with 2 ms critical sections, cancellable call with a 250 ms deadline) failed 50 of 50 runs on the previous locks and pass 50 of 50 on the new ones.
Dependencies
go-ctxsyncis now a direct requirement pinned at 23c9cb6, the first revision fixing gomaja/go-ctxsync#1, where cancelled waiters could take tens of seconds to return under the race detector with many Ps; there is no API change.