Prove the futex words behind Linux's refusals
Follows #348, which added the published waiter census the shim's wake fast
path reads, and #315.
The bug
Linux answers EINVAL when a requeue without FUTEX_CMP_REQUEUE_PI meets a
waiter holding an rt_waiter or a pi_state. futex_requeue() in
kernel/futex/requeue.c runs that test from inside its walk, before it
decides whether the waiter in hand is woken or moved, and without consulting
either address.
elfuse narrowed it three ways:
| Linux | elfuse | |
|---|---|---|
| address condition | none | required uaddr != uaddr2 |
PI waiter inside nr_wake | refused | skipped, then woken |
nr_requeue == 0 | still walks and checks | did not look at all |
Each is reachable from unprivileged guest code with a thread parked in
FUTEX_LOCK_PI(X): futex(X, FUTEX_REQUEUE, 0, 1, X),
futex(X, FUTEX_REQUEUE, 1, 1, Y), and futex(X, FUTEX_REQUEUE, 1, 0, Y).
Why the first one is more than an errno
That rejection is also what keeps the census charge move away from a PI
waiter, whose bucket charge futex_lock_pi's wrapper owns for the whole call.
Letting one through runs the move, which credits the destination but debits
w->pub_bucket only under pub_follows. The bucket is then charged for a
waiter that has gone, futex_wake_fast reads it as occupied and declines from
then on, and repeating the call wraps the uint32, at which point a zero
reading is a wake answered 0 at EL1 while a waiter is parked.
The fix
Walk every waiter the call could act on, bounded by
wake_count + requeue_count the way Linux's own
task_count - nr_wake >= nr_requeue break bounds it. The sum is taken in 64
bits because both halves are guest-supplied and wake-all passes INT_MAX.
Both halves of the charge move go under pub_follows as well. It is
unreachable for a PI waiter now, but a credit sitting outside the branch that
carries its debit is what let the two come apart, and pairing them keeps a
later narrowing of the guard from reintroducing it.
The test
tests/test-futex-requeue-pi.c parks a PI waiter in FUTEX_LOCK_PI and puts
one case to each narrowing, retrying the first while the call reports zero
because that is its answer before the waiter parks. Every assertion is an
errno, so the matrix can put the same source to a reference kernel; registered
in run_unit_tests and only there, since the manifest's scope note keeps a
cross-checkable test out of that file.
The census gets no assertion. An over-count is invisible from the guest (the
host answers every wake correctly either way), and the restored rejection
makes it unreachable.
Scope
Only the first narrowing was in the report this PR started from. The other two
were found while checking that one against futex_requeue(), and they are the
same if, three lines apart, answered by the same paragraph of kernel source.
Fixing one and leaving two would have made the claim in this PR's title untrue.
Verification
tests/driver.sh passes 97 of 98, and make check-format, check-ascii,
check-syscall-coverage, check-eintr-contract, check-lock-order,
check-atomics and check-skill-refs are clean. The new test fails before the
fix and passes after it. Every existing futex lane still passes, including
test-futex-pi, test-futex-requeue-account, test-futex-waitv,
test-shim-futex-fast and test-shim-futex-toctou.
test-hello and the test-matrix lanes want aarch64-none-elf-as, whose
Homebrew cask needs a sudo installer this machine cannot run, so those are
unbuilt rather than passing. For the same reason the EXPECTED_BASELINES
floors are untouched: a raised floor would be a number nobody observed, and the
comment above them already says a floor too low costs nothing.
Across two full driver runs test-clone3 failed once, on stack reuse mmap returned -1, and passed on the rerun and on three isolated runs. It is in
QEMU_SKIP already, and its failure mode is address-space placement, disjoint
from anything here.