fix(sdk): fence lock-lease commits and wait on lock progress, not a fixed deadline

Merged
UN
8d ago
+1662-299

Summary

Two concurrency defects in the SDK lock layer, fixed together because they share the lock acquisition path.

GH-1549 — lost update after lease takeover (pm-gh1549)

  • New acquireLockLease() returns { release, fence }. fence(commit) verifies the lease token under the per-lock stale-cleanup gate (the same gate stale removal and lock publication hold) and keeps it for the whole commit, so a takeover cannot interleave with a fenced commit.
  • A lost lease refuses with lock_lost (exit 4, context.lease + context.current_owner) and writes neither item nor history; the newer value is kept.
  • mutateItem / deleteItem additionally refuse with stale_revision when the stored item bytes changed after the locked read (unlocked edits, extension lock overrides). restore fences its write with the lease.
  • acquireLock is now a thin wrapper, so all other callers keep their exact contract. acquireLockLease / LockLease are public SDK so packages (e.g. pm-csv) can fence their own state.

GH-1516 — queued writers fail at a fixed deadline (pm-iop1pg)

  • locks.wait_ms / PM_LOCK_WAIT_MS is now a no-progress window: every observed owner handoff restarts it, capped at 120 s total. A stalled single owner still refuses after the budget; explicit maxWaitMs ceilings and 0 fail-fast are unchanged.
  • lock_conflict recovery reports waited_ms and owner_handoffs.
  • Per-parent hierarchy lock scope and lock-free appends stay on pm-lvyd.

Evidence

  • tests/unit/core/lock/lock-lease-fencing.spec.ts: takeover → lock_lost with commit never run; released lease; the issue's same-process SDK repro now keeps the newer title and history; stale_revision; handoff-restarted wait; stalled owner; maxWaitMs ceiling.
  • Multi-process conformance: 10 comment appends + 10 parented creates on one plain-directory tracker under a 1 s window lose zero writes; the same test fails on the previous build (2/3 red vs 4/4 green).
  • Regenerated: error catalog, contract snapshot, SDK public surface (acknowledged additive changes), docs (SDK, CONFIGURATION, SDK_STORAGE_LIFECYCLE), CHANGELOG via pm-changelog.

Also files pm-0trehk: the Release Reliability monitor is red since #1548 re-sharded the nightly matrix (new legs at 0 attempts → insufficient_completed_attempts).

Closes #1549

Summary by Sourcery

Harden SDK lock acquisition and commits against lease takeovers while waiting for active lock progress instead of a fixed deadline.

New Features:

  • Expose fenceable SDK lock leases through the public API for protecting long-running commits from lease takeovers.

Bug Fixes:

  • Prevent expired or superseded lock holders from overwriting newer item, history, workspace, or extension-backed writes.
  • Reject item mutations when the stored revision changed or was removed after it was read.
  • Allow configured progress-aware lock users to continue waiting while lock ownership changes hands instead of failing at a fixed deadline.
  • Protect workspace lock renewal and snapshot activation from publishing stale lock state after ownership changes.

Enhancements:

  • Add structured lock-loss and contention diagnostics, including lease ownership, wait duration, and owner handoff counts.
  • Keep the existing acquireLock contract while routing lock acquisition through the new lease implementation.

Documentation:

  • Document lease fencing, stale revision conflicts, progress-aware waiting, and extension-provider responsibilities in SDK and configuration guidance.

Tests:

  • Add unit, integration, and multi-process coverage for lease fencing, stale revisions, lock-gate contention, progress-aware waiting, diagnostics, workspace recovery, and concurrent writer conformance.

Chores:

  • Regenerate error catalogs, contract snapshots, SDK surface metadata, generated documentation, and changelog entries.

Summary by cubic

Fixes two concurrency defects in the SDK lock layer: a lease takeover can no longer overwrite newer commits, and contended writers wait on lock progress instead of a fixed deadline.

Bug Fixes

  • acquireLockLease() returns { release, fence }; fence(commit) verifies the lease token under the stale-cleanup gate for the whole commit. A lost lease refuses with lock_lost (exit 4); fence_gate_unavailable is used when the gate never frees, with retry and pm gc --scope locks --dry-run advice. Neither item nor history is written.
  • mutateItem/deleteItem refuse with stale_revision when item bytes changed after the locked read or the file is removed or unreadable at commit time; restore uses lock_lost or its own changed-while-waiting conflict.
  • item_store_write and non-dry-run item_store_delete run inside the fence, so storage overrides only persist after lease and revision verification.
  • Workspace snapshot activation and heartbeat renewal now run under the lease fence: the lock copy plus root swap refuse with lock_lost when ownership was lost, and renewals run under the stale-cleanup gate so they can never overwrite a taken lock.
  • Extension lock overrides own their lifecycle, so fence() runs the commit unfenced there; packages must fence in their provider or compare-and-write atomically at the storage boundary.
  • locks.wait_ms stays a fixed total budget by default; waitPolicy: "progress" restarts it on each observed owner handoff (120 s cap, 0 fail-fast unchanged), and the item-store lease, restore, and hierarchy lock opt in.
  • lock_lost and lock_conflict diagnostics carry lease, current_owner, item_id, waited_ms, and owner_handoffs through JSON, text, and compact CLI output.
  • acquireLockLease and LockLease are public SDK; acquireLock stays a thin wrapper with the default contract.
  • New tests cover gate contention, never-free gates, overrides reached only under the fence, removed or unreadable item files, snapshots and renewals after takeover, wait budgets, and a conformance run of 10 appends plus 10 parented creates losing zero writes.