Add scoped Context source withdrawal function

SI
5d ago
+494-0

Summary

Adds public.withdraw_context_request_source(p_actor, p_owner, p_request, p_source default null, p_source_version default null), the first lifecycle mutation reachable from the shared Context operation, plus a fence that keeps a withdrawn source withdrawn.

  • Scoped withdrawal. Rechecks membership through the existing authorize_context_case_actor, locks the request, resolves the source (directly, or from one of its version IDs, which brief input manifests expose as sourceVersionIds), requires it to be owned by the workspace and recorded as an input of that request through saved result lineage, then reuses the existing withdraw_context_source primitive. Exactly one identifier is required (22023 otherwise).
  • Late writers cannot republish. Every writer re-creates sources with insert … on conflict (owner_id, kind, source_url) where withdrawn_at is null, which ignores withdrawn rows, so before this change a slower or later job (e.g. save_context_metadata with a capture time before the withdrawal, or complete_context_enrichment) recreated the URL as a new active source and accepted it again. A BEFORE INSERT trigger on context_sources now first waits on the active row (FOR KEY SHARE, which blocks behind the withdrawal's FOR UPDATE) and then makes any row whose owner, kind and URL were already withdrawn born withdrawn. Its evidence can never be accepted or read, and the existing writers fail closed (Context acceptance conflict / Source no longer acceptable) instead of publishing. Other workspaces and other source kinds are unaffected. A small partial index (context_withdrawn_source_url) backs the lookup. There is no restore operation in this slice.
  • Receipt (context-source-withdrawal-v1). request_id, resolved source_id, withdrawn_at, already_withdrawn, and affected = the source's recorded lineage: results that depend on it, documents holding such a result, and every request whose attempts produced them. It is identical on replay, even after later withdrawals. Replay does not touch timestamps or revisions.

What changes and what stays: rows are kept, but the reused primitive relabels every result on the source as withdrawn, replacing its earlier status (e.g. stale, partial). Documents whose current result used the source lose it (revision bumped). read_context_documents, accept_context_result and read_context_brief withhold the evidence, and dependent saved briefs read back state: unavailable. No original bytes are deleted, nothing is recomputed, and no provider is called. Lock order: membership → request → source → results/documents. A writer that already holds a document lock (e.g. complete_context_enrichment) takes the opposite order, so a concurrent writer on a shared document can deadlock with a withdrawal. PostgreSQL then aborts one of them, and the API returns a retryable 409. Service-role only. No existing table, grant or function changes; the trigger and index are additive.

Refs recoupable/app#2124 (child of epic recoupable/app#2116).

Merge sequencing

docs → database → api → app. Sibling PRs on branch claude/ctx-2124-source-withdrawal: recoupable/docs#364 (OpenAPI contract) and recoupable/api#1008, which calls this function and must deploy after this migration. It does not depend on any open PR: version IDs are already exposed on main through brief input manifests. It complements database#92 (read-only evidence manifest) and does not touch the original-receipt/delivery functions of database#94/#95. Writers added by other open PRs that insert into context_sources inherit the same fence automatically. Before this PR no API path called withdraw_context_source (checked by grep over api lib/ and app/), so no legacy re-created active rows are expected. Migration name follows the agreed collision rule (20261010 + 2124 + 00). CI runs a separate .github/workflows/source-withdrawal.yml (PostgreSQL 15/16 matrix, this file only) so release-cases.yml is untouched.

Verification

  • Ran (local disposable PostgreSQL 17.7, LC_ALL=C): python3 -m unittest discover -s tests -p 'test_source_withdrawal.py' -v gave 13 passed, 0 failed (12 withdrawal cases + the inherited release-cases lifecycle test).
  • Ran (PostgreSQL 15.10): same command gave 13 passed, 0 failed.
  • New or strengthened cases:
    • every receipt key and JSON type is asserted, and value checks use IS DISTINCT FROM, so a missing key fails;
    • source-version resolution, plus rejection of both/neither identifiers and of foreign, out-of-lineage and unknown versions;
    • lineage-stable affected counts that do not drift after a later withdrawal;
    • the real save_context_metadata writer, captured before the withdrawal, fails closed and does not republish, and the shared upsert returns a born-withdrawn row while another kind and another workspace stay active;
    • two concurrent sessions in a throwaway database copy: a service_role writer started while the withdrawal is uncommitted waits, then fails closed;
    • the accept_context_result revision fence on a live source, both stale (refused) and current (accepted).
  • Mutation checks (scratch copies, each run against the full file):
    • trigger removed: 2 tests fail (late writer, concurrent writer);
    • lock wait removed: 1 fails (concurrent writer);
    • receipt without already_withdrawn/affected: 6 fail;
    • documents count without the lineage filter: 4 fail;
    • previous "current result is null" documents definition: 1 fails;
    • lineage check removed: 2 fail;
    • membership check removed: 1 fails;
    • revision fence removed from accept_context_result: 1 fails;
    • owner filter removed from the version lookup: 0 fail. This mutant is equivalent, because the following source lookup is owner-scoped.
  • Red → green: with the migration absent, every new case fails (function … does not exist). The late-writer probe from review (a late save_context_metadata after withdrawal) republished the document before the trigger and fails closed after it.
  • Not re-run this round: test_release_cases.py / test_company_assessment.py as separate suites (they do not load this migration; the release-cases lifecycle test passes inside this suite).
  • Not verified: hosted deployment, application to any hosted/production database, live providers, PostgreSQL 16 locally (CI matrix covers 15 and 16).

Out of scope / known gaps

  • No restore/un-withdraw operation: once withdrawn, the same workspace URL of that kind stays withdrawn, so a later ingestion that needs it accepted fails rather than silently re-adding it.
  • Earlier result statuses are not preserved when relabelled withdrawn (pre-existing primitive behavior, now documented).
  • Scoped corrections (correct_context_spotify_release still has no API caller), refresh, recomputation within budget, and duplicate-article corroboration rules.
  • Customer deletion/retention of original private bytes and storage objects (database#94/#95 + api#986–#991).
  • Workflows, paid calls, UI.

🤖 Generated with Claude Code