Remove the leftover upload copy after a retried upload is admitted

Merged
BI
1mo ago
+235-0

User description

Summary

  • Before: when an upload was retried after an earlier attempt with an unknown outcome, the retry found an identical file already at its intended storage path. It then stored a second recovery copy and admitted that one, leaving the first copy in private storage with nothing referencing it (Gitar follow-up from PR #3546).
  • After: once this request's own admission is durable on the recovery copy, the collided object at the intended path is removed. If removal fails, it is queued on the existing storage_cleanup_jobs ledger. If both fail, the upload still succeeds and the path is logged for manual reconciliation.
  • Why this is safe: the document id is fixed by the correlation id, so once this admission commits no other admission for that document can commit. Nothing else can reference the intended-path object. The release never runs before the admission is durable, and never touches the path the committed row uses (both pinned by tests).
  • No database change. It uses the existing cleanup ledger table only.
  • Records the owner's 16 Oct decision to leave a job stuck on its final attempt to the existing 6-hourly ingestion check and queue recovery, rather than a new database fix (docs/decisions/2026-10-16-stuck-final-attempt-uses-existing-recovery.md, docs/ingestion-state-machine.md C1a). Drops the two ledger inbox rows that this change and that decision close.

Areas touched: Source intake and indexing, Delivery and assurance, Knowledge and records

Verification

  • Checks run:
    • npm run verify:cheap: Test Files 1943 passed (1943) and Tests 31531 passed | 2 skipped (31533), lint and typecheck clean.
    • npm run test:focused on the changed files: Tests 228 passed (228).
    • npm run check:decisions: 0 errors, 0 warnings.

Risk and rollout

  • Risk: low. It only deletes an object under this document's own storage prefix, after its own admission committed, and only when the upload path recovered from a collision.
  • Rollback: revert this PR. A leftover copy would then stay in storage as before.
  • Provider or production effects: none beyond the normal app deploy on merge.

Clinical Governance Preflight

  • Source, citation, fencing and server-only access checks retained. The admission and its stored source bytes are unchanged.
  • No clinical content, answer generation, ranking or retrieval change.
  • No patient data and no new provider call.
  • General product legal/clinical classification and unrestricted release readiness established (unchanged by this PR).

Notes

  • Draft until the owner marks it ready. Marking ready lets Gitar merge it once green.

🤖 Generated with Claude Code

https://claude.ai/code/session_01M6gRtN6r4vZDgDfzMtjs95

Generated by Claude Code

Summary by cubic

Removes the orphaned upload copy left behind when a retried upload collides with an identical file already at its intended storage path.

Previously, a retried upload with an unknown earlier outcome stored a second recovery copy and admitted it, leaving the first copy in private storage with nothing referencing it. Now, once this request's own admission is durable on the recovery copy, the collided object at the intended path is removed, or queued on the existing storage_cleanup_jobs ledger if removal fails. Each cleanup call has a 5-second deadline so a committed upload is not held open; if both fail, the failure reasons and path are logged and the upload still succeeds. This is safe because the document id is fixed by the correlation id, so no other admission for that document can commit.

  • Records the owner's decision to leave a job stuck on its final attempt to the existing 6-hourly ingestion check and queue recovery, rather than adding a database fix. The check reads only the 25 oldest open jobs; queue recovery scans all open jobs but applies at most --limit actions (default 20) per run, via --apply, --yes, or an interactive confirm.
  • No database change; only the existing cleanup ledger table is used.

CodeAnt-AI Description

Clean up collided files after retried uploads succeed

What Changed

  • After a retried upload is admitted using its recovery copy, the identical file left at the intended path is removed.
  • If removal fails, the file is queued for cleanup; cleanup failures do not undo the successful upload.

Impact

✅ Fewer orphaned files after retried uploads
✅ Successful uploads despite cleanup failures

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

Summary by CodeRabbit

  • Bug Fixes
    • When an upload is saved under a replacement storage path after a collision, the superseded file is now removed or queued for cleanup after the upload is committed.
    • Cleanup issues that cannot be recorded no longer cause an otherwise successful upload to fail.
  • Documentation
    • Clarified how ingestion jobs with uncertain outcomes on their final attempt are identified and recovered.