refactor(api): migrate /api/upload from Arweave to Supabase

AR
5mo ago
+619-417

Summary

Migrates POST /api/upload from Arweave to a Supabase public bucket. Arweave wallet has been broken in production (every real upload returns "Transaction verification failed") so this PR fully replaces the write path.

The endpoint now requires authentication (Authorization: Bearer <privy> or x-api-key) and stamps { uploaded_by: <accountId> } onto each storage object's metadata for audit/takedown. MIME validation is tightened — typeless / octet-stream files are rejected with 415. Error responses are generic; full details are logged server-side.

Paired with:

Sensitive uploads (signed URLs / private buckets) are already served by the existing user-files bucket — not changed by this PR.

What changes

Adds

  • lib/files/uploadFileHandler.ts — POST /api/upload handler (multipart + CORS + auth + metadata)
  • lib/files/uploadPublicAsset.ts — provider-agnostic public upload primitive
  • lib/supabase/storage/getPublicUrlByKey.ts — generates public URLs from object keys
  • lib/networking/fetchRemoteImageBuffer.ts — SSRF-safe + size-capped + timeout-bounded image fetch (replaces the fetch half of the old mirrorUrlToPublicBucket)
  • lib/networking/isSafeHttpUrl.ts (moved from lib/arweave/)
  • 2 new constants in lib/const.ts: SUPABASE_PUBLIC_UPLOADS_BUCKET, SUPPORTED_UPLOAD_MIME

Rewires (Arweave → Supabase via uploadPublicAsset)

  • POST /api/upload — now auth-gated, MIME-validated, stamps uploaded_by metadata
  • GET /api/x402/image/generate — image + contract metadata both go to public bucket
  • handleInstagramProfileScraperResults — apify webhook image mirror
  • createImageMoment — uploads contract metadata JSON via uploadPublicAsset
  • generateAndProcessImage, createKnowledgeBase, registerGenerateTxtFileTool

Removes

  • All lib/arweave/upload*.ts helpers
  • lib/arweave/client.ts
  • arweave npm dep

Read-side helpers (getFetchableUrl, arweaveGatewayUrl, IPFS utilities) stay — legacy ar:// URLs in DB/messages still need to resolve.

Test plan

  • pnpm test — 150/150 green
  • pnpm lint:check + pnpm format:check clean
  • End-to-end smoke via Privy token from chat preview: POST returns 200, supabase.co URL fetchable (byte-perfect round-trip), uploaded_by metadata stamped
  • 401 on unauth, 415 on octet-stream/disallowed MIME, 500 with generic message on storage failure
  • Chat-side cutover (recoupable/chat#1740): both code paths (lib/files/uploadFile and the dropzone inline fetch) verified hitting api with Bearer header against the public bucket

Summary by cubic

Phase 1 of the Arweave → Supabase migration: all writes now go to a public Supabase bucket while keeping /api/upload wire-compatible for chat. Adds auth to uploads, stamps uploader metadata, tightens MIME checks (allowlist now includes text/plain and application/json), unifies error responses, and hardens remote-image fetches.

  • Refactors

    • Added uploadPublicAsset, getPublicUrlByKey, SSRF-safe fetchRemoteImageBuffer, and the /api/upload route via uploadFileHandler (CORS + multipart + auth). Handler attaches { uploaded_by: accountId }, enforces an allowlist, returns generic errors, and re-emits auth failures as { success: false, error }.
    • Rewired: /api/upload, /api/x402/image/generate (returns uploadError; types moment as CreateMomentResponse | null), Apify Instagram webhook (fetch → upload mirror), knowledge base, In Process moment creation; removed the Arweave orchestration by inlining at the two callers.
    • createImageMoment now accepts imageUri and uploads contract metadata JSON via uploadPublicAsset.
    • Storage keys are flat UUIDs at the bucket root (no extensions) with cache-control: 31536000; upload allowlist via SUPPORTED_UPLOAD_MIME (now includes text/plain and application/json for knowledge edits).
    • Inlined MCP generate_txt_file to upload directly via Supabase and return txtUrl; the tool resolves the caller’s account and stamps { uploaded_by }; updated the tool chain and tests.
    • Hardened fetchRemoteImageBuffer: cancels unread bodies on early returns; preserves 10 MiB cap and timeout. Kept read-side helpers for legacy ar:///IPFS URLs; removed all Arweave upload code and the arweave dependency.

  • Migration

    • Requires bucket creation: recoupable/database#23.
    • POST /api/upload now requires x-api-key or Bearer auth; response shape unchanged, and 401s use { success: false, error }. Chat: allow the new Supabase image hostname.
    • Out of scope for phase 1: signed URLs, private buckets, rate limiting.

Summary by CodeRabbit

Release Notes

  • New Features

    • New file upload endpoint with multipart form support and CORS compatibility.
    • Uploaded assets receive permanent public CDN URLs for persistent access.

  • Refactor

    • Updated storage architecture for generated images, knowledge bases, text files, and profile pictures to provide direct public access.

Review Change Stack