acpmux: resumable uploads report where they stand, check their bytes, and fail on size only when the disk is full

Merged
TE
11d ago
+606-53

Resumable uploads that say where they stand, check their bytes, and fail on size only when the disk is full. Follows #25730 (landed). Contract: #25774, sections 3 and 4.

Leo: "why the FUCK SHOULD THERE BE A FUCKING SIZE CAP ANYWAYS JUST FUCKING / I ALREADY SAID WE NEED ARBITARRILY LARGE FILES OF ANY TYPE". The spec this follows: uploads are resumable with progress, and the only failure allowed is disk-full, with host and path.

Change (src/hub/file_uploads.rs)

  • _acpmux/file_put_status {sessionId, uploadId} answers {uploadId, fileName, size, offset, finished, path?}, so a sender resumes at offset after a disconnect or a daemon restart. Routed like the other file_put_* calls: refused to Web (filePut.remote), forwarded to the peer that owns a peer chat.
  • finish takes an optional sha256 (64 lowercase hex). A mismatch is upload.corrupt {sha256} and removes the upload. It answers {path, size, uploadId} (size and id added, so a native sender can report the upload finished). A repeated finish (its answer was lost) answers the same.
  • Reasons where there were none: upload.unknown, upload.offset {offset} (resume there), upload.incomplete {offset}.
  • A full disk: ENOSPC/EDQUOT on any write, or a begin whose size is past the free space (statvfs), is upload.diskFull {host, path, needed, free, offset, retryable: false}. A short write is cut back to where it began, so the sender's resume offset stays right.
  • Calls on one upload (append, status, finish, cancel) never interleave: they hold one of 64 striped locks keyed by the canonical upload id, so every spelling of an id shares it. A finish cut short after its rename recovers: status says finished, and the repeated finish answers the path. A repeated finish checks a given sha256 against the finished file.
  • The age sweep decides and deletes an upload under that upload's own lock as well as IN_USE, so it never deletes one mid-write, mid-finish or mid-status. It keeps an unfinished upload whose .part was written since its cutoff. A status call marks the upload it answers for used, finished or not, under the sweep's lock (IN_USE, then the upload's), and refuses one it cannot mark (upload.unreadable). Status after the chat was purged is refused, like the other upload calls.
  • A peer on an older build is refused filePut.unsupported for status and for a finish with sha256, before anything is sent, so its upload is never taken as checked when it was not.
  • The chunk write is one shared append (the binary-frames PR reuses it).
  • Feature uploadResume in initialize.

Risk: low. Existing file_put_* callers see the same successes; failures gain a data.reason, and a repeated finish now succeeds instead of failing with "unknown upload".

Tests

tests/upload_resume.rs an_upload_resumes_from_its_status_and_its_finish_is_checked: status mid-upload, a stale chunk (upload.offset with the resume offset), an early finish (upload.incomplete), finish with sha256, a repeated finish, status after finish, a corrupt finish, a wrong sha256 on a repeated finish, a begin of 2^62 bytes (upload.diskFull with this host, the chat's path, needed and free), and Web refused. calls_on_one_upload_never_interleave_and_a_cut_short_finish_recovers: 8 concurrent writes at offset 0 (half with the upper-case id) where one lands, 2 concurrent finishes giving one path, and a finish cut short after its rename. a_peer_that_cannot_check_uploads_is_refused_before_anything_is_sent. The sweep unit test keeps an old upload still being written, and status marks a finished upload used. Plus a unit test that ENOSPC maps to upload.diskFull with host and path.

  • On #25730's head plus the test (run 40425229563): fails (no uploadResume, no status).
  • The newest tests on their parents: the finish answer at bad763809de (run 40443503891) and status marking at 115ab57aea6 (run 40444959707) both fail.
  • At this head 85a0b100709 (run 40444956338): upload_resume 3 passed, file_put 7, prompt_uploads 8, send_attach 12, conn_auth 3, lib file_uploads/prompt_uploads/protocol 15, hub_integration protocol 1, cargo clippy -p acpmux --all-targets -D warnings clean.

Agent session: agent session

šŸ¤– Generated with Claude Code