PR #34604 - manaflow-ai/cmux

Leo (2026-11-03), on the sync speed-ups: "yeah let's do alllll of those things! :DDD". This is the content-addressed blob store from that list.

What it does

cmux link sync push --store PLATFORM|default (and start-chat copy --store, which the app passes for a personal account on non-production auth) sends a folder through the user's sync store on the cmux platform instead of byte by byte over the link:

The Mac hashes each file it has to write. Files up to 4 MiB are one blob keyed by their sha256; bigger files are cut by content-defined chunking (FastCDC with normalized chunking, 512 KiB min / 2 MiB average / 8 MiB max), so an edit in the middle changes only the chunks around it.
It asks the store which blobs it lacks (POST /v1/sync/blobs/missing), uploads only those (PUT /v1/sync/blobs/{sha}, 16 at once, each piece re-read and re-hashed before it goes), then uploads the plan (itself a blob: every file's path, mode, size, sha256 and chunks).
It mints a read-only grant (POST /v1/sync/grants, csg_…, one hour) and sends the machine blob.fetch.start {store:{origin, grant}, plan}. The machine never sees the account credential.
The machine daemon (new capability blob-fetch-v2, ops blob.fetch.start / .status / .cancel / .forget) fetches the plan and blobs from the store, 16 files at once, verifies each chunk's and each file's sha256, and writes through the same Roots sandbox and PendingWrite staging as fs.write (atomic rename, permissions set before commit). A path outside the roots is refused and named.
The Mac reads each file's revision back and records it in the sync manifest as before.

The direct link copy stays the fallback and is unchanged:

a machine without blob-fetch-v2, no --store, or a store that fails before the machine writes: everything goes over the link.
a file the machine refused: that file goes over the link.
less than 1 MiB to write (a typical one-file edit): over the link, since the store's round trips cost more than the bytes (store.reason: "small").

Blobs are keyed per user (v1/u/<user>/<sha[0..2]>/<sha>), so identical files are stored once per user, a second machine costs the Mac only the plan, and a push cut off partway is resumed by the next push re-planning: the store already has the blobs, so the Mac uploads nothing again.

Server side: manaflow-ai/cmux-platform#746 (merged; the routes are live in development only, so staging and production answer 404 and the push falls back to the link).

What I took from Mutagen

Content is staged and verified by digest, then renamed into place atomically (Mutagen's staging + transition). Here: PendingWrite plus a per-chunk and whole-file sha256 check.
The receiver pulls what it lacks instead of the sender pushing everything (Mutagen's rsync-style stage from the receiving end).
The sync state (our manifest) is kept separate from the transfer.

Where it differs: Mutagen has two ends; this puts a store in the middle, so a third end (another machine, a resumed machine) costs the Mac nothing. Rsync deltas inside a changed chunk are a possible follow-up.

Performance

Same harness for both columns: cargo test -p cmux-tui --test link_sync_store -- --nocapture (a fake platform and a fake machine daemon running the real FsService and BlobFetchService, loopback). Repo: 600 files of 2-24 KiB, a vendored 200 KiB file twice, a 24 MiB asset, committed. "Bytes from the Mac" is bytes over the link plus bytes uploaded to the store. Run on the Linux pool (fast checks).

metric	before (direct link copy)	after (store)	delta	harness
first sync, bytes from the Mac	89,610,640 B	66,955,373 B (134,865 link + 66,820,508 store)	-25.3%	link_sync_store, run 41809330696
second machine, bytes from the Mac	89,610,640 B	235,291 B (134,865 link + 100,426 store)	-99.7%	link_sync_store, run 41809330696
one-file edit, bytes from the Mac	131,459 B	131,123 B (over the link: under 1 MiB)	-0.3%	link_sync_store, run 41809330696
1 byte edited in a 24 MiB file, bytes from the Mac	33,682,014 B	3,115,369 B (126,805 link + 2,988,564 store)	-90.8%	link_sync_store, run 41809330696

Wall time on loopback, where bandwidth is free, is about the same: first sync 1.08 s before and 1.17 s after, second machine 1.07 s and 1.09 s. On a real link, the bytes are what it pays for. The link bytes in each store push are the folder listings and the manifest. Before the 1 MiB floor, a one-file edit took 401 ms through the store and 103 ms over the link, so small writes stay on the link. Numbers are measured on this branch merged with feat-cmux-next, which sends git loose objects as one pack (#34537).

Tests

blob_fetch unit tests: a chunked plan written with its permissions, each file's revision from its commit, and a rerun after a machine-side edit; a blob that does not match its hash fails and writes nothing; a path outside the roots is named and not written; a plan file outside the job's folder fails the plan; bad origins, grants and targets refused before any fetch; starts past the running cap refused; a blob body past its size cut off; a symlink at the path refused; a job held mid-fetch writes nothing after a newer start or a cancel under its folder; a cancel names every job kept there until it is forgotten; only the four ops are frames.
sync_cli::store unit tests: chunks cover the file within bounds; an edit in the middle changes at most two pieces; the same bytes always cut the same way.
link_sync_store: the end-to-end test above, plus a store that refuses the credential still lands every file over the link, and a lost blob.fetch.start answer (with the link) fails the push as unsettled, then the next push fences the job and lands the whole tree. A machine that advertises the pre-release blob-fetch-v1 (no cancel) gets a plain link copy, with or without --store. A push whose fetch a newer push cancelled stops without a link copy. A file only a failed push's fetch wrote is deleted by the next push whose tree drops it.
The link gate tests admit the four ops and refuse an unknown blob.fetch.* one.

Review findings fixed in this PR: #34788 (each written file's revision comes from its commit), #34789 (no direct link fallback while the machine's fetch may still be running), #34790 (blob bodies are cut off past their size). Also fixed from review: at most 4 live fetch workers (blob.busy), a cancelled one counting until its thread ends; a symlink at a target path is refused; and a fence: every job names its folder (target), a start and the new blob.fetch.cancel cancel each running job under an overlapping folder and answer only once none of them can commit (a commit and its cancelled check share one lock), and every push to a machine serving blob-fetch-v2 fences first. The capability is blob-fetch-v2: pre-release builds of this branch advertised blob-fetch-v1 without cancel, and a Mac treats those machines as serving no store. So a retry with a changed plan, a push under the 1 MiB floor, or a link fallback after a lost start answer is never overwritten by an older job. A start answer that is lost stops the job, or fails the push when the stop cannot be confirmed. A push whose fetch was cancelled by a newer push stops (blob.superseded) instead of copying over the link. What a cancelled or failed fetch committed is in no manifest, so the cancel answers every job kept under the folder, the next push reads what each wrote and counts those files as its own at that revision (deleting any its tree dropped), and it forgets the jobs once its manifest is saved (blob.fetch.forget). Still open as follow-ups: #34791, #34861.

Risk

Medium. New daemon ops behind a capability and the existing fs gate; a new Mac-side path behind --store. Every failure falls back to the existing link copy. Only NEXT uses this code.

Why this is one PR (about 1,900 lines with tests): the machine op and the Mac client are two ends of one wire format (the plan), and the end-to-end test that measures it needs both.

Follow-ups: pack small files into one blob; a machine-side chunk cache; per-user quota and GC (cmux-platform#751); presigned direct-to-R2 uploads.

Changelog

Added: Folder sync to a Cloud machine can go through your sync store: only content the store lacks is uploaded, and a second machine or a resumed sync moves almost nothing from your Mac.

Agent session: agent session

🤖 Generated with Claude Code