Snapshot shm via APFS fclonefileat for safe CoW

Merged
JS
5mo ago
+395-69

Fork previously sent the parent's live shm_fd via SCM_RIGHTS and the child mapped it MAP_PRIVATE. The parent stayed on MAP_SHARED, so any page the child had not yet COW'd reflected the parent's current bytes. That was benign for typical aarch64 workloads but corrupted x86_64-via-Rosetta guests: translator-internal structures (TLS slabs, code caches, indirect-call tables, block lists) cross page boundaries and observe parent-side mid-update reads. Issue #45 tracked the resulting fall back to a per-region byte copy through the IPC socket -- 10-16x slower per fork than the CoW path.

sys_clone now lifts the !g->is_rosetta gate and always asks fork_snapshot_shm_via_clonefile() for an APFS fclonefileat snapshot of the shm file. The clone shares blocks with the parent until either side writes, so the parent's later writes never reach the child's backing, and the existing guest_init_from_shm MAP_PRIVATE flow on the child consumes the snapshot unchanged. The snapshot helper uses a mode 0700 mkdtemp directory (clone inside, then unlink + rmdir) rather than an earlier mkstemp + unlink + fclonefileat sequence whose freed /tmp basename gave a local-user TOCTOU window that could DoS the fast path via EEXIST.

Fallback differs by guest. Rosetta drops use_shm on clonefile failure and falls through to the legacy region-copy path; sending the live fd would re-introduce the issue #45 corruption. Native guests keep use_shm and send g->shm_fd directly, preserving the original CoW behavior so a non-APFS /tmp does not silently slow forks down to per-region copy cost.

Overlay sync (pwrite of file-backed MAP_SHARED overlay bytes into shm_fd) moves before the IPC header so the cloned file picks up overlay-backed bytes and the header has_shm field reflects the post-clonefile outcome.

guest_init_from_shm now closes shm_fd on the compute_infra_layout failure path so the take-ownership contract holds on every error, not just the post-mmap ones.

tests/bench-fork-cost.sh is added as the regression baseline. Per-fork wall-clock means from three back-to-back runs on M1 (subshell fork, no exec, per-fork numbers exclude startup via a 0-iter baseline subtraction):

  rss      aarch64 CoW    Rosetta (before / after)   ratio
  0 MiB    ~113 ms/fork   ~1058-1196 / ~113 ms       10x -> ~1x
  1 MiB    ~113 ms/fork   ~1090-1250 / ~117 ms       10x -> ~1x
  16 MiB   ~114 ms/fork   ~1125-1230 / ~120 ms       10x -> ~1x
  64 MiB   ~114 ms/fork   ~1400-1840 / ~220 ms       12-16x -> ~2x

The 64 MiB Rosetta residual is APFS clone metadata plus child-side MAP_PRIVATE materialization, not byte-copy bandwidth. test-cow-fork (5/5), make check, and the 71-test make test-rosetta-all suite stay green.

Close #45

Summary by cubic

Snapshot guest shared memory at fork with APFS fclonefileat to give the child a safe CoW snapshot, fixing Rosetta torn-read corruption and restoring fast‑path performance (addresses #45). Adds per-guest fallbacks, syncs overlay data before the snapshot, and includes a fork-cost benchmark.

  • New Features

    • Snapshot g->shm_fd via APFS fclonefileat; child maps the clone MAP_PRIVATE.
    • Attempt clone for all guests; on failure, Rosetta falls back to region-copy, native guests send the live shm_fd.
    • Sync overlay bytes into shm_fd before snapshot; IPC header has_shm reflects the final path.
    • Add tests/bench-fork-cost.sh to track per‑fork cost.

  • Bug Fixes

    • Remove TOCTOU by cloning inside a mode 0700 mkdtemp dir, then unlinking.
    • Ensure guest_init_from_shm closes shm_fd if compute_infra_layout fails.
    • Avoid fd double‑close/leaks (reset vfork_notify_fds[1], close snapshot fd on all exits) and reap failed children to prevent zombies.