Add OCI image support

Draft
HE
3mo ago
+16654-163

Prior work: https://github.com/sysprog21/elfuse/pull/34
Tracking issue: https://github.com/sysprog21/elfuse/issues/31

We introduce elfuse-oci, a standalone Go companion binary that owns the OCI image pipeline: pull, unpack, inspect, run, list (alias images), rmi, and prune, backed by a real OCI image-layout store. elfuse itself stays a pure Linux syscall-to-Darwin runtime with no OCI awareness; the two binaries meet only at the existing elfuse --sysroot <rootfs> <entrypoint> <args> launch path.

OCI images are used here strictly as a distribution format for Linux root filesystems — a reproducible replacement for hand-built --sysroot trees. This is not a container runtime. There is no isolation layer: the guest shares the host network identity, PID space, and clock, and unresolved guest paths fall back to host truth. A workload that needs namespaces, cgroups, port mapping, exec into a running container, or a daemon needs a real container runtime, not an ELF personality.

Why a separate Go binary

The OCI ecosystem is a Go ecosystem. Rather than grow the C runtime with an image pipeline, the acquisition/lifecycle half is ~4.4k lines of Go (plus ~7.3k lines of tests) built on github.com/google/go-containerregistry, and validated for on-disk conformance against crane, skopeo, and umoci. Keeping it out of the C tree keeps both sides small and lets each lean on its native tooling.

Design

docs/oci-design.md is the source of truth: the C/Go boundary, the image-layout store and its refs.json pin table, hardened layer application, the run paths, the concurrency/locking model, and an explicit accounting of which OCI features are and are not implemented. docs/usage.md covers the commands; docs/testing.md the offline/CI split; docs/internals.md the host-literal path fallback.

Highlights:

  • Store: spec-shape OCI image-layout other tools can read; refs pinned by digest; an exclusive flock serializes metadata writes so concurrent pulls cannot lose pins.
  • Unpack: os.Root-bounded extraction (no symlink/hardlink escape), correct whiteout/opaque handling, staged temp-dir + atomic rename so readers never see a partial tree.
  • run: resolves Entrypoint/Cmd/Env/User/WorkingDir with env(1)/Docker precedence, guarantees a PATH, resolves symbolic --user against the image's own /etc/passwd+/etc/group (no-follow), injects host /etc/{resolv.conf,hosts,hostname}, then execs elfuse in place so signals and the pid pass through.
  • macOS rootfs: per-digest case-sensitive APFS sparsebundle with per-run clonefile COW; liveness/lifecycle decided by per-digest advisory flocks (attach.lock/run.lock), not pids. --plain-rootfs remains available.
  • Lifecycle GC: rmi/prune use reachability GC (shared blobs survive while any ref reaches them) and never reclaim a cache a live run still holds — the run-lock rides through the exec into elfuse and releases exactly on guest exit.

Try it

make elfuse elfuse-oci

build/elfuse-oci pull alpine:3
build/elfuse-oci inspect alpine:3
build/elfuse-oci list

build/elfuse-oci run alpine:3 /bin/sh -c 'echo hello from elfuse'
build/elfuse-oci run --entrypoint /usr/local/bin/python3 python:3.12 \
  -c 'import json,math; print(json.dumps({"pi":round(math.pi,5),"ok":True}))'

build/elfuse-oci rmi --force alpine:3
build/elfuse-oci prune --cache --all

Images are stored under $ELFUSE_OCI_STORE, or ~/.local/share/elfuse/oci by default.

CI

Split by runner capability

  • a Linux job exercises the pure-Go store paths (pull, unpack, inspect, lifecycle) plus image-layout conformance and cross-tool interop (crane/skopeo/umoci) with no Hypervisor.framework
  • a hosted macOS job builds and tests the darwin sparsebundle code and a run-less lifecycle smoke

Summary by cubic

Adds OCI image support with a new Go CLI, elfuse-oci, an extracted elfuse_launch, and a macOS case‑sensitive APFS sparsebundle rootfs with per‑run COW. Delivers a full pull → inspect/unpack → run → rmi/prune lifecycle with reachability GC, crash‑durable store writes, and stronger path, symlink, exec, AF_UNIX, and inotify handling.

  • New Features

    • OCI CLI/store: pull, unpack, inspect/list --json, run, rmi, prune; spec‑shape OCI image‑layout with refs pinned by digest; synced pin/index/blob writes; reachability GC; --platform (default linux/arm64); timed keychain; built on github.com/google/go-containerregistry.
    • Run: resolves Entrypoint/Cmd/Env/User/WorkingDir; adds --user/--workdir/--env/--clear-env with env(1) semantics; guarantees PATH; creates missing WorkingDir; resolves symbolic users from in‑image passwd/group (no‑follow); injects host /etc/{resolv.conf,hosts,hostname}; forwards INT/TERM/QUIT/HUP; uses -- before guest argv; execs elfuse --sysroot.
    • macOS rootfs: per‑digest case‑sensitive APFS sparsebundles with per‑run clonefile COW; advisory flock lifecycle (attach.lock/run.lock) that survives exec; reuse vs. force‑detach of stale mounts; --keep sidecars; --plain-rootfs retained; safe prune/rmi that skip busy caches.
    • Filenames/paths: case‑escape codec + case‑exact walk; clamps .. at guest root; guest‑namespace symlink targets; PT_INTERP resolution; /proc/self/{exe,fd} reverse‑map; pathname AF_UNIX translate/shorten and reverse‑map; inotify path translate/name decode; enforce O_NOFOLLOW/O_NONBLOCK; unify /tmp//var/tmp/~/.ccache redirects.
    • C runtime: extracted elfuse_launch; new launch flags mapped to staged initial IDs, absolute Workdir validation, and conflict guards.

  • Docs & CI

    • Docs: added docs/oci-design.md; updates to docs/usage.md, docs/testing.md, docs/internals.md.
    • Tests/CI: image‑layout conformance and cross‑tool interop (crane/skopeo/umoci); sparsebundle round‑trip; exec checks; end‑to‑end lifecycle; per‑image workloads (python/node/go/jvm/c). Linux runs pure‑Go store paths; hosted macOS builds/tests darwin code; self‑hosted Apple Silicon runs full HVF flows. New hvf-elfuse-setup composite action.