fix(ci): actually prune the BuildKit layer cache

Merged
WA
2mo ago
+82-15

Summary

The max-cache-size-mb input I added in 8fa7f0cc4e never did anything. Every Docker build since has logged:

##[warning]Unexpected input(s) 'max-cache-size-mb', valid inputs are ['cache-key',
'buildx-version', 'platforms', 'nofallback', 'github-token', 'buildkit-version',
'skip-integrity-check', 'driver-opts', 'max-parallelism']

setup-docker-builder v1 accepted that input and pruned in its own post step. The v2 rewrite dropped it, and we pin v2.1.0. GitHub only warns on an unknown composite input, so it failed silently — the app disk sat above 200 GB for a full day while the config looked correct.

Scanned every tag rather than infer:

Taghas max-cache-size-mb
v2.1.0 (pinned), v2.0.1, v2.0.0no
v1.12.0, v1.11.0, v1.10.0, v1.9.0, v1.8.0yes

The fix

Rather than downgrade a builder rewrite to reach a config knob, run the prune ourselves — v1's command verbatim (from its dist/index.js pruneBuildkitCache), against the fixed address v2 itself uses for buildctl du and debug workers:

sudo buildctl --addr tcp://127.0.0.1:1234 prune --all --keep-storage <MB>

Both flags verified against BuildKit master, not assumed

  • buildctl's --all is not docker buildx prune --all. It means "Include internal/frontend references". cache/manager.go shows the only records skipped without it:
    if !opt.all {
        if recordType == UsageRecordTypeInternal || recordType == UsageRecordTypeFrontend || shared {
            continue
        }
    }
    
    It does not wipe the cache. (I initially justified --all as being required to reclaim RUN --mount=type=cache dirs — that was wrong. Those are typed exec.cachemount and are reclaimed either way. --all is kept because it's what v1 used and it prunes strictly more.)
  • --keep-storage is not deprecated. In cmd/buildctl/prune.go it maps straight onto the modern MaxUsedSpace field — it is buildctl's spelling of --max-used-space, and the only size flag buildctl exposes.

Why a foreground command

Turborepo's equivalent eviction runs on a detached thread that is never joined, and on a 206 GB cache it makes no measurable progress inside a 150s job (measured: the disk moved 206.2 → 206.7 GB across 17h of builds). A blocking buildctl prune either completes or reports why not.

Warns rather than fails — an oversized cache isn't worth failing a deploy over — but prints buildctl du either side, because a silent no-op is precisely the failure mode that hid this for a day.

Verification

  • Extracted v1's and v2's bundled dist/index.js; confirmed v1's exact command and that v2 hardcodes tcp://127.0.0.1:1234 and already shells out to buildctl, so binary and address are both present at that point in the job.
  • Flag semantics read from BuildKit master source (cmd/buildctl/prune.go, client/prune.go, cache/manager.go, client/diskusage.go).
  • actionlint output identical to the staging baseline; shellcheck clean on the new run block.
  • bun run lint and all 39 check:audits pass.

Effect is not yet measurable — it applies from the first build after merge. I'll report the real du deltas from build logs rather than assume.

Type of Change

  • Bug fix

Testing

As above. The real verification is the before prune -> / after prune -> lines this adds to every Docker build log.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)