build: Reduce repeated work in the Docker image builds
Stacked: #37344 (syft SBOM) targets this branch. Merge this first.
Summary
The Docker images were built by four near-identical build-push-action steps in
sequence, each using about a quarter of the runner. This moves targets, tags,
pins and build args into one docker/docker-bake.hcl and builds them
concurrently — so CI and pnpm build:docker finally drive the same definition.
The release images were previously produced by a code path nobody ran locally.
Measured on amd64, same runner class:
| before | after | |
|---|---|---|
| Release image builds | 615s | 140s |
| amd64 release job | 17m09s | 4m48s |
| Image size | 2.19GB | 2.16GB |
Largest contributor first: bake parallelism (−406s), dropping the SBOM scan,
relocating the release smoke checks, native modules compiling above
COPY ./compiled, and one copy of the 1.3GB application tree instead of two.
Blacksmith metrics confirm the mechanism — CPU during the build step goes from
~26% to 71%.
The second commit adds the verification the first leans on: a probe asserting the
published manifest is an OCI image index, with a negative control that still
reproduces the 2.26.0 format, and the kafka binding check moved into
smoke-n8n-image.mjs so a broken native build fails on the PR rather than the
next nightly.
Three decisions that look wrong
- Every job passes the same
n8n-io/n8ncache key. Splitting per image was
the point of the v2 upgrade, but Blacksmith never populates a newly created
sticky disk — two new keys stayed at 0 bytes across four runs that each wrote
11GB and logged a successful commit. Splitting would disable layer caching
outright. Upstream: useblacksmith/setup-docker-builder#125. - Two grafts deliberately skip
COPY --link. Their destinations traverse
pnpm symlinks, and a--linkcopy resolves against an empty root — it replaces
node_modules/sqlite3with a directory, with no build error and only a runtime
failure. Hit and fixed during development. sbom: trueis gone, which #31997 added deliberately. It was forcing the
OCI index format as a side effect;oci-mediatypes=truegets the same format
without the syft scan, and without the attestation manifests that were
themselves what containerd misread.
Two smaller trade-offs: the e2e docker-image-size metric stops being emitted
(no images in the daemon, and that image was never the shipped artifact), and the
8vCPU bump is a latency choice at ~2× runner cost — 209s → 175s on the bake step.
How to test
Green on this PR: docker-build-smoke (two-arch, no-cache, all four image smoke
checks), ci-pull-requests (tarball write + shard restore), test-e2e-helm.
docker-build-push.yml has no pull_request trigger, so the release path needs
a dispatch:
gh workflow run docker-build-push.yml --ref devp-docker-bake-consolidation \
-f push_enabled=true -f include_arm64=false
gh workflow run docker-sbom-probe.yml -f sbom=false
Already run: release build green with pushing; published manifest is
application/vnd.oci.image.index.v1+json with zero attestation manifests; and
the image pulls and serves on AKS — the platform whose containerd failed on
2.26.0 — verified with a staging deploy.
Related Linear tickets, Github issues, and Community forum posts
Follow-up to #31997 (manifest format).
Review / Merge checklist
- I have seen this code, I have run this code, and I take responsibility for this code.
- PR title and summary are descriptive. (conventions)
- Docs updated or follow-up ticket created.
- Tests included.
- PR Labeled with
Backport to Beta,Backport to Stable, orBackport to v1(if the PR is an urgent fix that needs to be backported)