feat(artifact-cas): verify uploaded content against the declared digest
Merged
JA
1mo ago+949+949+012345678901234567890123456789-507-507-012345678901234567890123456789
The Artifact CAS keyed stored artifacts by their client-declared SHA256 without ever
checking that the uploaded bytes hash to that digest. Only the OCI backend verified, via
its own layer-digest check. On S3, S3 access point and Azure Blob any identity holding
artifact:upload could store arbitrary content under an arbitrary digest key.
The CAS now streams each upload to a local staging file, hashes it while receiving, and
hands the backend a file only once its digest matches the one the client declared. A
mismatch is rejected with InvalidArgument and nothing reaches the backend.
What changed
- A single spill-then-upload path shared by all backends, replacing the per-backend
streaming branch and theStreamingUploaderinterface. Verification is no longer
something each backend implements, and any future backend inherits it. - Staging files are unlinked as soon as they are created and used through the open
descriptor, so the kernel reclaims the space whenever the process exits, including when
it is killed mid-upload. Nothing needs to sweep or reap the staging volume. staging_diris required: the CAS refuses to start without one and proves it is
writable before serving traffic, so a misconfiguration fails at boot rather than
producing a healthy service that rejects every upload.- Helm: a dedicated staging emptyDir mounted unconditionally at a fixed path. It sets no
sizeLimit, since exceeding one evicts the Pod rather than failing the upload that
overran it. - Staging on disk keeps the service's own memory use independent of artifact size. The
OCI backend still requires the whole layer in memory, unchanged here. - The S3 backend relies on this service-layer verification rather than a per-object
checksum precondition, which cannot be expressed for multipart uploads.
Addresses PFM-6988.
This contribution was produced with the assistance of Claude Code.