fix(cli): evaluate policies against the material on disk in policy devel eval
chainloop policy devel eval fed policies the redacted copy of a material instead of the file on disk. A policy hunting for leaked credentials in a CHAINLOOP_AI_CODING_SESSION saw sanitized input and reported no violations — silently, with no error. The tool used to author policies therefore disagreed with what production does.
Cause
The crafter marks a redacted material with chainloop.material.redacted, and that marker is what makes GetEvaluableContent read the untouched local file rather than the stored content. devel eval replaced the crafter's annotation map with the one built from --annotation flags, dropping the marker. With no flags at all the map was emptied outright, and since a dry run always stages inline, the sanitized bytes were what policies evaluated.
Change
User annotations are now layered on top of the crafter's rather than replacing them, so the redaction marker and the rest of the crafted metadata survive.
The chainloop. namespace is treated as crafter-owned and is not overridable, so an --annotation chainloop.material.redacted=false cannot put the old behaviour back. This matches how Crafter.stageMaterial protects contract-provided annotations on attestation add.
Only the stored copy stays redacted; that part is unchanged.
AI disclosure
Assisted by Claude Code.
🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri