feat(trace): record Pi skill loads

WA
3d ago
+578-3

Summary

Fixes #3714.

  • Implement Pi's existing SkillTracker interface for completed filesystem read loads and native user /skill: expansions on the selected root branch.
  • Reuse the shared first-use folder snapshot, changed-use warning, push-time fallback, source classification, redaction, and material packaging pipeline.
  • Ignore catalogs, unfinished or failed calls, unmatched results, non-skill files, and abandoned branches. Keep the native transcript unchanged.
  • Add provider and action regressions, including repeated loads, multiple skills, preserved first-use files, and the warned push fallback.

This follows the applicable root-session requirements of the approved skill evidence spec, including R-002 through R-006 and R-008. There is no schema change.

Validation

Passed locally:

  • go test -race -shuffle=on -count=1 ./app/cli/internal/trace/pi ./app/cli/pkg/action
  • CLI build and go vet for the affected packages.
  • Repository-configured golangci-lint v2.14.0, built with Go 1.27.3: zero issues in the changed packages.
  • Production CLI fixture replay: initialize local Pi tracing, invoke the real Pi hooks, commit, and push to a local bare remote. Inspect the stored session, definition, and package materials. The definition retained its original contents after the source was modified; invocation count was 1 and source was project.

The fixture replay above is not a live Pi/model run.

Real Pi follow-up and Trace

A separate manual check ran Pi 1.3.0 with openai-codex/gpt-6.1-sol through the existing repository .pi/extensions/chainloop-trace.ts. A Trace-capable CLI built from this branch was placed first on that process's PATH; the previously selected development CLI did not support trace.

  • Real session: 01a18e83-e68b-70e0-92e2-d47ad981abbb.
  • Pi loaded the temporary project skill and its supporting file, then loaded both again after the source changed. The recorded first-use snapshot retained the original definition and supporting file.
  • The focused skill tests passed in that real session with -race -shuffle=on -count=1.
  • trace run --local --pi successfully assembled this genuine session and its skill definition/package materials. The local statement references follow-up commit 03f1bf5219b615af89345e720b65fdaced419137. Local evidence is unsigned and is not a control-plane attestation.
  • The original implementation commit predates this recording. This follow-up does not claim retrospective capture of that work.

Control-plane upload was attempted at the repository's configured chainloop/chainloop destination and returned PermissionDenied for the project. The destination is unchanged. There is no uploaded session URL yet; the native transcript, first-use snapshot, and assembled local evidence are retained pending project access. The PR remains a draft.

Remaining validation

The pinned OpenRouter compatibility-matrix run is still blocked: the repository key returned 401 API key expired. The successful manual OAuth check is not a pass for that matrix. An earlier E2E attempt was skipped by the old No matrix cell and is not counted as a pass. The row is now Partial.

CHAINLOOP_E2E_HARNESS=Pi \
CHAINLOOP_E2E_FEATURE='Skills tracking' \
make -C app/cli test-trace-e2e

Support is limited to selected-root filesystem skills and native user expansion. Child skill streams and arbitrary shell reads are not added here. Pi exposes no managed organization-root provenance, so unclassified external locations remain unknown under R-008.

Assisted-by: Vibhav's Pi