feat(trace): capture pinned Pi subagent evidence

Closed
WA
2d ago
+817-37

Summary

Fixes #3713.

  • Parse single, parallel, and chain results from the official Pi subagent example pinned at 71023aec, shipped with Pi 1.3.0.
  • Preserve child conversations and tool results in deterministic, distinct raw-session streams.
  • Aggregate native child usage exactly once under the existing v1 contract: input plus output forms total tokens, caches stay separate, and reported cost remains authoritative.
  • Preserve root branch selection and ignore unrelated extension records or malformed optional child details without blocking push.
  • Add parser and material-packaging regressions plus a supported-format live E2E scenario that verifies the shipped example source hashes.

No schema or dependency change. Arbitrary third-party extensions and child inference from parentSession are outside this scope.

Validation

Passed locally:

  • go test -race -shuffle=on -count=1 ./app/cli/internal/trace/pi ./app/cli/internal/trace/e2e ./app/cli/pkg/action
  • CLI build and go vet for the affected packages.
  • Repository-configured golangci-lint v2.14.0, built with Go 1.27.3: zero issues in the changed packages.
  • Production CLI fixture replay through real hooks, commit, and a local bare-remote push. The stored material retained a distinct child stream and its read result. Root input/output of 2/2 plus child 3/5 produced 5/7 and total 12; caches remained 7/11 and reported cost remained 0.02.

The fixture replay above is not a live Pi/model run. Unit regressions cover single/parallel/chain modes, failures, malformed optional details, repeated results, and abandoned branches.

Real Pi follow-up and Trace

A separate manual check ran Pi 1.3.0 with openai-codex/gpt-6.1-sol, the existing repository .pi/extensions/chainloop-trace.ts, and the hash-verified official subagent example. It used a Trace-capable CLI built from this branch.

  • Real parent session: 01a18e94-0f8c-7007-a09c-25559a2d7ea5.
  • One real worker completed with exit code 0. Its read tool returned the first README heading, ## How it works.
  • The native completed result contains the child's conversation, tool result, model, and usage. The child reported input/output of 21272/418, cache read/write of 11776/0, cost of 0.0479016, and 2 turns. These are native observations, not a claim that the pinned matrix passed.
  • The focused subagent tests passed in the real parent session with -race -shuffle=on -count=1 across trace/pi, trace/e2e, and pkg/action.
  • Signed follow-up commit afad7548ff9e35733faf624bb29af818694d3785 documents that manual validation. The original implementation commit predates the recording and is not retrospectively claimed as traced.

The repository's chainloop/chainloop Trace destination is unchanged. The current CLI login is denied access to that project; the control-plane attempt returned PermissionDenied. The genuine native evidence is retained, but no uploaded session URL or attestation is claimed. The PR remains a draft.

Remaining validation

The pinned OpenRouter compatibility-matrix run is still blocked: the repository key returned 401 API key expired. The successful manual OAuth check is not a pass for that matrix.

CHAINLOOP_E2E_HARNESS=Pi \
CHAINLOOP_E2E_FEATURE=Subagents \
make -C app/cli test-trace-e2e

The capability remains Partial because support is limited to the pinned official result format.

Assisted-by: Vibhav's Pi