Keep push-back feedback out of first-run prompts

Merged
YO
3d ago
+131-5

Summary

Closes #3614.

Feedback on one step reached the prompts of harness.run and chat.completion steps that had never run. In the plan-first workflow, feedback at approve_plan went into every attempt of implement, and a real Codex agent acted on it:

implement the change

## Push-back Context

Push-back iteration: 1

Reviewer feedback:
- feedback: Add a final section titled PUSHBACKOK.

Reading the code turned up a worse case than the issue reports. When pushBackIteration is set, chat.completion sends only its system messages and the feedback, because it assumes the conversation already holds the prompt. So a chat step first running after a push-back lost its prompt entirely.

The decision

The issue listed three options. I first tried not stamping steps that never ran, and TestApprovalPushBackExposesHistoricalFeedbackEnvAcrossRewoundScope failed: it asserts that such a step does receive FEEDBACK and DAG_PUSHBACK. A later step reading the reviewer's notes is a real use, so that stays.

This PR separates the two things:

Step that ran beforeStep that never ran
Environment (FEEDBACK, DAG_PUSHBACK)yesyes, unchanged
Written into the agent's requestyesno

Changes

  • setupExecutorSideChannels passes push-back context to executors only when PushBackPreviousStdout is set, which marks a step that ran.
  • ApplyPushBack keeps that path when a step is reset again before it reruns. Otherwise a second reset would make a step that ran look as if it never had.
  • Spec 031 states the rule.

Verification

  • TestPushBackFeedbackSkipsPromptOfHarnessStepThatNeverRan runs through the binary and records the prompts a stub agent receives. Before the fix it failed with the ## Push-back Context block on publish.
  • TestStepExecutorKeepsPushBackFromStepsThatNeverRan and TestApplyPushBackKeepsPreviousStdoutAcrossResets, which also failed before the fix.
  • Locally: go test ./internal/runtime/ ./internal/dagrun/ ./internal/humantask/, the human-task conformance suite, and the approval and human-task integration tests.

The Outer Loops guide describes the old behaviour, so I'll update it once this merges.


Summary by cubic

Stops push-back feedback from rewriting the prompts of steps that never ran.

A push-back stamped every rewound step, but executors treated the stamp as a request to revise earlier work. A harness.run step that had never run got another step's feedback appended to its prompt and a real agent acted on it; a chat.completion step lost its prompt entirely. Steps now receive the feedback in their environment as before, but only steps that produced stdout get it written into the request they send.

  • Executors receive push-back context only when PushBackPreviousStdout is set.
  • ApplyPushBack preserves PushBackPreviousStdout across resets so a step that ran stays marked.
  • Spec 031 states the rule.

Summary by CodeRabbit

  • Bug Fixes

    • Push-back feedback is now included in prompts only for steps that ran before the push-back. Steps without a prior run keep their original prompts and receive feedback through their environment.
    • Previous output is preserved across repeated push-backs, so later resets retain the correct context.
  • Documentation

    • Clarified how push-back feedback is delivered to workflow steps.