PR #2529 - harbor-framework/harbor

Summary

Lets a task ship prior agent context as an ATIF trajectory.json beside the instruction.md it belongs to. Harbor seeds it as the agent's session before the agent runs, so a task whose instruction refers to an earlier conversation can carry that conversation itself.

This is Option 3 ("New Special File") from the task-trajectory design doc, per the design review conclusion. It builds directly on the run-level trajectory loading in #2619 / #2628 / #2630: discovery resolves to the existing load_trajectory input, so conversion and seeding are reused unchanged — this PR adds where the file comes from, not how it is loaded.

my-task/
├── instruction.md
├── trajectory.json   ← prior context, picked up by location
├── task.toml
├── environment/
└── tests/

No new task.toml field. The convention extends to steps for free: steps/<name>/trajectory.json.

Design decisions

Task level is ATIF only. A native session file is tied to the agent that recorded it, so accepting one at task level would silently limit the task to that agent. Run-level --load-trajectory still accepts both.

--load-trajectory wins over a task's trajectory.json. The flag is an explicit statement about this run; the task file is a property of the task.

oracle and nop skip a task trajectory rather than being refused. Without this, a task shipping prior context was refused for every agent lacking ATIF support — and since oracle is the default agent, harbor run -p <task> failed outright, breaking the oracle pass task authors use to validate a task. The check sits in _resolve_load_trajectory beside the regrade skip, matching the existing oracle special-case a few lines below in the same file, so no agent class is touched.

Worth noting why nothing like this exists upstream: --load-trajectory and --resume-trajectory are opt-in, so a utility agent never meets one (_validate_resume_support likewise just raises on not SUPPORTS_RESUME). A task-declared trajectory is the first implicit case, which is what creates the need.

A run-level --load-trajectory with those agents still fails fast: that is an operator asking for something impossible, not a property of the task. Keying off the agent name means a custom conversationless agent passed as --agent module:Class (where config.agent.name is None) is not covered; a declared capability on BaseAgent would fix that, but it changes the agent contract for a case that does not exist yet, so it is deliberately left out.

No flag to skip a task's trajectory. An earlier revision of this PR added --ignore-task-trajectory; it is gone. The prior context is part of the task, like the instruction.md that refers to it, so running without it scores a different task — an ablation that reads as a real result but is 0 by construction. An agent that cannot load an ATIF trajectory therefore fails on such a task, exactly as an agent without native resume fails on --resume-trajectory. Because failures are scoped to the trial (see below), a mixed dataset still runs the rest. This PR adds no CLI flags and no config fields.

Only the first step of a multi-step task may declare one. Seeding happens once, before the first step (_step_loads(index) is first-step-only, and the agent is constructed once per trial). A later step declaring a trajectory.json therefore cannot be honored, so it fails the trial with the offending step named rather than starting a step without the context it asked for. Per-step re-seeding would mean re-seeding a live agent mid-trial; that is a bigger change to the agent contract and is deliberately left out.

Failures are deferred to _prepare(), not raised from Trial.create(). Trials are scheduled in a TaskGroup, so an exception during construction cancels every sibling trial and aborts the job. A run-level flag failing that way is fine — it is wrong for every trial — but a task-declared trajectory is per-task: one bad task in a 500-task dataset must not take the job down. Task-level problems are held and re-raised at the start of _prepare(), before any environment spend, and recorded as that trial's exception. Run-level --load-trajectory keeps its existing fail-fast behaviour. This is also what makes the "no opt-out flag" decision workable: an agent that cannot load one task's trajectory fails that trial, not the job.

Packaging is one line, deliberately. Packager.collect_files collects trajectory.json so a published task keeps its prior context — without it the feature would silently break on publish. Packaging the media a multimodal trajectory references is a separate concern and is not in this PR; it is noted as a known limitation in the docs and left to a follow-up.

Verified end to end

Docs and the example task live in the stacked #2744, so the run below uses the example from that branch. It instructs the agent to recall a filename and content that appear only in the trajectory. Real claude-code / claude-sonnet-4-5, local Docker:

Run	Reward
`harbor run -p examples/tasks/hello-task-trajectory -a claude-code`	1.000
same task, copied with `trajectory.json` deleted (control)	0.000

The seeded run resumed the session id from the task's trajectory.json (d7d4e19e-…) and its own reasoning reads "Looking back at the conversation history, I can see that I created a file called hello.txt"; it wrote hello.txt / Hello, world!. The control wrote unknown, the instruction's documented fallback for "no earlier conversation".

Scope

Source and tests only. Docs and the example task are split into the stacked #2744 so the mechanism can be reviewed on its own; the CHANGELOG.md entry stays here with the code it describes.

No CLI flags, no config fields. Two things were written for it and then deliberately removed to keep it reviewable: media packaging for multimodal trajectories (a references.py traversal module plus its share of the packager, ~490 lines), which will return as its own PR, and the --ignore-task-trajectory opt-out, which was cut on the reasoning above. Audio content parts are separately in #2605.

Test plan

tests/unit/trial/test_task_trajectory_convention.py — single-step root, first-step multi-step, root-ignored-for-multi-step, later-step refusal, flag precedence, ignore flag, regrade skip, malformed/invalid/missing-media deferral, unsupported-agent deferral vs. run-level fail-fast
tests/unit/publisher/test_packager_trajectory.py — root and step trajectory collection, no-trajectory tasks unchanged, content-hash sensitivity

Full unit suite: 5755 passed, 16 skipped. ruff check / ruff format clean; ty check at the same 7 pre-existing diagnostics as main (none in changed files).

Notes for review

_validate_load_trajectory_support() was refactored to read the resolved trajectory instead of config.agent.load_trajectory directly, and to attribute errors to whichever source supplied the file. Behaviour for the run-level flag is unchanged; its existing tests pass untouched apart from setting the new resolved attribute.
The resolved attributes are declared at class level on Trial so the many tests that build trials via object.__new__ keep working without edits.
A task whose trajectory references media that is not present fails when a trial seeds it, with the missing path named (this reuses TrajectoryValidator's existing image-path checking). Since referenced media is not packaged yet, multimodal task trajectories work locally but lose their media when published — called out in the docs with a warning callout.

History of this PR

This PR was originally [environment].atif_trajectory_path, then [agent].load_trajectory at task level. Both were the wrong shape, per review: the field was modelled in the wrong place, and the change carried more than the feature needed.

The branch has been force-pushed to the design-review conclusion (Option 3, the convention above), rebuilt on current main on top of the merged run-level work rather than duplicating it. The previous implementation is not lost — it remains at ec479d66 on this branch's reflog if anything from it is wanted.

Two things I had wrong in the earlier revisions and have corrected here: task level accepted .jsonl (it is now ATIF only), and [environment] is in fact defensible for a file baked into the image — it just is not the right home for context that belongs to the instruction.