docs: define trajectory architecture and implementation plan

CONTEXT.md

@@ -20,6 +20,22 @@ _Avoid_: Trajectory, session export
 A caller-facing Run Output that exposes a stable, selected sequence of Agent Run events as they occur.
 _Avoid_: Internal event bus, execution trace
 
+**Source Record**:
+An agent-specific record observed at a CLI, SDK, or protocol boundary and consumed by an adapter; it may already be a projection of the agent's internal state.
+_Avoid_: Native event, journal entry
+
+**Native Event**:
+An agent-independent runtime fact accepted by the core directly or normalized from a Source Record.
+_Avoid_: Source record, public event record, trajectory step
+
+**Journal Entry**:
+A Native Event together with the identity, ordering, and recording metadata required for durable history.
+_Avoid_: Persistence envelope, trajectory step
+
+**Event Journal**:
+A durable append-only sequence of Journal Entries for an Agent Run from which derived artifacts can be rebuilt.
+_Avoid_: Trajectory, session, public event stream
+
 **Partial Content**:
 An unfinished representation of a message, reasoning block, or tool input that is exposed before the logical content is complete.
 _Avoid_: Final message, completed event
@@ -40,6 +56,10 @@ _Avoid_: Run output, trajectory
 A task-scoped record of observations, actions, results, and outcome prepared for evaluation or offline analysis.
 _Avoid_: Execution trace, session, transcript
 
+**ATIF Trajectory**:
+A Trajectory represented by the Agent Trajectory Interchange Format and exposed as the runtime's public trajectory contract.
+_Avoid_: Event journal, native trajectory
+
 **Session**:
 Durable agent state that can span multiple runs and supports continuing or branching prior work.
 _Avoid_: Agent run, trajectory, transcript

docs/dev_notes/en/0.8.x.md

@@ -105,7 +105,7 @@ The next work therefore has three independent parts:
 
 - Add `httpx` as a direct dependency so container installation does not rely on a transitive relationship.
 - Add the Harbor adapter to the project, fixing and versioning its installation command, headless CLI invocation, environment-variable forwarding, and version probing so future benchmark runs can reuse it instead of rewriting a temporary adapter each time.
-- Design canonical internal events, `stream-json`, and a trajectory writer so the adapter can convert the native trajectory to ATIF and reliably populate token, cost, and step statistics. These changes improve the installability, reproducibility, and observability of the working benchmark path.
+- Design canonical internal events and a trajectory writer so the agent produces ATIF directly and the adapter can read it to populate token, cost, and step statistics reliably; implement `stream-json` later as separate run output. These changes improve the installability, reproducibility, and observability of the working benchmark path.
 
 ### Adding the Harbor adapter to the project
 
@@ -119,4 +119,56 @@ The configuration boundary is fixed as well. `ANTHROPIC_API_KEY`, `ANTHROPIC_BAS
 
 The formal adapter's container boundary is first covered by five contract tests built on Harbor 0.21.0's real base class: published-release pins, Git revision pins, mutually exclusive source arguments, instruction piping and logging, and both environment-variable mappings. The same task was then rerun once through the formal adapter in a real Docker environment. Harbor completed the full path — starting the container, installing the specified Git revision, injecting model configuration, invoking the agent, collecting logs, and running the official verifier — with zero agent or verifier infrastructure exceptions, demonstrating that the adapter itself works end to end. The task received a reward of 0, with 5 of 6 official verifier tests passing. The only failure was that the agent's `check_cert.py` depended on `cryptography`, which was unavailable in the verifier's Python environment. This was a portability defect in the task solution, not an adapter infrastructure failure.
 
-In addition, the Harbor CLI version command and dynamic import in the isolated workspace verify the host-side integration, while a build of the root project confirms that its wheel contains neither the adapter nor a Harbor dependency. The first two follow-up items — owning `httpx` directly and adding the formal adapter — are therefore complete. The remaining work is to design canonical internal events, `stream-json`, and a trajectory writer so that the adapter can convert its native trajectory to ATIF and reliably populate token, cost, and step statistics.
+In addition, the Harbor CLI version command and dynamic import in the isolated workspace verify the host-side integration, while a build of the root project confirms that its wheel contains neither the adapter nor a Harbor dependency. The first two follow-up items — owning `httpx` directly and adding the formal adapter — are therefore complete. The remaining trajectory work is to design canonical internal events and an Event Journal, have the agent produce ATIF directly, and let the adapter read it to populate token, cost, and step statistics reliably. `stream-json` is a separate form of run output and will be implemented in a later PR.
+
+### Implementing Trajectory
+
+Based on the research ([the boundary between agent output and trajectory](../../research/en/agent_output_and_trajectory.md), [mapping agent events to ATIF](../../research/en/agent_events_to_atif_examples.md), [OpenRouter cost accounting](../../research/en/openrouter_cost_accounting.md), and [the unified OpenRouter model protocol](../../research/en/openrouter_unified_protocol.md)), the implementation path has converged on a replayable internal **Event Journal**, while the public `--trajectory` option produces only **ATIF-v1.7**. `stream-json` remains a separate form of run output that shares the same runtime facts with trajectory; it and the corresponding `--output-format` CLI interface are outside this trajectory implementation and will be delivered in a later PR.
+
+One completed `read` tool execution illustrates the distinction between a `Native Event` and a `Journal Entry`.
+
+The agent loop first produces a Native Event that describes only what happened:
+
+```json
+{
+  "type": "tool.completed",
+  "payload": {
+    "tool_call_id": "call-read-1",
+    "tool_name": "read",
+    "result": "file contents",
+    "is_error": false,
+    "duration_ms": 330
+  }
+}
+```
+
+After receiving it, the journal writer adds the identity, order, and record time needed for persistence, producing a Journal Entry:
+
+```json
+{
+  "schema_version": 1,
+  "run_id": "run-123",
+  "seq": 17,
+  "recorded_at": "2026-08-23T08:00:01.420Z",
+  "type": "tool.completed",
+  "payload": {
+    "tool_call_id": "call-read-1",
+    "tool_name": "read",
+    "result": "file contents",
+    "is_error": false,
+    "duration_ms": 330
+  }
+}
+```
+
+These describe the same occurrence rather than two events. A `Native Event` is the runtime fact produced by core; a `Journal Entry` is the persistable record of that fact after it enters the Event Journal, additionally answering which run it belongs to, where it appears in the sequence, and when it was recorded. The ATIF projector consumes Journal Entries ordered by `seq` and folds multiple facts into trajectory steps.
+
+The implementation order is:
+
+1. **Define runtime facts.** Establish a versioned contract for Native Events and Journal Entries. The first Native Event version covers `run.started`, `user.message`, `model.started/completed`, `tool.started/completed`, and `run.completed/failed`, carrying message and tool-call IDs, reliable `source_timestamp` values, and precise durations. Journal Entries then uniformly add `schema_version`, `run_id`, a strictly increasing `seq`, and UTC `recorded_at`.
+2. **Make the agent loop emit facts only.** Emit events around model calls, tool execution, and run finalization. `model.completed` retains the complete response, actual model, stop reason, token and cache usage, provider response ID, and OpenRouter generation ID. Project the existing text output from those facts while preserving current user-visible behavior, leaving one shared data source for a future `stream-json` implementation.
+3. **Append to the internal Event Journal.** The journal writer wraps every event in a Journal Entry and appends it incrementally as JSONL under restricted file permissions, preserving the last complete fact even if the process is interrupted. It also records truncation metadata for large output and treats the log explicitly as sensitive. The journal is an internal reconstruction source, is not exposed through `--trajectory`, and does not define ATIF steps.
+4. **Complete usage and actual cost.** Record `usage.cost` directly when present. On the current OpenRouter Messages path, preserve `X-Generation-Id` and use bounded retries during run finalization to query the generation's `total_cost`, then append `model.cost_resolved`. Query failure does not change the task result; unknown cost is not written as `0`, and summaries explicitly indicate partial data.
+5. **Implement a one-way ATIF projector.** Replay the Event Journal, mapping `user.message` to a user step and folding one model call plus its tool calls and results into an agent step. Map timestamps, tokens, cache, cost, terminal state, and `final_metrics`; only values with reliable sources and complete attribution enter standard fields, with other information under `extra`. At each checkpoint or run end, update the complete ATIF snapshot through a temporary file and atomic rename.
+6. **Connect the CLI and Harbor.** Add an independent `--trajectory PATH`, preserve stdout's current text behavior, and reject `-` to avoid competing for stdout. The Harbor adapter only passes the log path, declares and reads ATIF, and then populates steps, tokens, and cost; it no longer converts a native trajectory. `--output-format` and `stream-json` remain for a later independent PR.
+7. **Validate in layers.** Start with unit tests for event order, ID associations, failed tools, pending and resolved cost, Journal replay, and the ATIF schema. Then test stdout independence, atomic CLI writes, and interruption recovery. Finally, use Harbor contract tests and one real trial to confirm that the trajectory is collected and that step, token, and cost statistics are populated.

docs/dev_notes/zh-CN/0.8.x.md

@@ -105,7 +105,7 @@ harbor run \
 
 - 把 `httpx` 补成直接依赖，保证容器安装不依赖传递关系
 - 把 Harbor adapter 正式纳入项目，固定并版本化安装命令、headless CLI 调用、环境变量传递和版本探测逻辑，使后续 benchmark 可以直接复用，不必每次重新编写临时适配器
-- 设计内部标准事件、`stream-json` 和 trajectory writer，让这个 adapter 可以把 native trajectory 转成 ATIF，并可靠回填 token、cost 和步骤统计。这些属于对已跑通链路的可安装性、可复现性与可观测性增强。
+- 设计内部标准事件和 trajectory writer，让 agent 直接产出 ATIF，再由 adapter 读取并可靠回填 token、cost 和步骤统计；`stream-json` 作为独立的 run output 后续实现。这些属于对已跑通链路的可安装性、可复现性与可观测性增强。
 
 ### Harbor adapter 正式纳入项目
 
@@ -119,4 +119,56 @@ harbor run \
 
 正式 adapter 的容器边界先由五个基于 Harbor 0.21.0 真实基类的契约测试覆盖，分别验证发布版 pin、Git revision pin、互斥参数、instruction 管道与日志、两种环境变量映射；随后又通过正式 adapter 在真实 Docker 环境中重跑了一次同一道题。Harbor 完成了启动容器、安装指定 Git revision、注入模型配置、调用 agent、收集日志和执行官方 verifier 的完整链路，agent / verifier 基础设施异常为 0，说明 adapter 本身已经端到端接通。这次任务 reward 为 0，官方 verifier 通过 5/6 项；唯一失败是 agent 编写的 `check_cert.py` 依赖 `cryptography`，而 verifier 的 Python 环境中没有该依赖。这属于任务解答的可移植性问题，不是 adapter 基础设施故障。
 
-此外，独立 workspace 中 Harbor CLI 的版本命令和动态导入检查了宿主侧集成，根项目 wheel 的构建则确认其中不包含 adapter 或 Harbor 依赖。原先三件后续工作中的前两件——`httpx` 直接依赖与正式 adapter——至此完成。剩余工作是设计内部标准事件、`stream-json` 和 trajectory writer，让 adapter 能把 native trajectory 转成 ATIF，并可靠回填 token、cost 和步骤统计。
+此外，独立 workspace 中 Harbor CLI 的版本命令和动态导入检查了宿主侧集成，根项目 wheel 的构建则确认其中不包含 adapter 或 Harbor 依赖。原先三件后续工作中的前两件——`httpx` 直接依赖与正式 adapter——至此完成。剩余的 trajectory 工作是设计内部标准事件与 Event Journal，由 agent 直接产出 ATIF，再让 adapter 读取并可靠回填 token、cost 和步骤统计。`stream-json` 属于独立的 run output，将在后续 PR 中实现。
+
+### 实现 Trajectory
+
+根据调研（[agent output 与 trajectory 边界](../../research/zh-CN/agent_output_and_trajectory.md)、[agent 事件到 ATIF 的映射](../../research/zh-CN/agent_events_to_atif_examples.md)、[OpenRouter cost 记账](../../research/zh-CN/openrouter_cost_accounting.md)、[OpenRouter 统一模型协议](../../research/zh-CN/openrouter_unified_protocol.md)），实现路线收敛为：内部保留可重放的 **Event Journal**，对外的 `--trajectory` 只产出 **ATIF-v1.7**。`stream-json` 仍是独立的 run output，只与 trajectory 复用同一组运行事实；它和对应的 `--output-format` CLI 接口不在本轮 trajectory 实现范围内，后续单开 PR。
+
+先用一次 `read` 工具执行完成来说明 `Native Event` 与 `Journal Entry` 的概念。
+
+agent loop 先产生一条只描述“发生了什么”的 `Native Event`：
+
+```json
+{
+  "type": "tool.completed",
+  "payload": {
+    "tool_call_id": "call-read-1",
+    "tool_name": "read",
+    "result": "file contents",
+    "is_error": false,
+    "duration_ms": 330
+  }
+}
+```
+
+journal writer 接收它以后，补上持久化所需的身份、顺序和记录时间，形成一条 `Journal Entry`：
+
+```json
+{
+  "schema_version": 1,
+  "run_id": "run-123",
+  "seq": 17,
+  "recorded_at": "2026-08-23T08:00:01.420Z",
+  "type": "tool.completed",
+  "payload": {
+    "tool_call_id": "call-read-1",
+    "tool_name": "read",
+    "result": "file contents",
+    "is_error": false,
+    "duration_ms": 330
+  }
+}
+```
+
+两者描述的是同一件事，不是两个事件：`Native Event` 是 core 产生的运行事实；`Journal Entry` 是这条事实进入 Event Journal 后的可持久化记录，额外回答“属于哪次 run、排在第几、何时被记录”。ATIF projector 消费的是按 `seq` 排列的 Journal Entry，再把多条事实折叠成 trajectory step。
+
+整体按以下顺序实现：
+
+1. **定义运行事实。** 建立 `Native Event` 与 `Journal Entry` 的版本化契约。Native Event 第一版覆盖 `run.started`、`user.message`、`model.started/completed`、`tool.started/completed`、`run.completed/failed`，并携带 message/tool call ID、可信的 `source_timestamp` 和精确 duration；Journal Entry 再统一添加 `schema_version`、`run_id`、严格递增的 `seq` 和 UTC `recorded_at`。
+2. **让 agent loop 只产生事实。** 在模型调用、工具执行和 run 收尾处发出事件；`model.completed` 保留完整回复、实际 model、stop reason、token/cache usage、provider response ID 和 OpenRouter generation ID。现有文本输出改为这些事实的投影，保持当前用户可见行为不变，也为后续 `stream-json` 留出同一数据源。
+3. **追加写入内部 Event Journal。** journal writer 把每个事件包装成 Journal Entry，以受限文件权限和 JSONL 增量落盘，使进程中断后仍保留最后一条完整事实；同时为大输出保留截断元数据，并把日志明确视为敏感文件。它是内部重建来源，不由 `--trajectory` 暴露，也不定义 ATIF step。
+4. **补齐 usage 与真实 cost。** 有 `usage.cost` 时直接记录；当前 OpenRouter Messages 路径则保存 `X-Generation-Id`，在 run 收尾时用有界重试查询 generation 的 `total_cost`，再追加 `model.cost_resolved`。查询失败不改变任务结果，未知 cost 不写成 `0`，汇总时明确标记 partial。
+5. **实现单向 ATIF projector。** 重放 Event Journal，将 `user.message` 映射为 user step，将一次模型调用及其 tool calls/results 折叠成 agent step，并映射 timestamp、token、cache、cost、终态和 `final_metrics`；只有来源可靠、归因完整的值才进入标准字段，其余信息放入 `extra`。每次 checkpoint 或 run 结束时通过临时文件加原子 rename 更新完整 ATIF 快照。
+6. **接通 CLI 与 Harbor。** 增加独立的 `--trajectory PATH`，保持 stdout 当前的文本行为，并拒绝 `-` 与 stdout 争用；Harbor adapter 只负责传入日志路径、声明并读取 ATIF，再回填 steps、tokens 和 cost，不再做 native trajectory 转换。`--output-format` 与 `stream-json` 留给后续独立 PR。
+7. **分层验收。** 先用单元测试覆盖事件顺序、ID 关联、异常工具、pending/resolved cost、Journal 重放和 ATIF schema；再验证 CLI 的 stdout 独立性、原子写入和中断恢复；最后用 Harbor 契约测试与一次真实 trial 确认 trajectory 被采集，步骤与 token/cost 统计能够回填。