Skip to content

docs: document end-to-end benchmark validation - #24

Merged
minixalpha merged 5 commits into
mainfrom
docs/update-0.8-dev-notes
Aug 23, 2026
Merged

docs: document end-to-end benchmark validation#24
minixalpha merged 5 commits into
mainfrom
docs/update-0.8-dev-notes

Conversation

@minixalpha

@minixalpha minixalpha commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Summary

  • Document the first end-to-end Terminal-Bench 2.1 validation of the headless CLI, including the Harbor adapter and Docker execution path.
  • Record the verified openssl-selfsigned-cert outcome: reward 1.0, 6/6 verifier tests passed, and no agent or verifier exceptions.
  • Explain the direct httpx dependency issue and the distinction between text-log tool counts and a structured benchmark trajectory.
  • Record the next steps: declare httpx directly, add a reusable Harbor adapter to the project, and build canonical events plus stream-json and trajectory output.
  • Regenerate the complete English 0.8.x development notes from the updated Chinese source.

Validation

  • Cross-checked the documented reward, verifier results, execution time, and tool-call counts against the saved Harbor artifacts.
  • Confirmed the Chinese and English development notes have matching section structure.
  • Confirmed the generated English development notes contain no untranslated Chinese text.
  • git diff --check main...HEAD

Scope

Documentation only; no runtime behavior changes.

@minixalpha minixalpha changed the title docs/update 0.8 dev notes docs: document end-to-end benchmark validation Aug 23, 2026
@minixalpha
minixalpha merged commit eb651e1 into main Aug 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant