feat: add harbor benchmark adapter

benchmarks/harbor/README.md

@@ -0,0 +1,56 @@
+# Harbor adapter
+
+[English](README.md) | [简体中文](README.zh-CN.md)
+
+This isolated workspace contains the nanoPyCodeAgent adapter for Terminal-Bench
+and other benchmarks run by Harbor. It is development infrastructure, not part
+of the end-user `nanoPyCodeAgent` package. Harbor is pinned to 0.21.0 in this
+workspace's lockfile.
+
+## Run a benchmark
+
+Set the same `ANTHROPIC_*` environment variables used by the CLI. For a
+third-party or proxy endpoint, set all three explicitly:
+
+```bash
+export ANTHROPIC_API_KEY="..."
+export ANTHROPIC_BASE_URL="https://gateway.example"
+export ANTHROPIC_MODEL="provider-model-name"
+```
+
+From the repository root, run Harbor through this workspace and pin the agent
+installed in the task container:
+
+```bash
+uv run --project benchmarks/harbor harbor run \
+  --task terminal-bench/openssl-selfsigned-cert \
+  --agent harbor_adapter:NanoPyCodeAgent \
+  --agent-kwarg git_ref=<commit-sha> \
+  --model anthropic/claude-sonnet-4-6 \
+  --env docker \
+  --n-concurrent 1 \
+  --n-attempts 1
+```
+
+Use `--agent-kwarg version=<released-version>` instead of `git_ref` to install a
+published PyPI release. The two pins are mutually exclusive. If neither is
+provided, the adapter installs the latest published release. For reproducible
+benchmark results, always provide one of them.
+
+The adapter sends the instruction through stdin, runs the agent in the task
+container's current directory, and saves combined stdout/stderr to
+`/logs/agent/nanopycodeagent.txt`. It uses the CLI's 50-turn default; override
+that with `--agent-kwarg max_turns=20`.
+
+`ANTHROPIC_MODEL` takes precedence over Harbor's `--model`. When it is absent,
+the adapter strips the first provider prefix from `--model`. Harbor-native
+provider credentials and configured base URLs are also normalized to the
+`ANTHROPIC_*` variables expected by nanoPyCodeAgent's SDK.
+
+## Test the adapter
+
+```bash
+uv run --project benchmarks/harbor pytest \
+  -c benchmarks/harbor/pyproject.toml \
+  benchmarks/harbor/tests
+```

benchmarks/harbor/README.zh-CN.md

@@ -0,0 +1,53 @@
+# Harbor adapter
+
+[English](README.md) | [简体中文](README.zh-CN.md)
+
+这个独立 workspace 包含 nanoPyCodeAgent 面向 Terminal-Bench 及其他 Harbor
+评测任务的 adapter。它属于开发基础设施，不是面向普通用户的
+`nanoPyCodeAgent` 软件包的一部分。该 workspace 的 lockfile 将 Harbor 固定在
+0.21.0。
+
+## 运行 benchmark
+
+设置与 CLI 相同的 `ANTHROPIC_*` 环境变量。使用第三方或代理 endpoint 时，显式
+设置以下三个变量：
+
+```bash
+export ANTHROPIC_API_KEY="..."
+export ANTHROPIC_BASE_URL="https://gateway.example"
+export ANTHROPIC_MODEL="provider-model-name"
+```
+
+从仓库根目录通过这个 workspace 运行 Harbor，并固定安装到 task 容器中的 agent：
+
+```bash
+uv run --project benchmarks/harbor harbor run \
+  --task terminal-bench/openssl-selfsigned-cert \
+  --agent harbor_adapter:NanoPyCodeAgent \
+  --agent-kwarg git_ref=<commit-sha> \
+  --model anthropic/claude-sonnet-4-6 \
+  --env docker \
+  --n-concurrent 1 \
+  --n-attempts 1
+```
+
+如需安装已经发布到 PyPI 的版本，请使用
+`--agent-kwarg version=<released-version>` 代替 `git_ref`。这两个版本参数互斥；
+如果都未提供，adapter 会安装最新发布版本。为了让 benchmark 结果可复现，请始终
+提供其中一个参数。
+
+adapter 通过 stdin 发送任务指令，在 task 容器的当前目录中运行 agent，并把合并后
+的 stdout/stderr 保存到 `/logs/agent/nanopycodeagent.txt`。它默认沿用 CLI 的 50
+轮限制；可以通过 `--agent-kwarg max_turns=20` 覆盖此设置。
+
+`ANTHROPIC_MODEL` 的优先级高于 Harbor 的 `--model`。未设置该环境变量时，adapter
+会移除 `--model` 中的第一个 provider 前缀。Harbor 原生的 provider 凭证和已经配置
+的 base URL 也会转换为 nanoPyCodeAgent SDK 所需的 `ANTHROPIC_*` 变量。
+
+## 测试 adapter
+
+```bash
+uv run --project benchmarks/harbor pytest \
+  -c benchmarks/harbor/pyproject.toml \
+  benchmarks/harbor/tests
+```

benchmarks/harbor/pyproject.toml

@@ -0,0 +1,23 @@
+[project]
+name = "nanopycodeagent-harbor-adapter"
+version = "0.1.0"
+description = "Repository-local Harbor adapter for nanoPyCodeAgent benchmarks."
+requires-python = ">=3.13"
+dependencies = [
+    "harbor==0.21.0",
+]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/harbor_adapter"]
+
+[dependency-groups]
+dev = [
+    "pytest>=9.1.1",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

benchmarks/harbor/src/harbor_adapter/__init__.py

@@ -0,0 +1,5 @@
+"""Public import for the repository-local nanoPyCodeAgent Harbor adapter."""
+
+from .adapter import NanoPyCodeAgent
+
+__all__ = ["NanoPyCodeAgent"]

benchmarks/harbor/src/harbor_adapter/adapter.py

@@ -0,0 +1,137 @@
+"""Harbor adapter for running nanoPyCodeAgent in benchmark containers."""
+
+import re
+import shlex
+import uuid
+from typing import override
+
+from harbor.agents.installed.base import (
+    BaseInstalledAgent,
+    CliFlag,
+    with_prompt_template,
+)
+from harbor.agents.model_connection import ModelConnectionSpec
+from harbor.environments.base import BaseEnvironment
+from harbor.models.agent.context import AgentContext
+
+_DEFAULT_MAX_TURNS = 50
+_PACKAGE_NAME = "nanoPyCodeAgent"
+_REPOSITORY_URL = "https://github.com/minixalpha/nanoPyCodeAgent.git"
+_UV_VERSION = "0.9.11"
+_PATH_SETUP = 'export PATH="$HOME/.local/bin:$PATH"; '
+
+
+class NanoPyCodeAgent(BaseInstalledAgent):
+    """Install and run nanoPyCodeAgent inside a Harbor task environment."""
+
+    MODEL_CONNECTION = ModelConnectionSpec(
+        api_key_envs=("ANTHROPIC_API_KEY",),
+        base_url_envs=("ANTHROPIC_BASE_URL",),
+    )
+    CLI_FLAGS = [
+        CliFlag(
+            "max_turns",
+            cli="--max-turns",
+            type="int",
+            default=_DEFAULT_MAX_TURNS,
+        )
+    ]
+
+    def __init__(self, *args, git_ref: str | None = None, **kwargs):
+        if git_ref is not None:
+            git_ref = git_ref.strip()
+            if not git_ref:
+                raise ValueError("git_ref must not be blank")
+        if git_ref is not None and kwargs.get("version") is not None:
+            raise ValueError("version and git_ref are mutually exclusive")
+        self._git_ref = git_ref
+        super().__init__(*args, **kwargs)
+
+    @staticmethod
+    @override
+    def name() -> str:
+        return "nanopycodeagent"
+
+    @override
+    def get_version_command(self) -> str:
+        return f"{_PATH_SETUP}nanoPyCodeAgent --version"
+
+    @override
+    def parse_version(self, stdout: str) -> str:
+        lines = [line.strip() for line in stdout.splitlines() if line.strip()]
+        if not lines:
+            return ""
+        match = re.fullmatch(r"nanoPyCodeAgent\s+(\S+)", lines[-1])
+        return match.group(1) if match else lines[-1]
+
+    def _install_target(self) -> str:
+        if self._git_ref is not None:
+            return f"git+{_REPOSITORY_URL}@{self._git_ref}"
+        if self._version is not None:
+            return f"{_PACKAGE_NAME}=={self._version}"
+        return _PACKAGE_NAME
+
+    @override
+    async def install(self, environment: BaseEnvironment) -> None:
+        await self.ensure_system_dependencies(
+            environment,
+            ("curl", "bash", "git", "python3", "ca_certificates"),
+        )
+        install_target = shlex.quote(self._install_target())
+        await self.exec_as_agent(
+            environment,
+            command=(
+                "set -euo pipefail; "
+                "if ! command -v uv >/dev/null 2>&1; then "
+                f"curl -LsSf https://astral.sh/uv/{_UV_VERSION}/install.sh | sh; "
+                "fi; "
+                f"{_PATH_SETUP}"
+                f"uv tool install --force {install_target}; "
+                "nanoPyCodeAgent --version"
+            ),
+        )
+
+    def _runtime_env(self) -> dict[str, str]:
+        model_connection = self.model_connection
+        env: dict[str, str] = {}
+        if model_connection.api_key:
+            env["ANTHROPIC_API_KEY"] = model_connection.api_key
+        if model_connection.configured_base_url:
+            env["ANTHROPIC_BASE_URL"] = model_connection.configured_base_url
+
+        model = (self._get_env("ANTHROPIC_MODEL") or "").strip()
+        if not model and self.model_name:
+            model = self.model_name.split("/", 1)[-1]
+        if model:
+            env["ANTHROPIC_MODEL"] = model
+        return env
+
+    @override
+    @with_prompt_template
+    async def run(
+        self,
+        instruction: str,
+        environment: BaseEnvironment,
+        context: AgentContext,
+    ) -> None:
+        instruction_shell_var = (
+            f"harbor_nanopycodeagent_instruction_{uuid.uuid4().hex}"
+        )
+        instruction_env_var = instruction_shell_var.upper()
+        env = {
+            **self._runtime_env(),
+            instruction_env_var: instruction,
+        }
+        cli_flags = self.build_cli_flags()
+        await self.exec_as_agent(
+            environment,
+            command=(
+                f"{_PATH_SETUP}"
+                f'{instruction_shell_var}="${instruction_env_var}"; '
+                f"unset {instruction_env_var}; "
+                f'printf "%s" "${instruction_shell_var}" | '
+                f"nanoPyCodeAgent {cli_flags} "
+                "2>&1 | tee /logs/agent/nanopycodeagent.txt"
+            ),
+            env=env,
+        )