Skip to content

docs(build-agent): annotate-trace is agent-callable on-platform via the annotate_trace op#15

Open
mmabrouk wants to merge 1 commit into
mainfrom
docs/annotate-trace-platform-op
Open

docs(build-agent): annotate-trace is agent-callable on-platform via the annotate_trace op#15
mmabrouk wants to merge 1 commit into
mainfrom
docs/annotate-trace-platform-op

Conversation

@mmabrouk

@mmabrouk mmabrouk commented Jul 6, 2026

Copy link
Copy Markdown
Member

What changed

references/annotate-trace.md (and a matching comment in
scripts/extras/annotate-trace.sh) claimed:

There is no agent-callable platform operation for this yet, so this is an
experimenter and demo tool. A self-reflecting agent cannot call it
autonomously today ... Do not build an agent whose instructions tell it to
annotate its own traces; that path does not exist yet.

That's now stale. Fixed to:

This script drives the public HTTP API from outside a run ... That said, an
agent built and run on the Agenta platform (with the default build kit)
can self-annotate at runtime: it has an annotate_trace platform op that
targets its own current trace and span automatically. The agent supplies
only references.evaluator.slug ... and data.outputs ...; a new slug
auto-creates a simple evaluator.

Why

The Agenta platform ships an annotate_trace op in the default agent build
kit (DEFAULT_BUILD_KIT_OPS) — it self-targets the run's own trace/span via
context binding, so an in-platform agent can grade itself without hitting
the public API from outside a run. annotate-trace.sh itself is unaffected:
it still drives the public HTTP API and stays an experimenter/demo tool for
out-of-run annotation, since there's no public-API equivalent bound to "your
own current trace."

…he annotate_trace op

The skill claimed no agent-callable platform operation for annotations
existed. Agenta now ships an annotate_trace platform op in the default
build kit that self-targets the current run's trace/span, so on-platform
agents can grade themselves at runtime. Clarify that the shell script
remains a human/out-of-run tool for the public HTTP API, while pointing
agent authors at the platform op for self-annotation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant