docs: document and stabilize token/cost field semantics

[zhongxuanwang-nv]

Overview

Document and stabilize NeMo Relay's LLM token and cost field semantics. This freezes the current behavior as a documented contract and locks it with characterization tests. There is no runtime behavior change. The branch is merged with current main, so the docs and tests reflect #304 (prefer normalized exporter fallbacks) and #305 (shared nemo_relay_types DTO crate).

• I confirm this contribution is my own work, or I have the right to submit it under this project's license.
• I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

Adds a canonical Token and Cost Field Semantics section to docs/integrate-into-frameworks/provider-response-codecs.mdx:

• Usage and CostEstimate field reference (names, units, optionality; optional fields can also be codec-computed or pricing-synthesized).
• Per-provider token normalization table (OpenAI Chat / OpenAI Responses / Anthropic → Usage).
• Granularity: per-call values; an exported call yields a user start step (no metrics) and an agent end step that carries metrics; final_metrics is the only aggregate (a per-trajectory sum of present step metrics, excluding embedded subagent trajectories).
• Exporter field-mapping table across ATOF / ATIF / OpenInference / OpenTelemetry: OpenTelemetry is cost-only and currency-aware; ATIF and OpenInference are USD-only; ATIF sources metrics from the codec-normalized usage with raw-payload fallback.
• A Stability subsection (stable as of ATOF 0.1 / ATIF-v1.7 / pricing catalog version: 1). The additive-compatibility promise is scoped to the serialized JSON/ATOF shapes; the Rust Usage/CostEstimate structs and CostSource enum are exhaustive, so adding a field/variant is source-breaking for Rust consumers.

Short field pointers + back-links were added to the OpenTelemetry, OpenInference, and ATIF exporter pages. Cost policy is stated once on the canonical page, per the runtime-contract docs convention.

Two characterization tests lock the freeze:

• The OpenTelemetry LLM span emits cost only — driven end-to-end through OtelEventProcessor, asserting the finished span carries nemo_relay.llm.cost.{total,currency} and no token-count attribute in any namespace (token-containing or gen_ai.* keys fail).
• OpenAIChatCodec drops completion_tokens_details, and Usage ignores unmodeled provider subfields (forward-compat: no serde catch-all).

Existing tests already cover the remaining projections, per-provider mapping, reasoning-tokens-in-api_specific, and the USD-only/currency-aware cost behavior.

Testing: targeted cargo test on the merged tree (the new/changed tests pass; perturbing the OTel exporter with a nemo_relay.prompt_tokens attribute makes the span test fail as intended, then reverted), just docs-linkcheck (0 errors), and pre-commit (SPDX, markdown linkcheck, cargo fmt/clippy/check) all pass.

Where should the reviewer start?

docs/integrate-into-frameworks/provider-response-codecs.mdx — the Token and Cost Field Semantics section (the exporter field-mapping table and the Stability subsection are the core contract). Then crates/core/tests/unit/observability/otel_tests.rs::llm_end_emits_cost_only_no_token_or_gen_ai_attributes.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• N/A

Summary by CodeRabbit

• Bug Fixes

  • Improved handling of provider usage data so only supported token fields are kept, while extra or future fields are safely ignored.
  • Refined end-span observability output to emit cost metrics without unexpected token or provider-specific attributes.

• Tests

  • Added coverage for response decoding, usage field handling, and emitted observability attributes.

• Documentation

  • Expanded integration and observability docs with clearer token and cost field semantics, including mapping details and expected output behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 10424c39-a36b-4fee-9fc0-40c162cfdd94

📥 Commits

Reviewing files that changed from the base of the PR and between 3eb248e and 2ec7e1c.

📒 Files selected for processing (7)

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs
• docs/integrate-into-frameworks/provider-response-codecs.mdx
• docs/observability-plugin/atif.mdx
• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx

📜 Recent review details

⏰ Context from checks skipped due to timeout. (9)

• GitHub Check: Apply PR labels
• GitHub Check: Rust / Test (windows-arm64)
• GitHub Check: Rust / Test (macos-arm64)
• GitHub Check: Rust / Test (windows-amd64)
• GitHub Check: Rust / Test (linux-amd64)
• GitHub Check: Rust / Test (linux-arm64)
• GitHub Check: Check / Run
• GitHub Check: Preview docs
• GitHub Check: Preview docs

🧰 Additional context used

📓 Path-based instructions (25)

{docs/**,README.md,CONTRIBUTING.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

{docs/**,README.md,CONTRIBUTING.md}: For docs-only changes, run targeted checks only if commands, package names, or examples changed. Use just docs for docs-site builds and just docs-linkcheck when links changed
Run docs site build with just docs

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

{docs/**,README.md,CONTRIBUTING.md,**/*.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Run docs link validation with just docs-linkcheck when links change

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

{docs/**,README.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Verify README and docs entry points still match current package names and paths for large or public-facing changes

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

{docs/**,examples/**,README.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Verify examples still run with documented commands for large or public-facing changes

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

{docs/**,README.md,**/Cargo.toml,**/package.json,**/*.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Ensure renamed public surfaces are reflected consistently in manifests and docs for large or public-facing changes

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

**/*.{md,mdx,py,sh,yaml,yml,toml,json}

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

Keep package names, repo references, and build commands current

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

**/*.mdx

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

In MDX files, top-of-file comments must use JSX comment delimiters: {/* to open and */} to close. Do not use HTML comments for MDX SPDX headers.

MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

**/*.{html,md,mdx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Include SPDX license header in HTML and Markdown files using HTML comment syntax

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

docs/**/*.{md,mdx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Update embedded documentation snippets, patch docs, and binding-support notes if examples or supported bindings changed

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

docs/**

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Run just docs or ./scripts/build-docs.sh html to regenerate ignored Fern API reference pages before validation for documentation site changes

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

**

⚙️ CodeRabbit configuration file

**:

AGENTS.md

This file provides guidance to agents, including Claude Code and OpenAI Codex, when working in this repository.

Project Overview

NeMo Relay is a multi-language agent runtime framework for execution scopes, lifecycle events, middleware, plugins, and observability around tool and LLM calls. The core runtime is Rust. Primary supported bindings are Rust, Python, and Node.js. Go, WebAssembly, and the raw C FFI are experimental and source-first.

The shared runtime model is:

1. Scope stacks decide where work belongs and which scope-local behavior is visible.
2. Middleware registries decide what guardrails and intercepts run around managed calls.
3. Plugins install reusable runtime behavior from configuration.
4. Events record runtime behavior in ATOF form.
5. Subscribers and exporters consume events in-process or export them to ATIF, OpenTelemetry, OpenInference, or other backends.

Repository Structure

The repository layout separates the Rust runtime, language bindings, documentation,
integration patches, and agent-facing skills.

crates/
  core/       # Rust core runtime crate, published as nemo-relay
  adaptive/   # Adaptive runtime primitives and plugin components
  python/     # PyO3 native extension for the Python package
  ffi/        # Raw C ABI layer used by downstream bindings such as Go
  node/       # NAPI Node.js binding and JavaScript/TypeScript entry points
  wasm/       # wasm-bindgen WebAssembly binding and JS wrappers
python/
  nemo_relay/  # Python wrapper package: scopes, tools, LLM, middleware, typed helpers, plugins, adaptive helpers
  tests/      # Python tests
go/
  nemo_relay/  # Experimental Go CGo binding and tests
fern/         # Fern documentation site
scripts/      # Stable wrappers and helper scripts; build/test/docs entry points live in justfile
third_party/  # P...

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs
• docs/integrate-into-frameworks/provider-response-codecs.mdx

{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.

Files:

• docs/observability-plugin/openinference.mdx
• docs/observability-plugin/opentelemetry.mdx
• docs/observability-plugin/atif.mdx
• docs/integrate-into-frameworks/provider-response-codecs.mdx

**/*.rs

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Use snake_case naming convention for Rust identifiers (e.g., nemo_relay_tool_call)

**/*.rs: Any Rust change must run just test-rust
Any Rust change must run cargo fmt --all
Any Rust change must run cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all for all FFI work since it is Rust work
Run just test-rust to validate FFI changes
Run cargo clippy --workspace --all-targets -- -D warnings to enforce strict linting on FFI work

When Rust files changed as part of Go work, also run cargo fmt --all, just test-rust, and cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all when Rust files are changed as part of Node work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files are changed as part of Node work
Run just test-rust when Rust files are changed as part of Node work

**/*.rs: Run cargo fmt --all to format all Rust code
Run cargo clippy --workspace --all-targets -- -D warnings to enforce all clippy lints as errors

**/*.rs: Run cargo fmt --all when Rust files changed as part of WebAssembly work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files changed as part of WebAssembly work

**/*.rs: If any Rust code changed, always run just test-rust
If any Rust code changed, also run cargo fmt --all
If any Rust code changed, also run cargo clippy --workspace --all-targets -- -D warnings
Run Rust formatting with cargo fmt --all
Run Rust linting with cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Use cargo fmt for Rust code formatting
Run cargo clippy -- -D warnings to lint Rust code and treat all warnings as errors
Use Rust snake_case naming convention for Rust identifiers
Include SPDX license header in all Rust source files using double-slash comment syntax
Validate Rust code with uv run pre-commit run --all-files to enforce cargo fmt formatting check, cargo clippy lints, and cargo deny aud...

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}

📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)

Maintain documented and tested validation and report behavior for adaptive surfaces

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

**/{Cargo.toml,**/*.rs}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Maintain consistency between Rust package names in Cargo.toml and their actual usage across the codebase

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

**/*.{h,hpp,c,cpp,rs}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Ensure FFI header and library naming follows consistent conventions across platform-specific builds

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

{crates/core,crates/adaptive}/**/*

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

Changes to crates/core or crates/adaptive must run the full language matrix

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

**/*.{rs,toml}

📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)

Update Rust crate names and module prefixes during coordinated rename operations

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

crates/core/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)

If the change touched crates/core or shared runtime semantics, also use validate-change for broader validation

crates/core/**/*.rs: Use Json = serde_json::Value in Rust-facing runtime APIs where the existing code expects JSON payloads.
Use Result<T> with FlowError in core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

crates/{core,adaptive}/**

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If crates/core or crates/adaptive changed, run the full matrix across Rust, Python, Go, Node.js, and WebAssembly

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

**/*.{rs,py,js,ts,tsx,jsx,go,sh,toml,yaml,yml,md}

📄 CodeRabbit inference engine (AGENTS.md)

Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

**/*.{rs,py,go,js,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Follow binding naming conventions: Rust and Python use snake_case, C FFI exports prefixed nemo_relay_, Go uses PascalCase for public APIs, Node.js uses camelCase.

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

crates/**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

crates/**/*.rs: Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Use Json = serde_json::Value in Rust-facing runtime APIs for JSON payload handling.

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

crates/{core,adaptive}/**/*.rs

⚙️ CodeRabbit configuration file

crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}

⚙️ CodeRabbit configuration file

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.

Files:

• crates/core/tests/unit/codec/openai_chat_tests.rs
• crates/core/tests/unit/codec/response_tests.rs
• crates/core/tests/unit/observability/otel_tests.rs

🔇 Additional comments (7)

crates/core/tests/unit/codec/openai_chat_tests.rs (1)

118-141: LGTM!

crates/core/tests/unit/observability/otel_tests.rs (1)

974-1009: LGTM!

crates/core/tests/unit/codec/response_tests.rs (1)

1460-1472: 📐 Maintainability & Code Quality

Run the required Rust/core validation suite and add the results here.

This crates/core test change needs cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, just test-rust, and the full matrix (just test-python, just test-node, just test-go, just test-wasm) before merge.

docs/integrate-into-frameworks/provider-response-codecs.mdx (1)

311-417: LGTM!

docs/observability-plugin/atif.mdx (1)

202-208: 📐 Maintainability & Code Quality

Run docs build and link validation for the new backlink.

This adds a new docs cross-link, so please confirm the docs site build and linkcheck were run before merge. As per coding guidelines, {docs/**,README.md,CONTRIBUTING.md,**/*.md}: Run docs link validation with \just docs-linkcheck` when links changeand{docs/**,README.md,CONTRIBUTING.md}: Run docs site build with `just docs``.

Source: Coding guidelines

docs/observability-plugin/openinference.mdx (1)

90-95: LGTM!

docs/observability-plugin/opentelemetry.mdx (1)

76-81: LGTM!

---

Walkthrough

Adds three unit tests validating codec behavior (dropping unmodeled Usage fields, stripping completion_tokens_details, and emitting cost-only OTel span attributes) and expands documentation with a new "Token and Cost Field Semantics" reference section covering usage fields, cost fields, exporter field mapping tables, and stability guarantees.

Changes

Token and Cost Field Semantics: tests and docs

Layer / File(s)	Summary
Codec unit tests: Usage field dropping and completion_tokens_details stripping crates/core/tests/unit/codec/response_tests.rs, crates/core/tests/unit/codec/openai_chat_tests.rs	test_usage_ignores_unmodeled_provider_subfields verifies unknown JSON keys are dropped on Usage re-serialization; test_decode_response_drops_completion_tokens_details verifies completion_tokens is preserved while completion_tokens_details and reasoning_tokens are stripped.
OTel observability test: cost-only span attributes crates/core/tests/unit/observability/otel_tests.rs	llm_end_emits_cost_only_no_token_or_gen_ai_attributes runs OtelEventProcessor for an "other" span and asserts nemo_relay.llm.cost.* attributes are present while all token-containing and gen_ai-prefixed keys are absent.
Token and Cost Field Semantics docs docs/integrate-into-frameworks/provider-response-codecs.mdx	Adds granularity rules, normalized Usage field definitions, CostEstimate field semantics, exporter field mapping tables (ATOF/ATIF/OpenInference/OpenTelemetry), and stability guarantees.
Per-exporter docs docs/observability-plugin/atif.mdx, docs/observability-plugin/openinference.mdx, docs/observability-plugin/opentelemetry.mdx	Each exporter doc now enumerates specific token/cost attribute keys it emits and links to the canonical Token and Cost Field Semantics page.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	Matches Conventional Commits, uses a valid docs type, and clearly summarizes the token/cost semantics change.
Description check	✅ Passed	Includes the required sections, checkboxes, details, reviewer start point, and a related-issues note; only the issue reference is placeholder.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Fern docs preview: https://nvidia-preview-pull-request-330.docs.buildwithfern.com/nemo/relay (​https://nvidia-preview-pull-request-330.docs.buildwithfern.com/nemo/relay​)

[willkill07]

Approving from dev perspective