Skip to content

feat(opencode): add OpenCode Zen and Go providers that send x-opencode-session - #39549

Open
codedoga wants to merge 5 commits into
BerriAI:litellm_internal_stagingfrom
codedoga:litellm_opencode_provider
Open

feat(opencode): add OpenCode Zen and Go providers that send x-opencode-session#39549
codedoga wants to merge 5 commits into
BerriAI:litellm_internal_stagingfrom
codedoga:litellm_opencode_provider

Conversation

@codedoga

@codedoga codedoga commented Sep 3, 2026

Copy link
Copy Markdown

TLDR

Problem this solves:

  • OpenCode will reject requests missing an x-opencode-session header
  • LiteLLM has no provider for OpenCode Zen or Go
  • Callers use openai/ plus api_base, so spend reads $0

How it solves it:

  • Adds opencode (Zen) and opencode_go providers
  • Sends x-opencode-session on every outbound request
  • Routes each model to the endpoint OpenCode serves it on
  • Adds pricing for 66 Zen and 33 Go models

User Flow

Before: a developer whose app calls OpenCode through the proxy gets every request refused once OpenCode enforces the session header

  1. The proxy admin's config has model_name: kimi-k3 pointing at openai/kimi-k3 with api_base: https://opencode.ai/zen/go/v1
  2. The developer sends POST http://localhost:4000/v1/chat/completions with {"model": "kimi-k3", "messages": [{"role": "user", "content": "hi"}]}
  3. The call comes back 400 with an OpenCode error saying the x-opencode-session header is missing, so the app sees no completion
  4. Switching the config to a model OpenCode serves elsewhere, such as minimax-m3 or gemini-3.5-flash-lite, fails too: OpenCode answers 500, or 401 Model grok-4.6 is not supported for format oa-compat
  5. They open http://localhost:4000/ui/?page=logs and see the requests logged as failed at $0 spend

After: the same requests succeed whatever the model, and repeat turns of one conversation stay pinned to the same upstream

  1. The proxy admin sets OPENCODE_API_KEY and points the config at opencode_go/kimi-k3, with no api_base needed
  2. The developer sends the same POST http://localhost:4000/v1/chat/completions, adding "litellm_session_id": "sess-abc" so every turn of the conversation carries one id
  3. The call comes back 200 with a normal chat completion
  4. Swapping the model for opencode_go/minimax-m3, opencode/claude-haiku-4-5 or opencode/gemini-3.5-flash-lite also returns 200, because each is sent to the endpoint OpenCode serves it on
  5. A second turn with the same litellm_session_id comes back 200 too, and OpenCode bills it at cached-read rates
  6. http://localhost:4000/ui/?page=logs shows every request at non-zero spend, priced from the published rate card

Relevant issues

Fixes #39503

Docs companion: BerriAI/litellm-docs#1170. test_env_keys.py requires OPENCODE_API_KEY to be documented, so the code-quality and documentation jobs stay red until that one merges.

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
    • code-quality and documentation stay red until the docs companion merges; they check out litellm-docs at main to assert OPENCODE_API_KEY is documented
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Notes for reviewers

OpenCode splits its catalogue across four wire formats, and the split is per model rather than per family. On Go, qwen and minimax are served by /messages while glm, kimi and deepseek are served by /chat/completions, so no prefix rule gets this right. Each model's endpoint is recorded in the cost map under the existing supported_endpoints field and the provider picks its config from that.

Each endpoint also authenticates differently, which is documented nowhere and was found by testing:

Endpoint Auth Models
/chat/completions Authorization: Bearer 38
/responses Authorization: Bearer 31
/messages x-api-key 23
/models/<id>:generateContent x-goog-api-key 7

The Gemini path needs its own transform_request because VertexGeminiConfig raises NotImplementedError there, building its body in a bespoke handler instead.

Routing is verified against the live API for all four formats on both surfaces. 90 unit tests cover the routing table, the per-endpoint auth, the session-id precedence chain and the tiered pricing.

Screenshots / Proof of Fix

All calls are real, against OpenCode's live API, billed to a real account. Model ids are OpenCode's own, taken from https://opencode.ai/zen/v1/models and https://opencode.ai/zen/go/v1/models.

Shared setup, one model per wire format OpenCode serves:

# config.yaml
model_list:
  - model_name: opencode-chat        # OpenCode serves this on /chat/completions
    litellm_params:
      model: opencode_go/glm-5.3-flash
      api_key: os.environ/OPENCODE_API_KEY
  - model_name: opencode-responses   # ... on /responses
    litellm_params:
      model: opencode_go/gpt-5.6-luna
      api_key: os.environ/OPENCODE_API_KEY
  - model_name: opencode-messages    # ... on /messages
    litellm_params:
      model: opencode/claude-haiku-4-5
      api_key: os.environ/OPENCODE_API_KEY
  - model_name: opencode-gemini      # ... on /models/<id>:generateContent
    litellm_params:
      model: opencode/gemini-3.5-flash-lite
      api_key: os.environ/OPENCODE_API_KEY
export OPENCODE_API_KEY=<key>
export LITELLM_MASTER_KEY=sk-1234
litellm --config config.yaml

Every case below is the same request shape against /v1/chat/completions. The caller never says which upstream endpoint to use; that comes from the cost map.

After (1cc53712a9)

Case 1: OpenAI-compatible route (opencode_go/glm-5.3-flash)

  1. Send the request
curl -sS http://localhost:4000/v1/chat/completions \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"model":"opencode-chat","messages":[{"role":"user","content":"Reply with exactly: OK"}],"max_tokens":512,"litellm_session_id":"wire-format-check"}'
  1. Observe 200 with a completion
{
  "model": "opencode-chat",
  "content": "OK",
  "prompt_tokens": 17,
  "completion_tokens": 32,
  "opencode_reported_cost": "0"
}

Case 2: Responses route (opencode_go/gpt-5.6-luna)

  1. Same request, "model":"opencode-responses"
  2. Observe 200. On the merge base this model answers 500 on /chat/completions
{
  "model": "opencode-responses",
  "content": "OK",
  "prompt_tokens": 11,
  "completion_tokens": 5
}

Case 3: Anthropic route (opencode/claude-haiku-4-5)

  1. Same request, "model":"opencode-messages"
  2. Observe 200. This route also needs x-api-key rather than a bearer token
{
  "model": "opencode-messages",
  "content": "OK",
  "prompt_tokens": 12,
  "completion_tokens": 4
}

Case 4: Gemini route (opencode/gemini-3.5-flash-lite)

  1. Same request, "model":"opencode-gemini"
  2. Observe 200. This route needs x-goog-api-key, a third auth scheme
{
  "model": "opencode-gemini",
  "content": "OK",
  "prompt_tokens": 5,
  "completion_tokens": 1
}

Case 5: one conversation pinned to one session (the issue's ask)

  1. Send two turns sharing "litellm_session_id":"conversation-99"
curl -sS http://localhost:4000/v1/chat/completions \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"model":"opencode-messages","messages":[{"role":"user","content":"Reply with exactly: OK"}],"max_tokens":64,"litellm_session_id":"conversation-99"}'

curl -sS http://localhost:4000/v1/chat/completions \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"model":"opencode-messages","messages":[{"role":"user","content":"Reply with exactly: STILL OK"}],"max_tokens":64,"litellm_session_id":"conversation-99"}'
  1. Both return 200, and both carry x-opencode-session: conversation-99 upstream
{"model": "opencode-messages", "content": "OK",       "prompt_tokens": 12, "completion_tokens": 4}
{"model": "opencode-messages", "content": "STILL OK", "prompt_tokens": 14, "completion_tokens": 6}
  1. The proxy records both turns under one session with real spend
$ psql -c 'select session_id, model, round(spend::numeric,8) from "LiteLLM_SpendLogs" order by "startTime"'

 conversation-99   | opencode/claude-haiku-4-5      | 0.00003200
 conversation-99   | opencode/claude-haiku-4-5      | 0.00004400
 wire-format-check | opencode_go/glm-5.3-flash      | 0.00001728
 wire-format-check | opencode_go/gpt-5.6-luna       | 0.00000820
 wire-format-check | opencode/claude-haiku-4-5      | 0.00003200
 wire-format-check | opencode/gemini-3.5-flash-lite | 0.00000400

http://localhost:4000/ui/?page=logs shows the same, with the two conversation-99 turns collapsed into one session row.

Note on Case 1: opencode_reported_cost is "0" because OpenCode Go is a subscription and reports no per-request cost, while LiteLLM tracks the published per-token rates. That is the first Medium caveat below, not a bug.

Type

🆕 New Feature

Caveats (if any)

Medium

  • Go's flat $10/month cap is not modelled
    • Per-token spend is tracked, and OpenCode reports cost: 0 for Go
  • DeepSeek models bill 2x during peak UTC hours
    • Cost map carries off-peak rates, so peak spend under-reports
  • opencode_go/hy3-preview has no published pricing
    • It is live but left out of the cost map

Low

  • Six models' endpoints are inferred rather than documented
    • grok-4.5 and qwen3.5-plus were confirmed live; the rest default to chat completions
  • Four Go models currently return No allowed providers are available
    • An upstream availability problem on OpenCode's side, not routing
  • Streaming is untested against the live API
    • Only non-streaming calls were exercised

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

…e-session

OpenCode starts rejecting inference requests that arrive without an
x-opencode-session header, and LiteLLM had no provider for either of its two
managed inference surfaces, so callers had to point openai/ at the base URL and
got no cost tracking.

Adds opencode (Zen, https://opencode.ai/zen/v1) and opencode_go
(https://opencode.ai/zen/go/v1). Both are OpenAI-compatible, so the configs only
override the default base URL, the credential lookup and the session header. The
header value is the caller's litellm_session_id, then metadata.session_id, then
litellm_trace_id, then litellm_call_id, and a caller-supplied header is never
overwritten.

Routes both providers through base_llm_http_handler, since the OpenAI SDK path
uses the handler's own validate_environment and would drop the header.

Adds pricing for the 66 Zen and 33 Go models the live /models endpoints list,
including the context-tiered rates, plus the four above_256k_tokens fields the
cost map needed but ModelInfoBase had not declared.
@CLAassistant

CLAassistant commented Sep 3, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codspeed-hq

codspeed-hq Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing codedoga:litellm_opencode_provider (1cc5371) with litellm_internal_staging (8699998)

Open in CodSpeed

@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.44751% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/main.py 95.45% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@greptile-apps

greptile-apps Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds OpenCode Zen and Go provider support with model-aware routing, endpoint-specific authentication, session headers, and pricing metadata

  • Registers both providers for model discovery and provider resolution
  • Adds adapters for chat completions, Anthropic Messages, Gemini, and Responses wire formats
  • Adds model pricing, endpoint capability metadata, and focused provider tests

Confidence Score: 5/5

The PR appears safe to merge

No blocking failure remains

Important Files Changed

Filename Overview
litellm/llms/opencode/common_utils.py Centralizes OpenCode credential, session-header, and model-endpoint resolution
litellm/llms/opencode/chat/transformation.py Implements endpoint-specific chat, Messages, and Gemini request configuration for OpenCode
litellm/llms/opencode/responses/transformation.py Adds Responses API URL, authentication, and session-header handling
litellm/utils.py Registers model-aware OpenCode adapter selection and preserves dynamic threshold pricing fields
provider_endpoints_support.json Advertises only the unified endpoint families supported by the registered OpenCode adapters
tests/test_litellm/llms/opencode/test_opencode_chat_transformation.py Covers routing, authentication, session precedence, and tiered billing behavior

Reviews (2): Last reviewed commit: "feat(opencode): route each model to the ..." | Re-trigger Greptile

Comment thread provider_endpoints_support.json Outdated
Comment thread litellm/main.py
OpenCode splits its catalogue across four wire formats and the split is per model,
not per family: on Go, qwen and minimax are served by /messages while glm, kimi and
deepseek are served by /chat/completions. Sending a model to the wrong one returns
500, or 401 "not supported for format oa-compat". Each endpoint also authenticates
differently: bearer for /chat/completions and /responses, x-api-key for /messages,
and x-goog-api-key for the Gemini path.

Records each model's endpoint in the cost map under the existing supported_endpoints
field and picks the matching config from it, adding an Anthropic-shaped config for
/messages, a Gemini-shaped one for models/<id>:generateContent, and mode=responses so
the built-in bridge handles the OpenAI-native models. Gemini needs its own
transform_request because VertexGeminiConfig raises NotImplementedError, building its
body in a bespoke handler instead.

Collapses the credential lookup to one OPENCODE_API_KEY, since a single account key
authenticates both Zen and Go.

Corrects the endpoint support metadata, which claimed a2a, interactions and messages
support inherited from the template it was copied from; only chat completions and
responses have adapters.

All four formats verified against the live API on both surfaces.
@codedoga

codedoga commented Sep 3, 2026

Copy link
Copy Markdown
Author

256k tiers are not dropped: utils.py:5941 copies any _above_* key through. Verified billing, 300k costs $0.3648 vs $0.1216 base. Tests added. @greptileai

…r branches

The responses configs were verified against the live API but had no unit tests, so
codecov reported them at 0%. Adds coverage for the default and overridden base URLs,
the bearer auth, the session header including a caller-supplied one, and the config
the provider manager hands back for each surface.

Takes the opencode package to 100% statement coverage.
@codedoga

codedoga commented Sep 3, 2026

Copy link
Copy Markdown
Author

Docs companion opened: BerriAI/litellm-docs#1170. Documents OPENCODE_API_KEY, which unblocks the code-quality and documentation jobs. Coverage gap also fixed.

…ap validator expects

Three CI failures, all from the routing work.

The Claude models are Anthropic re-exports, so they need the same
prompt_cache_min_tokens the azure_ai, databricks and openrouter re-exports carry.
Without it the router treats short prefixes as uncacheable and skips prompt-cache
affinity, which is the routing this provider's session header exists to enable.
The four Gemini flash models get their published minimum for the same reason.

The cost map validator enumerates supported_endpoints, and Gemini's native route was
not among them; it now sits alongside /vertex_ai/live and /v1beta/interactions, which
are provider-specific routes already in that list. The validator also did not know the
two 256k tier fields this branch introduced.

Also pins the completion-wiring tests to the bundled cost map. They routed off the
fetched map, so whether a model reached /chat/completions or /messages depended on
which map happened to be loaded, and claude-opus-5 passed locally while failing in CI.
It now asserts against /messages, the endpoint OpenCode actually serves it on, with an
Anthropic-shaped response.
…uards require

The Claude entries are Anthropic re-exports, so they have to carry the flags that decide
the request shape LiteLLM emits. Without supports_adaptive_thinking the transformations
send the legacy thinking.type='enabled' and the provider 400s; Fable 5 additionally needs
thinking_always_on, and Fable 5 with Opus 4.7 and 4.8 reject sampling params. These are
copied from the canonical upstream entry rather than guessed.

Drops the ModelInfoBase declarations for the four above_256k_tokens fields added earlier
in this branch. The cost calculator resolves tier fields dynamically from the raw cost
map, which is how the above_256k entries that predate this branch already worked, so the
declarations bought nothing and pulled in a matching CustomPricingLiteLLMParams mirror.
The 256k tiers stay verified by the billing test.

Teaches the cost map validator about Gemini's native route, alongside /vertex_ai/live and
/v1beta/interactions, which are provider-specific routes already in that list.
hao3039032 pushed a commit to hao3039032/litellm that referenced this pull request Sep 7, 2026
Port upstream PR BerriAI#39549 (commit 1cc5371)
adding opencode/opencode_go provider support, with fixes on top:

- route per-model wire format via the cost map so mixed-case model group
  names (e.g. MiniMax-M3) still resolve the correct endpoint
- stamp the mandatory x-opencode-session header from the inbound header,
  litellm_session_id, metadata.session_id, trace id, or call id
- serve /messages models with x-api-key auth via Anthropic-format configs
- delegate streaming to the Anthropic response iterator for /messages
@lmtrr

lmtrr commented Sep 7, 2026

Copy link
Copy Markdown

Seem like they don't review community PR anymore

@J-eremy

J-eremy commented Sep 7, 2026

Copy link
Copy Markdown

Well this is disappointing. I was directed here by a couple of different models that say LiteLLM was told about this problem and are seemingly ignoring it. Thanks for the effort in trying to fix it for us @codedoga

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Send x-opencode-session header on API requests (required by OpenCode Go from 09/05)

4 participants