feat(opencode): add OpenCode Zen and Go providers that send x-opencode-session - #39549
feat(opencode): add OpenCode Zen and Go providers that send x-opencode-session#39549codedoga wants to merge 5 commits into
Conversation
…e-session OpenCode starts rejecting inference requests that arrive without an x-opencode-session header, and LiteLLM had no provider for either of its two managed inference surfaces, so callers had to point openai/ at the base URL and got no cost tracking. Adds opencode (Zen, https://opencode.ai/zen/v1) and opencode_go (https://opencode.ai/zen/go/v1). Both are OpenAI-compatible, so the configs only override the default base URL, the credential lookup and the session header. The header value is the caller's litellm_session_id, then metadata.session_id, then litellm_trace_id, then litellm_call_id, and a caller-supplied header is never overwritten. Routes both providers through base_llm_http_handler, since the OpenAI SDK path uses the handler's own validate_environment and would drop the header. Adds pricing for the 66 Zen and 33 Go models the live /models endpoints list, including the context-tiered rates, plus the four above_256k_tokens fields the cost map needed but ModelInfoBase had not declared.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Greptile SummaryThe PR adds OpenCode Zen and Go provider support with model-aware routing, endpoint-specific authentication, session headers, and pricing metadata
Confidence Score: 5/5The PR appears safe to merge No blocking failure remains
|
| Filename | Overview |
|---|---|
| litellm/llms/opencode/common_utils.py | Centralizes OpenCode credential, session-header, and model-endpoint resolution |
| litellm/llms/opencode/chat/transformation.py | Implements endpoint-specific chat, Messages, and Gemini request configuration for OpenCode |
| litellm/llms/opencode/responses/transformation.py | Adds Responses API URL, authentication, and session-header handling |
| litellm/utils.py | Registers model-aware OpenCode adapter selection and preserves dynamic threshold pricing fields |
| provider_endpoints_support.json | Advertises only the unified endpoint families supported by the registered OpenCode adapters |
| tests/test_litellm/llms/opencode/test_opencode_chat_transformation.py | Covers routing, authentication, session precedence, and tiered billing behavior |
Reviews (2): Last reviewed commit: "feat(opencode): route each model to the ..." | Re-trigger Greptile
OpenCode splits its catalogue across four wire formats and the split is per model, not per family: on Go, qwen and minimax are served by /messages while glm, kimi and deepseek are served by /chat/completions. Sending a model to the wrong one returns 500, or 401 "not supported for format oa-compat". Each endpoint also authenticates differently: bearer for /chat/completions and /responses, x-api-key for /messages, and x-goog-api-key for the Gemini path. Records each model's endpoint in the cost map under the existing supported_endpoints field and picks the matching config from it, adding an Anthropic-shaped config for /messages, a Gemini-shaped one for models/<id>:generateContent, and mode=responses so the built-in bridge handles the OpenAI-native models. Gemini needs its own transform_request because VertexGeminiConfig raises NotImplementedError, building its body in a bespoke handler instead. Collapses the credential lookup to one OPENCODE_API_KEY, since a single account key authenticates both Zen and Go. Corrects the endpoint support metadata, which claimed a2a, interactions and messages support inherited from the template it was copied from; only chat completions and responses have adapters. All four formats verified against the live API on both surfaces.
|
256k tiers are not dropped: utils.py:5941 copies any |
…r branches The responses configs were verified against the live API but had no unit tests, so codecov reported them at 0%. Adds coverage for the default and overridden base URLs, the bearer auth, the session header including a caller-supplied one, and the config the provider manager hands back for each surface. Takes the opencode package to 100% statement coverage.
|
Docs companion opened: BerriAI/litellm-docs#1170. Documents OPENCODE_API_KEY, which unblocks the code-quality and documentation jobs. Coverage gap also fixed. |
…ap validator expects Three CI failures, all from the routing work. The Claude models are Anthropic re-exports, so they need the same prompt_cache_min_tokens the azure_ai, databricks and openrouter re-exports carry. Without it the router treats short prefixes as uncacheable and skips prompt-cache affinity, which is the routing this provider's session header exists to enable. The four Gemini flash models get their published minimum for the same reason. The cost map validator enumerates supported_endpoints, and Gemini's native route was not among them; it now sits alongside /vertex_ai/live and /v1beta/interactions, which are provider-specific routes already in that list. The validator also did not know the two 256k tier fields this branch introduced. Also pins the completion-wiring tests to the bundled cost map. They routed off the fetched map, so whether a model reached /chat/completions or /messages depended on which map happened to be loaded, and claude-opus-5 passed locally while failing in CI. It now asserts against /messages, the endpoint OpenCode actually serves it on, with an Anthropic-shaped response.
…uards require The Claude entries are Anthropic re-exports, so they have to carry the flags that decide the request shape LiteLLM emits. Without supports_adaptive_thinking the transformations send the legacy thinking.type='enabled' and the provider 400s; Fable 5 additionally needs thinking_always_on, and Fable 5 with Opus 4.7 and 4.8 reject sampling params. These are copied from the canonical upstream entry rather than guessed. Drops the ModelInfoBase declarations for the four above_256k_tokens fields added earlier in this branch. The cost calculator resolves tier fields dynamically from the raw cost map, which is how the above_256k entries that predate this branch already worked, so the declarations bought nothing and pulled in a matching CustomPricingLiteLLMParams mirror. The 256k tiers stay verified by the billing test. Teaches the cost map validator about Gemini's native route, alongside /vertex_ai/live and /v1beta/interactions, which are provider-specific routes already in that list.
Port upstream PR BerriAI#39549 (commit 1cc5371) adding opencode/opencode_go provider support, with fixes on top: - route per-model wire format via the cost map so mixed-case model group names (e.g. MiniMax-M3) still resolve the correct endpoint - stamp the mandatory x-opencode-session header from the inbound header, litellm_session_id, metadata.session_id, trace id, or call id - serve /messages models with x-api-key auth via Anthropic-format configs - delegate streaming to the Anthropic response iterator for /messages
|
Seem like they don't review community PR anymore |
|
Well this is disappointing. I was directed here by a couple of different models that say LiteLLM was told about this problem and are seemingly ignoring it. Thanks for the effort in trying to fix it for us @codedoga |
TLDR
Problem this solves:
x-opencode-sessionheaderopenai/plusapi_base, so spend reads $0How it solves it:
opencode(Zen) andopencode_goprovidersx-opencode-sessionon every outbound requestUser Flow
Before: a developer whose app calls OpenCode through the proxy gets every request refused once OpenCode enforces the session header
model_name: kimi-k3pointing atopenai/kimi-k3withapi_base: https://opencode.ai/zen/go/v1{"model": "kimi-k3", "messages": [{"role": "user", "content": "hi"}]}x-opencode-sessionheader is missing, so the app sees no completionminimax-m3orgemini-3.5-flash-lite, fails too: OpenCode answers 500, or 401Model grok-4.6 is not supported for format oa-compatAfter: the same requests succeed whatever the model, and repeat turns of one conversation stay pinned to the same upstream
OPENCODE_API_KEYand points the config atopencode_go/kimi-k3, with noapi_baseneeded"litellm_session_id": "sess-abc"so every turn of the conversation carries one idopencode_go/minimax-m3,opencode/claude-haiku-4-5oropencode/gemini-3.5-flash-litealso returns 200, because each is sent to the endpoint OpenCode serves it onlitellm_session_idcomes back 200 too, and OpenCode bills it at cached-read ratesRelevant issues
Fixes #39503
Docs companion: BerriAI/litellm-docs#1170.
test_env_keys.pyrequiresOPENCODE_API_KEYto be documented, so thecode-qualityanddocumentationjobs stay red until that one merges.Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or morecode-qualityanddocumentationstay red until the docs companion merges; they check outlitellm-docsatmainto assertOPENCODE_API_KEYis documented@greptileaito re-request a review after pushing changes)Notes for reviewers
OpenCode splits its catalogue across four wire formats, and the split is per model rather than per family. On Go, qwen and minimax are served by
/messageswhile glm, kimi and deepseek are served by/chat/completions, so no prefix rule gets this right. Each model's endpoint is recorded in the cost map under the existingsupported_endpointsfield and the provider picks its config from that.Each endpoint also authenticates differently, which is documented nowhere and was found by testing:
/chat/completionsAuthorization: Bearer/responsesAuthorization: Bearer/messagesx-api-key/models/<id>:generateContentx-goog-api-keyThe Gemini path needs its own
transform_requestbecauseVertexGeminiConfigraisesNotImplementedErrorthere, building its body in a bespoke handler instead.Routing is verified against the live API for all four formats on both surfaces. 90 unit tests cover the routing table, the per-endpoint auth, the session-id precedence chain and the tiered pricing.
Screenshots / Proof of Fix
All calls are real, against OpenCode's live API, billed to a real account. Model ids are OpenCode's own, taken from
https://opencode.ai/zen/v1/modelsandhttps://opencode.ai/zen/go/v1/models.Shared setup, one model per wire format OpenCode serves:
Every case below is the same request shape against
/v1/chat/completions. The caller never says which upstream endpoint to use; that comes from the cost map.After (
1cc53712a9)Case 1: OpenAI-compatible route (
opencode_go/glm-5.3-flash){ "model": "opencode-chat", "content": "OK", "prompt_tokens": 17, "completion_tokens": 32, "opencode_reported_cost": "0" }Case 2: Responses route (
opencode_go/gpt-5.6-luna)"model":"opencode-responses"/chat/completions{ "model": "opencode-responses", "content": "OK", "prompt_tokens": 11, "completion_tokens": 5 }Case 3: Anthropic route (
opencode/claude-haiku-4-5)"model":"opencode-messages"x-api-keyrather than a bearer token{ "model": "opencode-messages", "content": "OK", "prompt_tokens": 12, "completion_tokens": 4 }Case 4: Gemini route (
opencode/gemini-3.5-flash-lite)"model":"opencode-gemini"x-goog-api-key, a third auth scheme{ "model": "opencode-gemini", "content": "OK", "prompt_tokens": 5, "completion_tokens": 1 }Case 5: one conversation pinned to one session (the issue's ask)
"litellm_session_id":"conversation-99"x-opencode-session: conversation-99upstream{"model": "opencode-messages", "content": "OK", "prompt_tokens": 12, "completion_tokens": 4} {"model": "opencode-messages", "content": "STILL OK", "prompt_tokens": 14, "completion_tokens": 6}http://localhost:4000/ui/?page=logsshows the same, with the twoconversation-99turns collapsed into one session row.Note on Case 1:
opencode_reported_costis"0"because OpenCode Go is a subscription and reports no per-request cost, while LiteLLM tracks the published per-token rates. That is the first Medium caveat below, not a bug.Type
🆕 New Feature
Caveats (if any)
Medium
cost: 0for Goopencode_go/hy3-previewhas no published pricingLow
grok-4.5andqwen3.5-pluswere confirmed live; the rest default to chat completionsNo allowed providers are availableFinal Attestation