fix(cost): warn on unpriced models; add Claude Opus 4.7/4.8/5, Sonnet 5, Fable 5/5.1 pricing - #5804
Open
arthi-arumugam-git wants to merge 1 commit into
Conversation
…5, Fable 5/5.1 A request for a model that is missing from the pricing tables was recorded at $0.00 with correct token counts and no log line: calculateModelCostBreakdown returned null on a registry miss and modelCost turned the legacy null into 0. - Log a console.warn naming the provider and model on a registry miss (calculate-cost.ts) and on a legacy miss that the registry cannot price either (costCalc.ts). Return types and persisted values are unchanged. - Add anthropic registry entries and legacy table rows for claude-opus-4-7, claude-opus-4-8, claude-opus-5, claude-sonnet-5, claude-fable-5 and claude-fable-5-1 at the rates on Anthropic's pricing page, including the 0.025x cache read multiplier on Fable 5.1. - Add tests for the warning and for each new model's rates; update the registry snapshot for the new entries.
Contributor
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
|
@arthi-arumugam-git is attempting to deploy a commit to the Helicone Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket
None.
Component/Service
What part of Helicone does this affect?
Type of Change
Deployment Notes
Screenshots / Demos
claude-opus-4-8, 50,000 input + 15,000 output tokens: cost recorded as $0.00, token counts correct, nothing loggednull/0, silentlynull/0, plus oneconsole.warnnaming the provider and modelExtra Notes
Part 1: make an unpriced model loud
packages/cost/models/calculate-cost.ts: on a registry miss,console.warnwith theproviderModelIdandproviderbefore returningnull(the return type is unchanged).packages/cost/costCalc.ts:modelCoststill returns0whencostOfPromptreturnsnull, but now warns first. The warning is suppressed when the model registry has an entry for the same model and provider: Jawn callsmodelCoston every non-gateway request before the registry path, andLoggingHandlerprefers the registry result, so a legacy miss on a registry-priced model (for examplegpt-5.4, which has no legacy row) is expected and would otherwise warn on every request.console.warnmatches the existing logging in this package (unified/providers.ts).ProxyForwarder.ts,costOfPrompt(...) ?? 0) is not touched; it already goes through the registry warning first.Part 2: Claude models missing from the registry
Added
packages/cost/models/authors/anthropic/{claude-4.7-opus,claude-4.8-opus,claude-5-opus,claude-5-sonnet,claude-5-fable,claude-5.1-fable}(each withmodel.tsandendpoints.ts, mirroringclaude-4.6-sonnet), wired intoauthors/anthropic/index.ts, plusequalsrows in the legacy tablepackages/cost/providers/anthropic/index.ts.claude-opus-4-7claude-opus-4-8claude-opus-5claude-sonnet-5claude-fable-5claude-fable-5-1All per MTok. Multipliers are the standard 1.25x / 2x / 0.1x, except the Fable 5.1 cache read, which is 0.025x and set explicitly because the engine has no default for it. Context 1M, max output 128K,
createdset to each model's release date, all from the model pages listed below.Choices a reviewer may want to change:
:anthropicendpoints are added. Bedrock, Vertex and OpenRouter rates are on other pages that I did not verify, and nopa/Helicone endpoint is assumed to exist.ptbEnabled: truematches every other:anthropicentry. Flip tofalseif pass-through billing should not be offered for these models yet.supportedParametersandsupportedPluginsmirror the 4.6 entries. The Sonnet 5 model page notes that non-defaulttemperature/top_p/top_kreturn a 400 on that model; I left the informational list unchanged rather than guess at gateway behaviour.equalsso thatclaude-fable-5cannot matchclaude-fable-5-1, which has a different cache read rate.Deliberately left out:
gpt-5.5and Gemini 3.5/3.6/3.7 Flash, which are also missing from the registry; this PR only adds models whose full rate row I verified on Anthropic's pricing page.modelDetailsentries,unified/models.tsplayground mappings and the Stripe meter list; none of them affect cost calculation.Tests
packages/__tests__/cost/modelCostFromRegistry.test.ts: 11 new tests. Five cover the warning (registry miss warns with provider and model; registry hit does not; legacy miss on a model unknown to both tables warns and returns 0; legacy miss on a registry-priced model does not warn; legacy hit does not warn). Six cover the new models, including the pricing page's worked example (50,000 in + 15,000 out at $5/$25 = $0.625) on Opus 4.7/4.8/5, the cache rates on Sonnet 5 and Fable 5, the 0.025x Fable 5.1 cache read, and the legacy rows for Opus 4.8 and Fable 5.1.packages/__tests__/cost: 13 suites, 201 tests, 8 snapshots pass (190 tests before this change).registrySnapshots.test.ts.snapwas updated with-u; the diff adds the six new entries and bumpsmodelCount15 → 21,totalEndpoints/totalModelProviderConfigs329 → 335,totalModelsWithPtb108 → 114, nothing else.Sources (fetched 2026-09-02)
https://platform.claude.com/docs/en/about-claude/pricing, model pricing table rows as shown:
(columns: Base input tokens | 5m cache writes | 1h cache writes | Cache hits and refreshes | Output tokens)
Worked example on the same page (Claude Opus 5): "Input tokens | 50,000 × $5 / 1,000,000 | $0.25" and "Output tokens | 15,000 × $25 / 1,000,000 | $0.375".
Model ids, release dates, context window and max output: https://platform.claude.com/docs/en/models/overview and the per-model pages
models/opus-4-7/overview,models/opus-4-8/overview,models/opus-5/overview,models/sonnet-5/overview,models/fable-5/overview,models/fable-5-1/overview(each states "Model ID", "Released", "Context window: 1M tokens", "Max output: 128K tokens"). The overview page notes: "Every Claude model ID is a pinned snapshot, including the dateless IDs used from the 4.6 generation on."Context
Why are you making this change?
A registry lookup is an exact
${providerModelId}:${provider}match (registry.ts). When it misses,calculateModelCostBreakdownreturnsnull(calculate-cost.ts), Jawn skips the breakdown (ResponseBodyHandler.ts), the legacycostOfPromptalso returnsnull,modelCostturns that into0(costCalc.ts), andLoggingHandler.tsstoresatLeastZero(... ?? 0). Nothing on that path logs, so every request for a model that is not in the tables is recorded at $0.00 with correct token counts, and dashboard spend for a whole model family silently reads zero.Today that covers every Anthropic model released after Claude 4.6. Anthropic's own worked example (50,000 input + 15,000 output tokens on Opus 5 at $5 / $25 per MTok) is billed at $0.625 and recorded by Helicone as $0.00; the same holds for
claude-opus-4-8, which is priced identically. This PR makes the miss visible and adds the six Anthropic models whose full rate row is on the pricing page.Screenshots / Demos
See the table above.