docs(pricing): document the batch and long-context batch pricing keys - #1221
Open
mateo-berri wants to merge 1 commit into
Open
docs(pricing): document the batch and long-context batch pricing keys#1221mateo-berri wants to merge 1 commit into
mateo-berri wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Author
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 0a61b7a. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Adds a "Batch pricing keys" section to the Add Model Pricing page listing the eight
*_batchescost-map keys, including theabove_272k_tokens_batchestier keys that BerriAI/litellm#39861 ships, and says how a batch line that crosses 272K prompt tokens billsLinear ticket
Relates to LIT-6903
Note
Low Risk
Documentation-only change to provider registration guidance; no runtime or billing code is modified.
Overview
Adds a Batch pricing keys section to the Add Model Pricing guide so contributors know how to set batch billing fields in
model_prices_and_context_window.json.The new text states that batch lines use the
*_batchescost-map keys when present, and otherwise bill at half the standard rate. It lists eight keys (input/output, cache read/create, plusabove_272k_tokens_batchestiers) and explains that long-context batch tiers apply when the prompt exceeds 272K tokens, with missing tier keys falling back to the flat*_batchesrate for those tokens.Reviewed by Cursor Bugbot for commit 0a61b7a. Bugbot is set up for automated code reviews on this repo. Configure here.