refactor: reduce basedpyright Any counts by ~5.5k across 12 hot files

[mateo-berri]

Types the highest-volume Any sources in the proxy and core paths: Prisma row seams, **row.model_dump() unpacks, response payloads, and untyped helper returns. No runtime behavior changes intended.

Measured effect

Both scans run with the same basedpyright 1.39.7 in the same virtualenv, so the numbers are comparable.

rule	before	after	delta
reportAny	26979	21998	-4981
reportExplicitAny	7432	6918	-514
all errors	157609	151413	-6196

basedpyright-code-budget.json is ratcheted down by 6215 across 48 rules (reportAny 37484 -> 32503, reportExplicitAny 10389 -> 9875) so the gains cannot be given back.

Gates

type_check_gate.py, budget_ratchet_check.py, type_discipline_gate.py, ruff_strict_gate.py, ruff check, ruff format --check, circular-import and import-safety checks all pass locally.

Reviewer note on suppressions

This adds 165 # cast-ok: and 128 # mutable-ok: comments, each with a stated reason. A cast is an unchecked assertion, so these move the Any out of the checker's view rather than proving it away; they deserve a closer read than the rest of the diff. LIT006 and LIT009 both still finish below their pre-change baselines, and no new # type: ignore was introduced.

Several # mutable-ok: uses were forced: the row-dump TypedDicts are splatted into pydantic models whose fields are declared List/Dict, and widening those to Sequence/Mapping introduced hundreds of reportArgumentType errors.

Net non-Any regression is 18 errors across six rules (reportArgumentType +7, reportAttributeAccessIssue +5, reportDeprecated +2, reportUnnecessaryComparison +2, reportAssignmentType +1, reportUnnecessaryIsInstance +1), all within budget.

Tests

The affected suites (proxy/auth, proxy/management_endpoints, litellm_core_utils, llms/custom_httpx, proxy/utils, proxy/db, test_proxy_utils.py) were run on this branch and at the merge-base with identical settings:

run	result
this branch	2 failed, 4722 passed, 2 skipped
merge-base	2 failed, 4722 passed, 2 skipped

The same two tests fail on both sides — test_get_custom_url and test_mcp_token_opens_sealed_passthrough_code_and_exchanges_with_minted_client, both asserting on a proxy base URL supplied by local .env config. No regressions.

---

Note

Medium Risk
Changes sit on the core streaming path where many providers are parsed; logic is meant to be unchanged but unchecked casts could hide mismatches if chunk shapes differ from the new TypedDicts.

Overview
Ratchets basedpyright-code-budget.json downward (notably reportAny 37484→32503 and reportExplicitAny 10389→9875, plus smaller cuts on several reportUnknown* and reportMissing* rules) so recent typing gains cannot regress.

In streaming_handler.py, adds TypedDicts and a small Protocol for provider SSE/JSON chunk shapes (Predibase, AI21, Maritalk, NLP Cloud, Aleph Alpha, Azure, Baseten) and narrows helpers (_next_sync_or_exhausted, is_async_iterable, calculate_total_usage, generic_chunk_has_all_required_fields) away from bare Any.

CustomStreamWrapper now types logging_obj as LiteLLMLoggingObject, reads logged params via _LoggedLiteLLMParams, and uses cast + _DumpedDeltaKwargs / _ModelResponseStreamInitKwargs at Delta(**…) and ModelResponseStream(**…) boundaries; cached-response handling is renamed to cached_chunk with explicit ModelResponseStream typing. No intended streaming behavior change.

^{Reviewed by Cursor Bugbot for commit 9958e4e. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codecov]

Codecov Report

❌ Patch coverage is 81.75500% with 210 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...oxy/management_endpoints/organization_endpoints.py	30.00%	49 Missing ⚠️
litellm/proxy/utils.py	72.72%	39 Missing ⚠️
litellm/llms/openai/openai.py	15.90%	37 Missing ⚠️
litellm/proxy/_experimental/mcp_server/db.py	78.81%	25 Missing ⚠️
...tellm/proxy/management_endpoints/team_endpoints.py	90.72%	23 Missing ⚠️
...y/management_endpoints/key_management_endpoints.py	81.25%	9 Missing ⚠️
litellm/litellm_core_utils/streaming_handler.py	89.74%	8 Missing ⚠️
...xy/management_endpoints/internal_user_endpoints.py	88.05%	8 Missing ⚠️
litellm/proxy/auth/auth_checks.py	97.07%	7 Missing ⚠️
...roxy/management_endpoints/common_daily_activity.py	91.66%	4 Missing ⚠️
... and 1 more

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR reduces basedpyright Any usage across core provider, streaming, authentication, database, proxy, and management paths

• Adds TypedDicts, protocols, narrower annotations, and explicit casts around dynamic payloads and Prisma rows
• Ratchets down the basedpyright error budget to preserve the resulting type-checking improvements

Confidence Score: 5/5

The PR appears safe to merge

No blocking failures remain

Important Files Changed

Filename	Overview
basedpyright-code-budget.json	Lowers type-checking limits to preserve the reduced diagnostic counts
litellm/litellm_core_utils/streaming_handler.py	Adds typed provider payloads and narrows streaming response construction without an accepted runtime defect
litellm/llms/custom_httpx/llm_http_handler.py	Narrows shared HTTP-handler request, response, coroutine, and callback types
litellm/llms/openai/openai.py	Types OpenAI adapter payloads and helper boundaries
litellm/proxy/_experimental/mcp_server/db.py	Types MCP repository tables, persisted payloads, credentials, and returned rows
litellm/proxy/auth/auth_checks.py	Adds typed repository-row and authorization payload boundaries without an accepted auth regression
litellm/proxy/management_endpoints/common_daily_activity.py	Types daily-activity database rows and response aggregation
litellm/proxy/management_endpoints/internal_user_endpoints.py	Types user-management Prisma rows and endpoint payload construction
litellm/proxy/management_endpoints/key_management_endpoints.py	Types virtual-key management rows and request data
litellm/proxy/management_endpoints/organization_endpoints.py	Types organization rows, memberships, and endpoint payloads
litellm/proxy/management_endpoints/team_endpoints.py	Types team rows, memberships, and management response construction
litellm/proxy/proxy_server.py	Narrows proxy configuration and request-path values
litellm/proxy/utils.py	Adds typed Prisma table protocols and narrows database and configuration helper boundaries

_{Reviews (2): Last reviewed commit: "fix: keep health check writes on the rep..." | Re-trigger Greptile}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Incorrect cast target TypedDicts
  • Expanded _DumpedDeltaKwargs to cover the real Delta / completion_obj fields (notably content), introduced a dedicated _DumpedStreamingChoiceKwargs for the StreamingChoices.model_dump() splat site, and widened _ModelResponseStreamInitKwargs to reflect the stream chunk fields (id, created, object, system_fingerprint, usage).

Or push these changes by commenting:

@cursor push 1959e6ac3b

Preview (1959e6ac3b)

diff --git a/litellm/litellm_core_utils/streaming_handler.py b/litellm/litellm_core_utils/streaming_handler.py
--- a/litellm/litellm_core_utils/streaming_handler.py
+++ b/litellm/litellm_core_utils/streaming_handler.py
@@ -72,13 +72,30 @@
 
 
 class _DumpedDeltaKwargs(TypedDict, total=False):
+    content: str | None
     role: str | None
+    function_call: Mapping[str, object] | None
+    tool_calls: Sequence[Mapping[str, object]] | None
+    reasoning_content: str | None
+    provider_specific_fields: Mapping[str, object] | None
     finish_reason: str | None
-    tool_calls: Sequence[Mapping[str, object]] | None
+    index: int | None
 
 
+class _DumpedStreamingChoiceKwargs(TypedDict, total=False):
+    finish_reason: str | None
+    index: int | None
+    logprobs: Mapping[str, object] | None
+    enhancements: object
+
+
 class _ModelResponseStreamInitKwargs(TypedDict, total=False):
+    id: str | None
+    created: int | None
     model: str | None
+    object: str | None
+    system_fingerprint: str | None
+    usage: Mapping[str, object] | None
 
 
 class _PredibaseStreamDetails(TypedDict):
@@ -986,7 +1003,7 @@
                             try:
                                 if isinstance(choice, BaseModel):
                                     choice_json = cast(  # cast-ok: choice dump feeds StreamingChoices(**kwargs) which takes extras
-                                        _DumpedDeltaKwargs,
+                                        _DumpedStreamingChoiceKwargs,
                                         choice.model_dump(),
                                     )
                                     choice_json.pop(

_{You can send follow-ups to the cloud agent here.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 9958e4e. Configure here.}

[cursor]

Incorrect cast target TypedDicts

Low Severity

_DumpedDeltaKwargs only declares role / finish_reason / tool_calls, yet it is used as the cast target for full Delta.model_dump() payloads (which include content and more) and even for StreamingChoices.model_dump() before StreamingChoices(**choice_json). _ModelResponseStreamInitKwargs similarly only declares model while casting heterogeneous stream args. cast does not filter at runtime today, but these shapes document the wrong contract and make a later “typed” splat easy to strip required stream fields.

Additional Locations (2)

• litellm/litellm_core_utils/streaming_handler.py#L987-L995
• litellm/litellm_core_utils/streaming_handler.py#L79-L81

 

^{Reviewed by Cursor Bugbot for commit 9958e4e. Configure here.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_decrease_anys (9958e4e) with litellm_internal_staging (b9b27c2)}