Skip to content

docs(auto_router): list mid-task stall escalation and one-click setup #1213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
moe-berri merged 2 commits into from
Sep 5, 2026
+6 −2
Merged

docs(auto_router): list mid-task stall escalation and one-click setup #1213

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
21d9376
docs(auto_router): list mid-task stall escalation and one-click setup
moe-berri Sep 5, 2026
1ef3c02
docs(auto_router): correct one-click setup to describe Configure auto…
moe-berri Sep 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/auto_router/feature_history.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,10 @@ Merged after the v1.100.0 release candidate was cut. Available in `v1.101.0-dev`
- **Shadow evals on teams and users.** Target a `key`, `team`, or `user`, so JWT-authenticated traffic can be evaluated. [#39015](https://github.com/BerriAI/litellm/pull/39015).
- **Shadow evals across several routers.** Compare multiple router configs on one job's sampled traffic, paired. [#39028](https://github.com/BerriAI/litellm/pull/39028).
- **1M context preset.** [#39490](https://github.com/BerriAI/litellm/pull/39490).
- **Mid-task stall escalation.** `stall_escalation_enabled: true` reads the assistant's own recent tool calls and bumps a request one tier when it's stuck in a retry loop, the same ladder `escalation_keywords` uses. Off by default. [#39809](https://github.com/BerriAI/litellm/pull/39809). [Post](/blog/auto-router-stall-escalation).
- **One-click Auto Router setup.** Configure automatically checks the chat model groups your proxy already serves and fills all four tiers, mixing providers when needed, without picking a template first. [#39693](https://github.com/BerriAI/litellm/pull/39693).

Posts: [Route on Context Size and Modality](/blog/auto-router-more-routing-configurations).
Posts: [Route on Context Size and Modality](/blog/auto-router-more-routing-configurations), [Mid-Task Stall Escalation](/blog/auto-router-stall-escalation).

## v1.100.0 (release candidate)

Expand Down
4 changes: 3 additions & 1 deletion docs/auto_router/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,12 @@ Already testing it? Share your results in [discussion #32168](https://github.com

<AutoRouterDiagram />

- **One-click setup.** Configure automatically checks the models your proxy already serves and fills all four tiers for you; no template to pick.
- **One model name in your clients.** The gateway classifies each request and picks the model.
- **Any model, any provider, per tier.** A single model, a random pool, or a Thompson-sampled pool.
- **Three classifiers.** Sub-millisecond heuristic scorer, a small LLM, or keyword rules.
- **Savings reported per request.** Against an all-frontier baseline, in the logs and in Cost Optimization.
- **Agent-safe.** Prompt caching, context-window escalation, modality routing, and optional session pinning.
- **Agent-safe.** Prompt caching, context-window escalation, modality routing, mid-task stall escalation, and optional session pinning.

## Results

Expand Down Expand Up @@ -123,6 +124,7 @@ items={[

## Release posts

- [Mid-Task Stall Escalation](/blog/auto-router-stall-escalation): bumps a request one tier when it's stuck in a retry loop
- [Auto Router v2](/blog/autorouter-v2): one router for complexity, semantic, and adaptive routing
- [1-click presets and Test Routing](/blog/auto-router-setup-and-testing)
- [Savings tab and per-request classifier cost](/blog/auto-router-spend-visibility)
Expand Down
Loading