diff --git a/docs/auto_router/feature_history.md b/docs/auto_router/feature_history.md index eb5c2ad21..094ed8317 100644 --- a/docs/auto_router/feature_history.md +++ b/docs/auto_router/feature_history.md @@ -17,8 +17,10 @@ Merged after the v1.100.0 release candidate was cut. Available in `v1.101.0-dev` - **Shadow evals on teams and users.** Target a `key`, `team`, or `user`, so JWT-authenticated traffic can be evaluated. [#39015](https://github.com/BerriAI/litellm/pull/39015). - **Shadow evals across several routers.** Compare multiple router configs on one job's sampled traffic, paired. [#39028](https://github.com/BerriAI/litellm/pull/39028). - **1M context preset.** [#39490](https://github.com/BerriAI/litellm/pull/39490). +- **Mid-task stall escalation.** `stall_escalation_enabled: true` reads the assistant's own recent tool calls and bumps a request one tier when it's stuck in a retry loop, the same ladder `escalation_keywords` uses. Off by default. [#39809](https://github.com/BerriAI/litellm/pull/39809). [Post](/blog/auto-router-stall-escalation). +- **One-click Auto Router setup.** Configure automatically checks the chat model groups your proxy already serves and fills all four tiers, mixing providers when needed, without picking a template first. [#39693](https://github.com/BerriAI/litellm/pull/39693). -Posts: [Route on Context Size and Modality](/blog/auto-router-more-routing-configurations). +Posts: [Route on Context Size and Modality](/blog/auto-router-more-routing-configurations), [Mid-Task Stall Escalation](/blog/auto-router-stall-escalation). ## v1.100.0 (release candidate) diff --git a/docs/auto_router/index.md b/docs/auto_router/index.md index 06b84cd89..e3cf0bcc8 100644 --- a/docs/auto_router/index.md +++ b/docs/auto_router/index.md @@ -21,11 +21,12 @@ Already testing it? Share your results in [discussion #32168](https://github.com +- **One-click setup.** Configure automatically checks the models your proxy already serves and fills all four tiers for you; no template to pick. - **One model name in your clients.** The gateway classifies each request and picks the model. - **Any model, any provider, per tier.** A single model, a random pool, or a Thompson-sampled pool. - **Three classifiers.** Sub-millisecond heuristic scorer, a small LLM, or keyword rules. - **Savings reported per request.** Against an all-frontier baseline, in the logs and in Cost Optimization. -- **Agent-safe.** Prompt caching, context-window escalation, modality routing, and optional session pinning. +- **Agent-safe.** Prompt caching, context-window escalation, modality routing, mid-task stall escalation, and optional session pinning. ## Results @@ -123,6 +124,7 @@ items={[ ## Release posts +- [Mid-Task Stall Escalation](/blog/auto-router-stall-escalation): bumps a request one tier when it's stuck in a retry loop - [Auto Router v2](/blog/autorouter-v2): one router for complexity, semantic, and adaptive routing - [1-click presets and Test Routing](/blog/auto-router-setup-and-testing) - [Savings tab and per-request classifier cost](/blog/auto-router-spend-visibility)