docs(proxy): correct tag_routing_prefix's unprefixed-tag behavior - #1223
docs(proxy): correct tag_routing_prefix's unprefixed-tag behavior#1223deepanshululla wants to merge 1 commit into
Conversation
Previously documented as "keeps going through today's existing handling unchanged", which described the bug fixed in PR #39903, not the corrected behavior: once tag_routing_prefix is configured, an unprefixed tag is a complete no-op, except key/team/project policy tags in inherited_tags, which keep applying regardless of prefix. Also notes the migration implication for operators already routing on plain unprefixed tags.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Live verification against the fix in litellm PR #39903, real Anthropic and OpenAI API calls, no mocks. Config used model_list: chat -> anthropic/claude-haiku-4-5 (tags: default, provider:anthropic, allow_fail_open: true), chat -> openai/gpt-5.4-mini (tags: provider:openai). router_settings: enable_tag_filtering: true. Quick example (tag_routing_prefix: "route:") Command: curl http://localhost:4000/v1/chat/completions -d '{"model": "chat", ..., "metadata": {"tags": ["feature:demo", "route:!provider:openai"]}}' Result: 200, x-litellm-model-name: anthropic/claude-haiku-4-5. feature:demo is inert, route:!provider:openai correctly excludes the OpenAI deployment. Prefixed tags and the fail-open guard Command: same, tags ["route:&provider:anthropic", "route:&custom-routing-key"] Result: 200, x-litellm-model-name: anthropic/claude-haiku-4-5. custom-routing-key matches no deployment's tags but is prefix-marked, so allow_fail_open still proceeds instead of raising. Migration caution, the new claim in this PR With tag_routing_prefix: "route:" configured, tags: ["provider:openai"] (no prefix): 200, x-litellm-model-name: anthropic/claude-haiku-4-5, not OpenAI. The plain tag is now a no-op and falls through to the default-tagged deployment. Same request with tag_routing_prefix: "" (baseline, prefix not configured): 200, x-litellm-model-name: openai/gpt-5.4-mini. The identical plain tag correctly selects the OpenAI deployment. Conclusion: confirms the corrected unprefixed-tag no-op behavior, the inherited-tag exemption's sibling fail-open behavior, and the migration caution this PR adds are all accurate against the real code. |
Documents the corrected behavior from BerriAI/litellm#39903, fixing BerriAI/litellm#39901.
Adds to docs/proxy/tag_routing.md and docs/proxy/config_settings.md:
Verified with npm run build locally; no new broken links or anchors on the edited pages.