Skip to content

docs(proxy): correct tag_routing_prefix's unprefixed-tag behavior - #1223

Open
deepanshululla wants to merge 1 commit into
BerriAI:mainfrom
deepanshululla:docs/tag-routing-unprefixed-noop
Open

docs(proxy): correct tag_routing_prefix's unprefixed-tag behavior#1223
deepanshululla wants to merge 1 commit into
BerriAI:mainfrom
deepanshululla:docs/tag-routing-unprefixed-noop

Conversation

@deepanshululla

Copy link
Copy Markdown
Contributor

Documents the corrected behavior from BerriAI/litellm#39903, fixing BerriAI/litellm#39901.

Adds to docs/proxy/tag_routing.md and docs/proxy/config_settings.md:

  • Once tag_routing_prefix is configured, an unprefixed request tag is a complete no-op: never matched, required, or excluded, and never a cause of no_deployments_with_tag_routing. The prior text ("keeps going through today's existing handling unchanged") described the bug this PR fixes, not the actual corrected behavior.
  • Key/team/project policy tags in metadata.inherited_tags are exempt from this scoping and keep applying regardless of prefix, since they are not caller-controlled.
  • A migration caution for operators already routing on plain unprefixed tags, since configuring tag_routing_prefix now stops that routing immediately for any caller not yet migrated to the prefix.

Verified with npm run build locally; no new broken links or anchors on the edited pages.

Previously documented as "keeps going through today's existing handling
unchanged", which described the bug fixed in PR #39903, not the corrected
behavior: once tag_routing_prefix is configured, an unprefixed tag is a
complete no-op, except key/team/project policy tags in inherited_tags, which
keep applying regardless of prefix. Also notes the migration implication for
operators already routing on plain unprefixed tags.
@vercel

vercel Bot commented Sep 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
litellm Ready Ready Preview Sep 5, 2026 4:09pm UTC

Request Review

@deepanshululla

Copy link
Copy Markdown
Contributor Author

Live verification against the fix in litellm PR #39903, real Anthropic and OpenAI API calls, no mocks.

Config used

model_list: chat -> anthropic/claude-haiku-4-5 (tags: default, provider:anthropic, allow_fail_open: true), chat -> openai/gpt-5.4-mini (tags: provider:openai). router_settings: enable_tag_filtering: true.

Quick example (tag_routing_prefix: "route:")

Command: curl http://localhost:4000/v1/chat/completions -d '{"model": "chat", ..., "metadata": {"tags": ["feature:demo", "route:!provider:openai"]}}'

Result: 200, x-litellm-model-name: anthropic/claude-haiku-4-5. feature:demo is inert, route:!provider:openai correctly excludes the OpenAI deployment.

Prefixed tags and the fail-open guard

Command: same, tags ["route:&provider:anthropic", "route:&custom-routing-key"]

Result: 200, x-litellm-model-name: anthropic/claude-haiku-4-5. custom-routing-key matches no deployment's tags but is prefix-marked, so allow_fail_open still proceeds instead of raising.

Migration caution, the new claim in this PR

With tag_routing_prefix: "route:" configured, tags: ["provider:openai"] (no prefix): 200, x-litellm-model-name: anthropic/claude-haiku-4-5, not OpenAI. The plain tag is now a no-op and falls through to the default-tagged deployment.

Same request with tag_routing_prefix: "" (baseline, prefix not configured): 200, x-litellm-model-name: openai/gpt-5.4-mini. The identical plain tag correctly selects the OpenAI deployment.

Conclusion: confirms the corrected unprefixed-tag no-op behavior, the inherited-tag exemption's sibling fail-open behavior, and the migration caution this PR adds are all accurate against the real code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant