Add embed model provider prefix for LiteLLM proxy compatability#2293
Merged
Conversation
Contributor
Greptile SummaryThis PR adds an optional
|
| Filename | Overview |
|---|---|
| nemo_retriever/src/nemo_retriever/common/api/util/string_processing/init.py | New prepend_model_provider_prefix helper: correctly handles None inputs, strips leading slashes on model name before prepending, and returns model unchanged when prefix is absent. |
| nemo_retriever/src/nemo_retriever/common/params/utils.py | Adds provider-prefix application logic to normalize_embed_kwargs; pops the key after applying so it is consumed exactly once. |
| nemo_retriever/src/nemo_retriever/models/inference/main_text_embed.py | Provider prefix threaded through the full async call chain; resolution cascade in create_text_embeddings_for_df checks task_config then transform config. Redundant or model_name fallback on line 269. |
| nemo_retriever/src/nemo_retriever/models/nim/util/init.py | Adds model_provider_prefix to infer_microservice; prefix applied only on the HTTP path, consistent with pre-PR behaviour for None inputs. |
| nemo_retriever/src/nemo_retriever/models/inference/runtime.py | Passes embed_model_provider_prefix to both TextEmbeddingConfig and the task_config dict. |
| nemo_retriever/src/nemo_retriever/service/vectordb_app.py | Correctly stores and forwards embed_model_provider_prefix through VectorDBState to infer_microservice. |
| nemo_retriever/src/nemo_retriever/models/inference/shared.py | Replaces dead-code endpoint-alias fallback with a proper normalize_embed_kwargs call. |
| nemo_retriever/src/nemo_retriever/service/services/pipeline_executor.py | Adds embed_model_provider_prefix to _TRUST_OWNED_EMBED_KEYS and propagates it into EmbedParams. |
| nemo_retriever/tests/test_embed_params.py | Five new tests covering prefix application scenarios including remote-only guard. |
| nemo_retriever/tests/test_service_vectordb_app.py | New test verifies _embed_queries_remote forwards the prefix to infer_microservice. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[CLI / YAML config\nembed_model_provider_prefix] --> B[build_embed_option_kwargs\nor build_embed_params]
B --> C{Endpoint\npresent?}
C -- Yes --> D[normalize_embed_kwargs\npops prefix, applies to\nmodel_name + embed_model_name]
C -- No --> E[prefix silently discarded\nlocal embedding unaffected]
D --> F[EmbedParams with prefixed model name]
F --> G{Embedding path}
G -- HTTP NIM --> H[infer_microservice]
G -- main_text_embed --> I[_http_embed_openai_compat]
G -- VectorDB query --> K[_embed_queries_remote]
H --> L[LiteLLM Proxy]
I --> L
K --> L
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[CLI / YAML config\nembed_model_provider_prefix] --> B[build_embed_option_kwargs\nor build_embed_params]
B --> C{Endpoint\npresent?}
C -- Yes --> D[normalize_embed_kwargs\npops prefix, applies to\nmodel_name + embed_model_name]
C -- No --> E[prefix silently discarded\nlocal embedding unaffected]
D --> F[EmbedParams with prefixed model name]
F --> G{Embedding path}
G -- HTTP NIM --> H[infer_microservice]
G -- main_text_embed --> I[_http_embed_openai_compat]
G -- VectorDB query --> K[_embed_queries_remote]
H --> L[LiteLLM Proxy]
I --> L
K --> L
Reviews (4): Last reviewed commit: "Change parameter name to embed_model_pro..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
ChrisJar
changed the title
jioffe502
approved these changes
Jul 6, 2026
ChrisJar
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Checklist