diff --git a/docs/docs/extraction/deployment-options.md b/docs/docs/extraction/deployment-options.md index 71f0b7510..d1c0d05f7 100644 --- a/docs/docs/extraction/deployment-options.md +++ b/docs/docs/extraction/deployment-options.md @@ -22,16 +22,9 @@ Build and run the NeMo Retriever service image with the [Docker service image gu 3. **Published Library Helm charts (supported):** cluster install and upgrade procedures are covered in the [NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) — use alongside the NeMo Retriever chart README for your release 4. [Environment variables](environment-config.md) and [Troubleshoot](troubleshoot.md) as needed -**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (see [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). See [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). +**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). - -For audio and video extraction in Kubernetes, set `service.installFfmpeg=true` -so the service container installs `ffmpeg` and `ffprobe` at startup. This -runtime install requires package-repository network egress, a writable root -filesystem, and security policy that allows the image's scoped sudo use. If -your cluster blocks startup package installation (for example air-gapped -environments), use a custom service image that already contains `ffmpeg` and -`ffprobe`, then set `service.image.repository` and `service.image.tag`. +For audio and video dependencies and OpenShift-specific Helm configuration, refer to [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-video-parakeet) and [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/openshift.md) in the Helm chart directory. ### I want examples and notebooks diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index 49ff109a5..1649d2bcf 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -11,8 +11,7 @@ Before you begin using [NeMo Retriever Library](overview.md), confirm your softw - For audio and video, `ffmpeg` and `ffprobe` must be on `PATH` (for example `sudo apt-get install -y --no-install-recommends ffmpeg` on Debian/Ubuntu). `ffmpeg-python` and `nemo-retriever[multimedia]` do not install these binaries. - On Helm with package-repo access, set `service.installFfmpeg=true`. For - air-gapped clusters, see [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment). + For container and Kubernetes guidance, refer to [Audio and video](audio-video.md). - For PDF extraction with `extract_method="nemotron_parse"`, install the Nemotron Parse client dependencies with `pip install "nemo-retriever[nemotron-parse]"` (pulls `open-clip-torch`, which provides the `open_clip` module required by the Nemotron Parse @@ -72,20 +71,20 @@ The production Helm chart enables these NIM microservices **by default** (for ex |-----------|-----|------| | `page_elements` | [nemotron-page-elements-v3](https://huggingface.co/nvidia/nemotron-page-elements-v3) | Page layout and element detection | | `table_structure` | [nemotron-table-structure-v1](https://huggingface.co/nvidia/nemotron-table-structure-v1) | Table structure extraction | -| `ocr` | [nemotron-ocr-v1](https://huggingface.co/nvidia/nemotron-ocr-v1) | Image OCR | +| `ocr` | [nemotron-ocr-v2](https://huggingface.co/nvidia/nemotron-ocr-v2) | Image OCR | | `vlm_embed` | [llama-nemotron-embed-vl-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-vl-1b-v2) | Multimodal (VL) embedding | ### OCR artifacts (Helm vs local Hugging Face) { #nemotron-ocr-v2-language-mode } !!! note - **Helm / NIM:** The production chart deploys **Nemotron OCR v1** under `nimOperator.ocr` (`nvcr.io/nim/nvidia/nemotron-ocr-v1:1.3.0`). For image defaults and upgrade notes, see [OCR NIM configuration](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#ocr-nim-configuration) in the Helm chart README. + **Helm / NIM:** The production chart deploys **Nemotron OCR v2** under `nimOperator.ocr` (`nvcr.io/nim/nvidia/nemotron-ocr-v2:1.4.0`). For image defaults and upgrade notes, refer to [OCR NIM configuration](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#ocr-nim-configuration) in the Helm chart README. - **Local Hugging Face inference:** When you deploy locally with HuggingFace model weights (for example `pip install "nemo-retriever[local]"` and GPU inference without remote OCR NIM URLs), the default OCR engine is **Nemotron OCR v2**, which runs in **multilingual** mode by default. For CLI flags and API parameters, see [Nemotron OCR v2 — language mode](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/docs/cli/README.md#nemotron-ocr-v2-language-mode). Remote OCR NIM endpoints use their own model and language behavior; local OCR language selectors are not sent on remote requests. + **Local Hugging Face inference:** When you deploy locally with HuggingFace model weights (for example `pip install "nemo-retriever[local]"` and GPU inference without remote OCR NIM URLs), the default OCR engine is **Nemotron OCR v2**, which runs in **multilingual** mode by default. For CLI flags and API parameters, refer to [Nemotron OCR v2 — language mode](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/docs/cli/README.md#nemotron-ocr-v2-language-mode). Remote OCR NIM endpoints use their own model and language behavior; local OCR language selectors are not sent on remote requests. Default OCR NIM container for release Helm deployments: -- **Image:** `nvcr.io/nim/nvidia/nemotron-ocr-v1:1.3.0` +- **Image:** `nvcr.io/nim/nvidia/nemotron-ocr-v2:1.4.0` Default VL embedder container and model for release deployments: @@ -94,7 +93,7 @@ Default VL embedder container and model for release deployments: ### Optional Helm NIMs (not auto-wired) { #optional-helm-nims-not-auto-wired-by-default } -These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For **26.05 production**, disable keys you do not need (see [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator..enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#nim-operator-sub-stack). +These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For **26.05 production**, disable keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator..enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#nim-operator-sub-stack). | Helm flag | NIM | Role | |-----------|-----|------| @@ -127,7 +126,7 @@ For published NIM model IDs and deployment-specific constraints, use the product NeMo Retriever Library supports the following GPU hardware given system constraints in the table. - **HF model weights** — approximate Hugging Face checkpoint footprint (files such as `model*.safetensors`, `weights.pth`, or other published weight bundles in the model repository). Values are rounded from the current public file listing and can change when the repository is updated. -- **NIM disk space** — approximate container and on-disk model cache for self-hosted NIM microservices (not the same as HF download size). For Nemotron 3 Nano Omni captioning, see the [NVIDIA NIM for Vision Language Models support matrix](https://docs.nvidia.com/nim/vision-language-models/latest/support-matrix.html#nemotron-3-nano-omni-30b-a3b-reasoning). +- **NIM disk space** — approximate container and on-disk model cache for self-hosted NIM microservices (not the same as HF download size). For Nemotron 3 Nano Omni captioning, refer to the [NVIDIA NIM for Vision Language Models support matrix](https://docs.nvidia.com/nim/vision-language-models/latest/support-matrix.html#nemotron-3-nano-omni-30b-a3b-reasoning). Model repositories and NIM references are linked in [Core and Advanced Pipeline Features](#core-and-advanced-pipeline-features) above. diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 6300e6c41..3f59f0a8f 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -84,6 +84,7 @@ nav: - "Authentication and API keys": extraction/api-keys.md - "3. Deployment options": - "Compare deployment options": extraction/deployment-options.md + - "OpenShift deployment (Helm)": https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/openshift.md - "Helm chart (Kubernetes)": https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/helm - "Docker service image": https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/docker.md - "4. Core workflows": diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 17c22848e..3aab0396c 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -50,6 +50,7 @@ nemo_retriever/helm/ ├── Chart.yaml ├── values.yaml ├── README.md <-- this file +├── openshift.md <-- OpenShift restricted-v2 install guide ├── .helmignore └── templates/ ├── _helpers.tpl @@ -78,15 +79,15 @@ nemo_retriever/helm/ ## Quick start -### 1. Service image +### 1. Service image { #1-service-image } -The chart defaults to the staging image published to NGC: +The chart defaults to the GA image published to NGC: ``` -nvcr.io/nvstaging/nim/nemo-retriever-service:043020205-001 +nvcr.io/nvidia/nemo-microservices/nrl-service:26.5.0 ``` -Pulling from `nvcr.io/nvstaging` requires an NGC pull secret — either set +Pulling from `nvcr.io` requires an NGC pull secret — either set `ngcImagePullSecret.create=true` (see below) or pre-create one in the namespace named `ngc-secret`. @@ -188,7 +189,7 @@ NIM (the VL reranker `rerankqa`, Nemotron Parse, Omni 30B, and the Parakeet `audio` ASR NIM) is **disabled by default** to honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md); -see [Recommended minimal install (26.05)](#recommended-minimal-install-2605) +refer to [Recommended minimal install (26.05)](#recommended-minimal-install-2605) for the opt-in `--set` flags that turn any of them on. ```bash @@ -219,7 +220,7 @@ helm install retriever ./nemo_retriever/helm \ > * Answer generation LLM — `--set nimOperator.answer_llm.enabled=true` > * Parakeet ASR — `--set nimOperator.audio.enabled=true` (also set `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` to wire ASR into the service, plus `service.installFfmpeg=true` if your image does not bundle ffmpeg) > -> This matches the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and avoids silently pulling ≈ 62 GiB of Omni weights, loading a large two-GPU LLM, or claiming extra dedicated GPUs on a "default" install. See the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table for per-NIM GPU and disk costs. +> This matches the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and avoids silently pulling ≈ 62 GiB of Omni weights, loading a large two-GPU LLM, or claiming extra dedicated GPUs on a "default" install. Refer to the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table for per-NIM GPU and disk costs. The chart auto-wires the operator-managed in-cluster URLs of the four "core" NIMs into the service's `nim_endpoints` block: @@ -287,8 +288,8 @@ short list of knobs you'll touch first. | Path | Default | Notes | |-------------------------------|------------------------------------|-------| -| `service.image.repository` | `localhost:32000/nemo-retriever-service` | Override to a published image. | -| `service.image.tag` | `latest` | | +| `service.image.repository` | `nvcr.io/nvidia/nemo-microservices/nrl-service` | GA NGC image; override to pin a different build or use a local registry. | +| `service.image.tag` | `26.5.0` | | | `service.replicas` | `1` | Hard cap = 1 while SQLite is the backend. | | `service.installFfmpeg` | `false` | Install `ffmpeg`/`ffprobe` at container startup by setting `INSTALL_FFMPEG=true`. Requires network egress, writable root filesystem, and sudo/setuid allowed. Not for air-gapped clusters — use a custom image instead. | | `service.resources.requests` | `16 / 16Gi` | Tune in tandem with `serviceConfig.pipeline.*Workers`. | @@ -296,19 +297,20 @@ short list of knobs you'll touch first. | `service.gpu.enabled` | `false` | The service does **not** need a GPU. | For audio and video extraction, set `service.installFfmpeg=true` when your -cluster allows runtime package installation. For air-gapped clusters, see -[Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment). +cluster allows runtime package installation. **OpenShift restricted-v2** blocks +that path — use a prebuilt service image instead; refer to [Audio and video on restricted OpenShift](./openshift.md#audio-and-video-ffmpeg-on-restricted-openshift). +For air-gapped clusters, refer to [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment). ### Audio and video (Parakeet ASR) { #audio-video-parakeet } To run self-hosted Parakeet for [audio and video extraction](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md): 1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install (26.05)](#recommended-minimal-install-2605)). -2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (see [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). +2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (refer to [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). 3. Confirm the GPU SKU in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) (footnote ⁴ lists Blackwell limitations). -4. Set `service.installFfmpeg=true` when the retriever service will process audio or video (see `service.installFfmpeg` above). +4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (refer to `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](./openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead. -The retriever service picks up the in-cluster ASR endpoint when `nimOperator.audio` is enabled; see [NIM Operator sub-stack](#nim-operator-sub-stack). +The retriever service picks up the in-cluster ASR endpoint when `nimOperator.audio` is enabled; refer to [NIM Operator sub-stack](#nim-operator-sub-stack). ### Service configuration (rendered into `retriever-service.yaml`) @@ -316,8 +318,8 @@ The retriever service picks up the in-cluster ASR endpoint when `nimOperator.aud |---------------------------------------------------|---------|-------| | `serviceConfig.server.port` | `7670` | Container + Service port. | | `serviceConfig.pipeline.realtimeWorkers` | `24` | Per-pod realtime worker count. | -| `serviceConfig.pipeline.batchWorkers` | `48` | Per-pod batch worker count. See [Timeouts and alleviating ingest failures](#timeouts-and-alleviating-ingest-failures) if embed or pool errors appear under load. | -| `serviceConfig.nimEndpoints.*InvokeUrl` | `""` | Override the auto-resolved NIM Operator URL. Available knobs: `pageElementsInvokeUrl`, `tableStructureInvokeUrl`, `ocrInvokeUrl`, `embedInvokeUrl`, and `captionInvokeUrl` (see [Image captioning (Omni 30B)](#image-captioning-omni-30b)). | +| `serviceConfig.pipeline.batchWorkers` | `48` | Per-pod batch worker count. Refer to [Timeouts and alleviating ingest failures](#timeouts-and-alleviating-ingest-failures) if embed or pool errors appear under load. | +| `serviceConfig.nimEndpoints.*InvokeUrl` | `""` | Override the auto-resolved NIM Operator URL. Available knobs: `pageElementsInvokeUrl`, `tableStructureInvokeUrl`, `ocrInvokeUrl`, `embedInvokeUrl`, and `captionInvokeUrl` (refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b)). | | `serviceConfig.nimEndpoints.captionModelName` | `""` | Model id sent to the remote VLM. Auto-set to `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` whenever a caption URL is resolved. | | `serviceConfig.llm.enabled` | `false` | Enables `POST /v1/answer`. Auto-flips to true when `nimOperator.answer_llm` is enabled and the operator URL resolves. | | `serviceConfig.llm.apiBase` | `""` | OpenAI-compatible LLM base URL. Explicit value wins; otherwise `answer_llm` opt-in resolves to `http://answer-llm:8000/v1` by default. | @@ -326,7 +328,7 @@ The retriever service picks up the in-cluster ASR endpoint when `nimOperator.aud | `serviceConfig.llm.model` | `""` | Optional explicit LiteLLM model id. Leave empty to inherit `nimOperator.answer_llm.model` when using the operator-managed answer LLM; set it for external endpoints. | | `serviceConfig.llm.ragSystemPromptPrefix` | `""` | Optional explicit RAG prompt prefix. Leave empty unless an endpoint needs model-specific prompt directives. | | `serviceConfig.llm.reasoningEnabled` | `true` | Request-level reasoning toggle for `/v1/answer`. Defaults to true for external OpenAI-compatible providers; set false for Nemotron endpoints that should receive portable no-reasoning controls. | -| `serviceConfig.vectordb.enabled` | `true` | Deploy the LanceDB vectordb Pod. When `true` the chart **requires** a resolvable embed endpoint (see [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint)); `helm install` / `helm upgrade` fails fast otherwise. | +| `serviceConfig.vectordb.enabled` | `true` | Deploy the LanceDB vectordb Pod. When `true` the chart **requires** a resolvable embed endpoint (refer to [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint)); `helm install` / `helm upgrade` fails fast otherwise. | | `serviceConfig.vectordb.lancedbUri` | `/data/vectordb` | LanceDB on the vectordb Pod's PVC. | | `serviceConfig.vectordb.embedModel` | `nvidia/llama-nemotron-embed-vl-1b-v2` | Passed to vectordb + worker `embed_model_name`. | @@ -353,7 +355,7 @@ resolved. Pick one of: 3. --set serviceConfig.vectordb.enabled=false ``` -Resolution order matches the rest of the chart (see [Mix and match NIM +Resolution order matches the rest of the chart (refer to [Mix and match NIM sources](#3-install-with-the-nim-operator-in-cluster-nims)): 1. Explicit `serviceConfig.nimEndpoints.embedInvokeUrl` always wins. @@ -469,8 +471,8 @@ pair gated on three conditions ALL holding: | `nimOperator.vlm_embed.image` | `nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:1.12.0` | Default VLM embed NIM image. | | `nimOperator.rerankqa.enabled` | `false` | VL reranker NIM (optional; not auto-wired). Set `true` to opt in. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the **VL** SKU (`llama-nemotron-rerank-vl-1b-v2`) per [prerequisites-support-matrix.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#default-helm-nims) — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR. | | `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `extract_method="nemotron_parse"`. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). Image tag follows the [image tag conventions](#image-tag-conventions). | -| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — see [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so 26.05 installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | -| `nimOperator.answer_llm.enabled` | `false` | Generic answer-generation LLM NIM (optional; Super-49B defaults). Set `true` to enable `/v1/answer` — see [Answer generation (operator-managed LLM)](#answer-generation-llm). Default `false` so installs do not silently claim answer-generation GPUs. | +| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so 26.05 installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | +| `nimOperator.answer_llm.enabled` | `false` | Generic answer-generation LLM NIM (optional; Super-49B defaults). Set `true` to enable `/v1/answer` — refer to [Answer generation (operator-managed LLM)](#answer-generation-llm). Default `false` so installs do not silently claim answer-generation GPUs. | | `nimOperator.answer_llm.model` | `openai/nvidia/llama-3.3-nemotron-super-49b-v1.5` | LiteLLM/OpenAI model id inherited by `serviceConfig.llm.model` when the operator-managed answer LLM is enabled and no explicit service model is set. | | `nimOperator.answer_llm.ragSystemPromptPrefix` | `""` | Optional prompt prefix inherited by `serviceConfig.llm.ragSystemPromptPrefix` only when explicitly set. Leave empty to keep the operator-managed LLM model-neutral and use `serviceConfig.llm.reasoningEnabled` for request-level reasoning control. | | `nimOperator.audio.enabled` | `false` | Parakeet ASR NIM (optional). Set `true` for audio/video transcription; pair with `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` so the retriever-service can reach it. | @@ -479,10 +481,10 @@ pair gated on three conditions ALL holding: | `nimOperator..authSecret` | `ngc-api` | NIM auth Secret name. | | `nimOperator..storage.pvc.size` | `25Gi` (50Gi for vlm_embed/rerankqa, 100Gi parse, 300Gi VL) | NIMCache PVC size. | | `nimOperator..replicas` | `1` | Per-NIMService replica count. | -| `nimOperator.nimServiceGpuLimit` | `1` | Default `nvidia.com/gpu` limit on every NIMService when per-NIM `resources` is `{}`. Set to `null` for operator-only reconciliation (not reliable on all NIM Operator versions — see [GPU limits and `helm upgrade`](#gpu-limits-and-helm-upgrade)). | +| `nimOperator.nimServiceGpuLimit` | `1` | Default `nvidia.com/gpu` limit on every NIMService when per-NIM `resources` is `{}`. Set to `null` for operator-only reconciliation (not reliable on all NIM Operator versions — refer to [GPU limits and `helm upgrade`](#gpu-limits-and-helm-upgrade)). | | `nimOperator..resources` | `{}` | Per-NIM override of the whole `resources` block. Empty uses `nimServiceGpuLimit`; non-empty replaces the chart default (may require `--force-conflicts` on later `helm upgrade`). | -| `nimOperator.modelProfile` | `{}` | Chart-wide NIMCache GPU/profile filter. Applied to every NIMCache that does not have its own override. See [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | -| `nimOperator..modelProfile` | `{}` | Per-NIM NIMCache GPU/profile filter. Non-empty values REPLACE the chart-wide default (no merge). See [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | +| `nimOperator.modelProfile` | `{}` | Chart-wide NIMCache GPU/profile filter. Applied to every NIMCache that does not have its own override. Refer to [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | +| `nimOperator..modelProfile` | `{}` | Per-NIM NIMCache GPU/profile filter. Non-empty values REPLACE the chart-wide default (no merge). Refer to [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | | `nimOperator..expose.service.port` | `8000` (9000 for audio) | HTTP port. | | `nimOperator..expose.service.grpcPort` | `8001` (50051 for audio) | gRPC port. | @@ -1035,156 +1037,13 @@ sanity check before opening Grafana. ## OpenShift deployment { #openshift-deployment } -The chart defaults target generic Kubernetes clusters that allow fixed numeric -UIDs (`runAsUser` / `runAsGroup` / `fsGroup` **1000**). **OpenShift 4.x** -namespaces under the default **restricted-v2** Security Context Constraint (SCC) -and **Pod Security Admission (PSA) `restricted`** profile assign a per-namespace -UID/GID range instead. A stock `helm install` without overrides therefore fails -SCC validation, emits PSA warnings, or crashes on log paths the random UID cannot -write. - -We do **not** change chart defaults for OpenShift-only behavior (that would affect -other platforms). Use the overrides below on OpenShift, or save the YAML block -into a local values file and pass `-f `. - -### Cluster posture (typical QA / hardened namespaces) - -| Control | Typical default on a new OpenShift project | -| --- | --- | -| SCC | **restricted-v2** (first match in priority order) | -| PSA | `pod-security.kubernetes.io/warn=restricted` (and often `audit=restricted`; `enforce` may be unset on dev clusters) | -| UID assignment | SCC injects `runAsUser` / `fsGroup` from the namespace range (for example `1000750000–1000759999`) | - -On clusters with **PSA `enforce=restricted`**, missing container `securityContext` -fields become hard rejections, not warnings. - -### Override reference (maps to chart limitations) - -| Symptom on stock install | Cause | Helm override | -| --- | --- | --- | -| `FailedCreate`: UID/GID **1000** not in namespace range | Hardcoded `service.podSecurityContext` UID/GID/fsGroup | Omit `runAsUser`, `runAsGroup`, and `fsGroup`; keep only `runAsNonRoot: true` | -| PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (see sample below) | -| `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | -| `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Disable vectordb for smoke tests, or patch the vectordb Deployment after install (below) | -| PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | - -### Recommended value overrides - -```yaml -# OpenShift overrides for nemo-retriever Helm chart (restricted-v2 / PSA restricted). -# Save locally, then: helm install retriever ./nemo_retriever/helm -f .yaml ... - -service: - podSecurityContext: - runAsNonRoot: true - # Do NOT set runAsUser, runAsGroup, or fsGroup — OpenShift SCC assigns them. - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - seccompProfile: - type: RuntimeDefault - -serviceConfig: - logging: - # Writable without persistence PVC (chart always mounts emptyDir at /tmp). - file: /tmp/retriever-service.log - vectordb: - # Set false for minimal service-only validation; see vectordb patch below if enabled. - enabled: false - -topology: - otel: - enabled: false -``` - -When **`persistence.enabled=true`**, you can keep the default log path under -`persistence.mountPath` (`/var/lib/nemo-retriever`) because the PVC is mounted and -SCC-assigned `fsGroup` applies. When persistence is off, always relocate logs to -`/tmp` (or another path backed by `service.extraVolumes`). - -### Example install on OpenShift 4.20 (service-only smoke test) - -Matches QA validation with external NIMs disabled, no persistence, and no results -PVC: - -```bash -oc new-project nemo-retriever - -oc create secret docker-registry ngc-secret -n nemo-retriever \ - --docker-server=nvcr.io --docker-username='$oauthtoken' \ - --docker-password="$NGC_API_KEY" - -oc create secret generic ngc-api -n nemo-retriever \ - --from-literal=NGC_API_KEY="$NGC_API_KEY" \ - --from-literal=NGC_CLI_API_KEY="$NGC_API_KEY" - -helm install retriever ./nemo_retriever/helm -n nemo-retriever \ - -f .yaml \ - --set ngcImagePullSecret.create=false \ - --set ngcApiSecret.create=false \ - --set nims.enabled=false \ - --set persistence.enabled=false \ - --set retrieverResults.enabled=false -``` - -Verify pods: - -```bash -oc get pods -n nemo-retriever -oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever -``` - -You should see SCC-assigned numeric `runAsUser` on containers that declare a -`securityContext` block, and no PSA warnings once overrides are applied. - -### Enabling the vectordb Deployment on OpenShift - -`serviceConfig.vectordb.enabled=true` renders a **vectordb** container from the -same image (`USER nemo`, non-numeric). The chart does not yet expose a -`securityContext` value for that container. After `helm install`, patch the -Deployment so OpenShift can inject a numeric UID into the container spec: - -```bash -RELEASE=retriever -NS=nemo-retriever -VDB_DEPLOY="${RELEASE}-nemo-retriever-vectordb" - -oc patch deployment "$VDB_DEPLOY" -n "$NS" --type=json -p='[ - {"op": "add", "path": "/spec/template/spec/containers/0/securityContext", "value": { - "allowPrivilegeEscalation": false, - "capabilities": {"drop": ["ALL"]}, - "runAsNonRoot": true, - "seccompProfile": {"type": "RuntimeDefault"} - }} -]' -``` - -Re-apply the patch after `helm upgrade` if the Deployment is recreated. A future -chart release may add first-class `topology.vectordb.securityContext` values. - -### Enabling the OpenTelemetry collector on OpenShift - -The chart’s otel-collector Deployment likewise lacks `securityContext` fields. -Prefer `topology.otel.enabled=false` (as in the sample values) unless you operate -your own collector or patch `*-otel` the same way as vectordb. - -### What we intentionally do not require on OpenShift - -Do **not** bind the namespace to **anyuid** SCC or set PSA `enforce=privileged` -unless your security team explicitly approves it. The overrides above are intended -to keep **restricted-v2** / PSA **restricted** posture. - -### Related documentation - -- [Pre-Requisites & Support Matrix](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md) -- [Deployment options](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) +OpenShift install procedures, **restricted-v2** / PSA **restricted** value overrides, prebuilt `ffmpeg` images, internal registry pull secrets, optional NIM `LD_LIBRARY_PATH` tuning, and install examples are in **[OpenShift deployment](./openshift.md)**. Pass `-f openshift-restricted.yaml` from that guide when you install on OpenShift. --- ## Air-gapped deployment { #air-gapped-deployment } -See [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment) for overview and workflow. Chart-specific reference for mirroring: +Refer to [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment) for overview and workflow. Chart-specific reference for mirroring: ### Container images to mirror (26.05 chart defaults) @@ -1232,7 +1091,8 @@ imagePullSecrets: - name: my-private-registry ngcImagePullSecret: - create: false # use secrets that authenticate to YOUR mirror + create: false + name: "" # Explicitly empty — clears the default "ngc-secret" nimOperator: page_elements: @@ -1243,8 +1103,8 @@ nimOperator: # Repeat for table_structure, ocr, vlm_embed, and any optional keys you enable. ``` -- Set `nimOperator..image.pullSecrets` to the Secret name your - `NIMService` resources should use (defaults to `ngc-secret`). +- Set `nimOperator..image.pullSecrets` to your mirror pull secret + (for example `my-private-registry`; chart default is `ngc-secret`). - Leave `serviceConfig.nimEndpoints.*` empty when operator-managed NIMs are in-cluster; set explicit URLs only for external or mirrored services outside the chart. @@ -1306,7 +1166,7 @@ helm template r nemo_retriever/helm \ ``` Both renders should succeed cleanly and parse as valid Kubernetes manifests -(`kubectl apply --dry-run=client -f /tmp/r.yaml`). See [VectorDB and the +(`kubectl apply --dry-run=client -f /tmp/r.yaml`). Refer to [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint) for why `helm template r nemo_retriever/helm` without flags is rejected as a misconfiguration. diff --git a/nemo_retriever/helm/openshift.md b/nemo_retriever/helm/openshift.md new file mode 100644 index 000000000..cda227314 --- /dev/null +++ b/nemo_retriever/helm/openshift.md @@ -0,0 +1,261 @@ +# OpenShift deployment + +Use this guide when you install the [NeMo Retriever Helm chart](./README.md) on **OpenShift 4.x** with the default **restricted-v2** Security Context Constraint (SCC) and **Pod Security Admission (PSA) `restricted`** profile. + +For general Kubernetes and Helm deployment choices, refer to [Deployment options](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/). For chart values and NIM wiring, refer to the [Helm chart README](./README.md). + +## Overview { #openshift-deployment } + +The chart defaults target generic Kubernetes clusters that allow fixed numeric UIDs (`runAsUser` / `runAsGroup` / `fsGroup` **1000**). OpenShift namespaces under **restricted-v2** assign a per-namespace UID/GID range instead. A stock `helm install` without overrides therefore fails SCC validation, emits PSA warnings, or crashes on log paths the random UID cannot write. + +We do **not** change chart defaults for OpenShift-only behavior (that would affect other platforms). Use the overrides below, or save the YAML block into a local values file and pass `-f ` on every `helm install` / `helm upgrade`. + +### Cluster posture (typical hardened namespaces) + +| Control | Typical default on a new OpenShift project | +| --- | --- | +| SCC | **restricted-v2** (first match in priority order) | +| PSA | `pod-security.kubernetes.io/warn=restricted` (and often `audit=restricted`; `enforce` may be unset on dev clusters) | +| UID assignment | SCC injects `runAsUser` / `fsGroup` from the namespace range (for example `1000750000–1000759999`) | + +On clusters with **PSA `enforce=restricted`**, missing container `securityContext` fields become hard rejections, not warnings. + +### Override reference (maps to chart limitations) + +| Symptom on stock install | Cause | Helm override | +| --- | --- | --- | +| `FailedCreate`: UID/GID **1000** not in namespace range | Hardcoded `service.podSecurityContext` UID/GID/fsGroup | Omit `runAsUser`, `runAsGroup`, and `fsGroup`; keep only `runAsNonRoot: true` | +| PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (refer to the sample below) | +| `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | +| `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Set `serviceConfig.vectordb.enabled=false` for a minimal service-only install, or patch the vectordb Deployment after install (below) | +| PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | +| Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (refer to [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | +| `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart-rendered `imagePullSecrets` may omit the namespace SA `dockercfg` secret required for internal-registry pulls | List every required pull secret under `imagePullSecrets` (refer to [Internal registry pull secrets](#internal-registry-pull-secrets)) | +| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths through `nimOperator..env` (refer to [Optional NIM runtime environment](#optional-nim-runtime-environment)) | + +### Recommended value overrides + +Save the block below as a local values file (for example `openshift-restricted.yaml`) and pass `-f openshift-restricted.yaml` on every `helm install` / `helm upgrade`. Use this profile for **restricted-v2** / PSA **restricted** namespaces with the NIM Operator, a prebuilt `ffmpeg` service image when you need [audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md), and optional NIM env overrides from later sections. + +```yaml +# OpenShift overrides for nemo-retriever Helm chart (restricted-v2 / PSA restricted). +# helm install retriever ./nemo_retriever/helm -f openshift-restricted.yaml ... + +service: + podSecurityContext: + runAsNonRoot: true + # Do NOT set runAsUser, runAsGroup, or fsGroup — OpenShift SCC assigns them. + securityContext: + runAsNonRoot: true + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + seccompProfile: + type: RuntimeDefault + # Leave installFfmpeg false on restricted-v2; use a custom image for audio/video. + installFfmpeg: false + +serviceConfig: + logging: + # Writable without persistence PVC (chart always mounts emptyDir at /tmp). + file: /tmp/retriever-service.log + vectordb: + # Set false for a minimal service-only install; refer to the vectordb patch below if you enable it. + enabled: false + +topology: + otel: + enabled: false +``` + +When **`persistence.enabled=true`**, you can keep the default log path under `persistence.mountPath` (`/var/lib/nemo-retriever`) because the PVC is mounted and SCC-assigned `fsGroup` applies. When persistence is off, always relocate logs to `/tmp` (or another path backed by `service.extraVolumes`). + +### Audio and video (ffmpeg) on restricted OpenShift { #audio-and-video-ffmpeg-on-restricted-openshift } + +The Helm chart supports `service.installFfmpeg=true`, which installs `ffmpeg`/`ffprobe` at container startup through passwordless `sudo`. On OpenShift **restricted-v2** SCC, that path is blocked: PSA restricted sets `allowPrivilegeEscalation: false` and the SCC enforces **no-new-privileges**, so the entrypoint cannot elevate to install packages. + +For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. Instead, extend the service image on a connected build host and point the chart at that tag (same pattern as [air-gapped custom service images](./README.md#1-service-image)): + +```dockerfile +FROM nvcr.io/nvidia/nemo-microservices/nrl-service: +USER root +RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \ + && rm -rf /var/lib/apt/lists/* +USER nemo +``` + +Push the result to NGC, your private registry, or the [OpenShift internal registry](#internal-registry-pull-secrets), then set `service.image.repository` / `service.image.tag` accordingly. + +### Internal registry pull secrets { #internal-registry-pull-secrets } + +When you rebuild the service image into the OpenShift internal registry (`image-registry.openshift-image-registry.svc:5000/...`), pods normally pull through the namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart renders an explicit `imagePullSecrets` list on every Pod, so you must list **every** secret each Pod needs—including the SA `dockercfg` secret when `service.image` points at the internal registry. + +If the rendered list contains only `ngc-secret` (for NGC NIM images) while `service.image` uses the internal registry, the service Pod can fail with `ImagePullBackOff`. + +**Internal-registry service image only** — clear the chart-managed NGC pull secret name so the helper does not inject `ngc-secret` alone: + +```yaml +ngcImagePullSecret: + create: false + name: "" # Explicitly empty — clears the default "ngc-secret" + +imagePullSecrets: + - name: default-dockercfg-xxxxx # replace with your SA secret (section below) +``` + +**Both NGC NIMs and an internal-registry service image** — list every secret the pods need: + +```yaml +ngcImagePullSecret: + create: false + name: ngc-secret + +imagePullSecrets: + - name: ngc-secret + - name: default-dockercfg-xxxxx +``` + +Look up the ServiceAccount pull secret name: + +```bash +oc get sa default -n nemo-retriever -o jsonpath='{.imagePullSecrets[*].name}{"\n"}' +``` + +Pre-create `ngc-secret` / `ngc-api` with `oc create secret` when `ngcImagePullSecret.create=false` and `ngcApiSecret.create=false`, as in the install examples below. + +### Optional NIM runtime environment { #optional-nim-runtime-environment } + +On some OpenShift clusters with the NIM Operator, **optional** NIM pods (for example Parakeet ASR or Omni caption) need an explicit `LD_LIBRARY_PATH` so CUDA/driver libraries from the GPU Operator stack are visible at runtime. Symptoms are `CrashLoopBackOff` and log lines referencing a missing `.so` (for example `libcudart` or `libcudnn`). + +Helm **replaces** the whole `env` list when you override `nimOperator..env` in a values file. Copy the chart defaults from `values.yaml` for that NIM, then append `LD_LIBRARY_PATH`: + +```yaml +nimOperator: + audio: + env: + # Retain chart defaults, then append LD_LIBRARY_PATH. + - name: NIM_TAGS_SELECTOR + value: "name=parakeet-1-1b-ctc-en-us,mode=ofl,vad=default,diarizer=disabled" + - name: NIM_TRITON_LOG_VERBOSE + value: "1" + - name: LD_LIBRARY_PATH + value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" + nemotron_3_nano_omni_30b_a3b_reasoning: + env: + # Retain chart defaults, then append LD_LIBRARY_PATH. + - name: NIM_HTTP_API_PORT + value: "8000" + - name: NIM_TRITON_LOG_VERBOSE + value: "1" + - name: LD_LIBRARY_PATH + value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" +``` + +Paths vary with GPU Operator version and node image. Inspect a healthy GPU workload on the same cluster (`oc exec` into a CUDA sample pod) or the failing NIM pod's filesystem before pinning production values. + +### Call the Omni NIM directly { #omni-caption-manual-smoke-testing } + +The retriever service caption profile already sends `chat_template_kwargs.enable_thinking=false` to the Omni NIM during ingest. To confirm the in-cluster Omni NIM returns caption text in `message.content`, call `/v1/chat/completions` directly (for example with `curl` against the `NIMService`) and include the same flag: + +```json +{ + "model": "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning", + "messages": [{"role": "user", "content": "..."}], + "chat_template_kwargs": {"enable_thinking": false} +} +``` + +For pipeline scope (PDF chart regions are not captioned), refer to [Image captioning (support matrix)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#image-captioning-2605) and [Image captioning (pipeline scope)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/multimodal-extraction.md#image-captioning) in the extraction docs. + +### Example install (service only, no in-cluster NIMs) + +Use this flow when you want only the retriever service pod: disable the in-cluster NIM Operator stack (`nims.enabled=false`), skip the persistence PVC, and skip the results PVC. Pre-create NGC pull/API secrets, then install with the restricted OpenShift values file: + +```bash +oc new-project nemo-retriever + +oc create secret docker-registry ngc-secret -n nemo-retriever \ + --docker-server=nvcr.io --docker-username='$oauthtoken' \ + --docker-password="$NGC_API_KEY" + +oc create secret generic ngc-api -n nemo-retriever \ + --from-literal=NGC_API_KEY="$NGC_API_KEY" \ + --from-literal=NGC_CLI_API_KEY="$NGC_API_KEY" + +helm install retriever ./nemo_retriever/helm -n nemo-retriever \ + -f openshift-restricted.yaml \ + --set ngcImagePullSecret.create=false \ + --set ngcApiSecret.create=false \ + --set nims.enabled=false \ + --set persistence.enabled=false \ + --set retrieverResults.enabled=false \ + --set service.image.repository=nvcr.io/nvidia/nemo-microservices/nrl-service \ + --set service.image.tag=26.5.0 +``` + +Verify pods: + +```bash +oc get pods -n nemo-retriever +oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever +``` + +You should see SCC-assigned numeric `runAsUser` on containers that declare a `securityContext` block, and no PSA warnings after overrides are applied. + +### Example install with NIM Operator (in-cluster NIMs) + +After you install the NIM Operator and GPU Operator, reuse `openshift-restricted.yaml` and the NGC secrets from the service-only example. Point `service.image` at a **ffmpeg-enabled** build when you use [audio and video](#audio-and-video-ffmpeg-on-restricted-openshift). Add [optional NIM `LD_LIBRARY_PATH`](#optional-nim-runtime-environment) overrides if ASR or Omni pods crash on missing libraries. + +```bash +helm install retriever ./nemo_retriever/helm -n nemo-retriever \ + -f openshift-restricted.yaml \ + --set ngcImagePullSecret.create=false \ + --set ngcApiSecret.create=false \ + --set service.image.repository=nvcr.io/nvidia/nemo-microservices/nrl-service \ + --set service.image.tag=26.5.0 +``` + +After install, confirm workloads reach Ready before you run ingest: + +```bash +oc get pods -n nemo-retriever +oc get nimservice -n nemo-retriever +``` + +When you enable optional NIMs, wait for Parakeet ASR and Omni caption pods to become Ready before you run [audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md) or caption extraction. If ASR or Omni pods crash with missing `.so` errors, apply the [LD_LIBRARY_PATH overrides](#optional-nim-runtime-environment) above and upgrade the release. + +### Enabling the vectordb Deployment on OpenShift + +`serviceConfig.vectordb.enabled=true` renders a **vectordb** container from the same image (`USER nemo`, non-numeric). The chart does not yet expose a `securityContext` value for that container. After `helm install`, patch the Deployment so OpenShift can inject a numeric UID into the container spec: + +```bash +RELEASE=retriever +NS=nemo-retriever +VDB_DEPLOY="${RELEASE}-nemo-retriever-vectordb" + +oc patch deployment "$VDB_DEPLOY" -n "$NS" --type=json -p='[ + {"op": "add", "path": "/spec/template/spec/containers/0/securityContext", "value": { + "allowPrivilegeEscalation": false, + "capabilities": {"drop": ["ALL"]}, + "runAsNonRoot": true, + "seccompProfile": {"type": "RuntimeDefault"} + }} +]' +``` + +Re-apply the patch after `helm upgrade` if the Deployment is recreated. A future chart release may add first-class `topology.vectordb.securityContext` values. + +### Enabling the OpenTelemetry collector on OpenShift + +The chart's otel-collector Deployment likewise lacks `securityContext` fields. Prefer `topology.otel.enabled=false` (as in the sample values) unless you operate your own collector or patch `*-otel` the same way as vectordb. + +### What we intentionally do not require on OpenShift + +Do **not** bind the namespace to **anyuid** SCC or set PSA `enforce=privileged` unless your security team explicitly approves it. The overrides above are intended to keep **restricted-v2** / PSA **restricted** posture. + +## Related topics + +- [Pre-Requisites & Support Matrix](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md) +- [Deployment options](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/) +- [Helm chart README](./README.md) +- [Audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md)