From 85d70e86a91f79d3d3b29fbab0797ceed2091184 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Fri, 29 May 2026 09:13:11 -0700 Subject: [PATCH 01/17] docs(helm): expand OpenShift guidance for NVBugs 6218358 follow-up Document RC9 QA findings: prebuilt ffmpeg image on restricted-v2, openshift-restricted values profile, internal registry pull secrets, optional NIM LD_LIBRARY_PATH overrides, and Omni caption smoke-test request shape. Cross-link from deployment-options and prerequisites-support-matrix. --- docs/docs/extraction/deployment-options.md | 7 +- .../prerequisites-support-matrix.md | 9 +- nemo_retriever/helm/README.md | 169 +++++++++++++++++- 3 files changed, 175 insertions(+), 10 deletions(-) diff --git a/docs/docs/extraction/deployment-options.md b/docs/docs/extraction/deployment-options.md index 71f0b7510..c01a6f3bd 100644 --- a/docs/docs/extraction/deployment-options.md +++ b/docs/docs/extraction/deployment-options.md @@ -30,8 +30,11 @@ so the service container installs `ffmpeg` and `ffprobe` at startup. This runtime install requires package-repository network egress, a writable root filesystem, and security policy that allows the image's scoped sudo use. If your cluster blocks startup package installation (for example air-gapped -environments), use a custom service image that already contains `ffmpeg` and -`ffprobe`, then set `service.image.repository` and `service.image.tag`. +environments or **OpenShift restricted-v2**), use a custom service image that +already contains `ffmpeg` and `ffprobe`, then set `service.image.repository` +and `service.image.tag`. For OpenShift overrides, see +[OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#openshift-deployment) +in the Helm chart README. ### I want examples and notebooks diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index b8910f2ea..64699d3a3 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -11,8 +11,11 @@ Before you begin using [NeMo Retriever Library](overview.md), confirm your softw - For audio and video, `ffmpeg` and `ffprobe` must be on `PATH` (for example `sudo apt-get install -y --no-install-recommends ffmpeg` on Debian/Ubuntu). `ffmpeg-python` and `nemo-retriever[multimedia]` do not install these binaries. - On Helm with package-repo access, set `service.installFfmpeg=true`. For - air-gapped clusters, see [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment). + On Helm with package-repo access, set `service.installFfmpeg=true`. On + OpenShift restricted-v2, use a prebuilt service image instead — see + [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-and-video-ffmpeg-on-restricted-openshift) + in the Helm chart README. For air-gapped clusters, see + [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment). - For PDF extraction with `extract_method="nemotron_parse"`, install the Nemotron Parse client dependencies with `pip install "nemo-retriever[nemotron-parse]"` (pulls `open-clip-torch`, which provides the `open_clip` module required by the Nemotron Parse @@ -118,6 +121,8 @@ For 26.05, use **`nemotron_3_nano_omni_30b_a3b_reasoning`** when you enable the To validate caption traffic during ingest, inspect metadata such as `page_elements_v3_counts_by_label`. If the figure is labeled `chart`, expect no Omni chat-completions requests for that region even when captioning is enabled. + For direct `/v1/chat/completions` smoke tests against the Omni NIM (outside the retriever service), set `chat_template_kwargs.enable_thinking=false` so the caption appears in `message.content`. The service caption profile already applies this during ingest. See [OpenShift deployment — Omni caption manual smoke testing](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#omni-caption-manual-smoke-testing) for a request example. + Optional features listed in the table above require additional GPU support, disk space, and feature-specific system dependencies beyond the four default NIMs. For published NIM model IDs and deployment-specific constraints, use the product support matrices linked under [Related Topics](#related-topics) below. diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 41204a3d5..d71d2a613 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -295,7 +295,10 @@ short list of knobs you'll touch first. | `service.gpu.enabled` | `false` | The service does **not** need a GPU. | For audio and video extraction, set `service.installFfmpeg=true` when your -cluster allows runtime package installation. For air-gapped clusters, see +cluster allows runtime package installation. **OpenShift restricted-v2** blocks +that path — use a prebuilt service image instead; see +[Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift). +For air-gapped clusters, see [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment). ### Audio and video (Parakeet ASR) { #audio-video-parakeet } @@ -971,23 +974,36 @@ fields become hard rejections, not warnings. | `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | | `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Disable vectordb for smoke tests, or patch the vectordb Deployment after install (below) | | PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | +| Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (see [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | +| `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart `imagePullSecrets` lists only `ngc-secret`; the namespace ServiceAccount `dockercfg` secret is not merged automatically | Add the SA pull secret under `imagePullSecrets` (see [Internal registry pull secrets](#internal-registry-pull-secrets)) | +| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths via `nimOperator..env` (see [Optional NIM runtime environment](#optional-nim-runtime-environment)) | ### Recommended value overrides +Save the block below as a local values file (for example `openshift-restricted.yaml`) +and pass `-f openshift-restricted.yaml` on every `helm install` / `helm upgrade`. +This is the profile QA used on OpenShift 4.20 with the NIM Operator: core PDF +pipeline, Parakeet ASR (with a prebuilt `ffmpeg` service image), and Omni captioning +all reached Ready when combined with the NGC secrets and any NIM-specific env +overrides noted in later subsections. + ```yaml # OpenShift overrides for nemo-retriever Helm chart (restricted-v2 / PSA restricted). -# Save locally, then: helm install retriever ./nemo_retriever/helm -f .yaml ... +# helm install retriever ./nemo_retriever/helm -f openshift-restricted.yaml ... service: podSecurityContext: runAsNonRoot: true # Do NOT set runAsUser, runAsGroup, or fsGroup — OpenShift SCC assigns them. securityContext: + runAsNonRoot: true allowPrivilegeEscalation: false capabilities: drop: ["ALL"] seccompProfile: type: RuntimeDefault + # Leave installFfmpeg false on restricted-v2; use a custom image for audio/video. + installFfmpeg: false serviceConfig: logging: @@ -1007,10 +1023,128 @@ When **`persistence.enabled=true`**, you can keep the default log path under SCC-assigned `fsGroup` applies. When persistence is off, always relocate logs to `/tmp` (or another path backed by `service.extraVolumes`). +### Audio and video (`ffmpeg`) on restricted OpenShift { #audio-and-video-ffmpeg-on-restricted-openshift } + +Elsewhere this README documents `service.installFfmpeg=true`, which installs +`ffmpeg`/`ffprobe` at container startup via passwordless `sudo`. On OpenShift +**restricted-v2** SCC, that path is blocked: PSA restricted sets +`allowPrivilegeEscalation: false` and the SCC enforces **no-new-privileges**, so +the entrypoint cannot elevate to install packages. + +For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. +Instead, extend the service image on a connected build host and point the chart at +that tag (same pattern as [air-gapped custom images](#1-service-image)): + +```dockerfile +FROM nvcr.io/nvstaging/nim/nrl-service: +USER root +RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \ + && rm -rf /var/lib/apt/lists/* +USER nemo +``` + +Push the result to NGC, your private registry, or the +[OpenShift internal registry](#internal-registry-pull-secrets), then set +`service.image.repository` / `service.image.tag` accordingly. + +### Internal registry pull secrets { #internal-registry-pull-secrets } + +When you rebuild the service image into the OpenShift internal registry +(`image-registry.openshift-image-registry.svc:5000/...`), pods pull through the +namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart +renders an explicit `imagePullSecrets` list on every Pod; that list **replaces** +the implicit SA default rather than merging with it. If you set only `ngc-secret` +(for NGC NIM images) but point `service.image` at the internal registry, the +service Pod can fail with `ImagePullBackOff`. + +**Internal-registry service image only** — omit the chart-managed NGC pull secret +name so the helper does not inject `ngc-secret` alone: + +```yaml +ngcImagePullSecret: + create: false + # Do not set name — leaves ngc-secret out of the rendered Pod spec. + +imagePullSecrets: + - name: default-dockercfg-xxxxx # replace with your SA secret (see below) +``` + +**Both NGC NIMs and an internal-registry service image** — list every secret the +pods need: + +```yaml +ngcImagePullSecret: + create: false + name: ngc-secret + +imagePullSecrets: + - name: ngc-secret + - name: default-dockercfg-xxxxx +``` + +Look up the ServiceAccount pull secret name: + +```bash +oc get sa default -n nemo-retriever -o jsonpath='{.imagePullSecrets[*].name}{"\n"}' +``` + +Pre-create `ngc-secret` / `ngc-api` with `oc create secret` when +`ngcImagePullSecret.create=false` and `ngcApiSecret.create=false`, as in the +install examples below. + +### Optional NIM runtime environment { #optional-nim-runtime-environment } + +On OpenShift 4.20 with the NIM Operator, QA found that some **optional** NIM pods +(for example Parakeet ASR or Omni caption) needed an explicit `LD_LIBRARY_PATH` +so CUDA/driver libraries from the GPU Operator stack were visible at runtime. +Symptoms are `CrashLoopBackOff` and log lines referencing a missing `.so` (for +example `libcudart` or `libcudnn`). + +Merge additional `env` entries under the relevant `nimOperator..env` list in +your values file (retain the chart's existing `env` entries for that NIM and +append `LD_LIBRARY_PATH`): + +```yaml +nimOperator: + audio: + env: + # ... chart defaults (NIM_TAGS_SELECTOR, NIM_TRITON_LOG_VERBOSE, etc.) ... + - name: LD_LIBRARY_PATH + value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" + nemotron_3_nano_omni_30b_a3b_reasoning: + env: + # ... chart defaults (NIM_HTTP_API_PORT, etc.) ... + - name: LD_LIBRARY_PATH + value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" +``` + +Paths vary with GPU Operator version and node image. Inspect a healthy GPU +workload on the same cluster (`oc exec` into a CUDA sample pod) or the failing +NIM pod's filesystem before pinning production values. + +### Omni caption manual smoke testing { #omni-caption-manual-smoke-testing } + +The retriever service caption profile already sends +`chat_template_kwargs.enable_thinking=false` to the Omni NIM during ingest. When +you call the Omni NIM **directly** at `/v1/chat/completions` (for example with +`curl` against the in-cluster `NIMService`), include the same flag so the +caption text lands in `message.content` instead of reasoning-only fields: + +```json +{ + "model": "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning", + "messages": [{"role": "user", "content": "..."}], + "chat_template_kwargs": {"enable_thinking": false} +} +``` + +See [Image captioning](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#image-captioning-2605) +for pipeline scope (chart regions are not captioned). + ### Example install on OpenShift 4.20 (service-only smoke test) -Matches QA validation with external NIMs disabled, no persistence, and no results -PVC: +Matches early QA repro with external NIMs disabled, no persistence, and no +results PVC: ```bash oc new-project nemo-retriever @@ -1024,12 +1158,14 @@ oc create secret generic ngc-api -n nemo-retriever \ --from-literal=NGC_CLI_API_KEY="$NGC_API_KEY" helm install retriever ./nemo_retriever/helm -n nemo-retriever \ - -f .yaml \ + -f openshift-restricted.yaml \ --set ngcImagePullSecret.create=false \ --set ngcApiSecret.create=false \ --set nims.enabled=false \ --set persistence.enabled=false \ - --set retrieverResults.enabled=false + --set retrieverResults.enabled=false \ + --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ + --set service.image.tag= ``` Verify pods: @@ -1042,6 +1178,27 @@ oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever You should see SCC-assigned numeric `runAsUser` on containers that declare a `securityContext` block, and no PSA warnings once overrides are applied. +### Example install with NIM Operator (full pipeline validation) + +After NIM Operator and GPU Operator are installed, reuse `openshift-restricted.yaml` +and pre-created NGC secrets. Point `service.image` at a **ffmpeg-enabled** build +when validating [audio and video](#audio-and-video-ffmpeg-on-restricted-openshift). +Add [optional NIM `LD_LIBRARY_PATH`](#optional-nim-runtime-environment) overrides +if ASR or Omni pods crash on missing libraries. + +```bash +helm install retriever ./nemo_retriever/helm -n nemo-retriever \ + -f openshift-restricted.yaml \ + --set ngcImagePullSecret.create=false \ + --set ngcApiSecret.create=false \ + --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ + --set service.image.tag= +``` + +QA validated this posture on OpenShift 4.20 with default PDF, Parakeet ASR, and +Omni caption smoke tests passing when the overrides above and any per-NIM env +fixes were applied. + ### Enabling the vectordb Deployment on OpenShift `serviceConfig.vectordb.enabled=true` renders a **vectordb** container from the From 1f9edfd2f9c3a43a512dca3a96fbe6446e775a62 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Fri, 29 May 2026 09:21:47 -0700 Subject: [PATCH 02/17] docs: align caption cross-links with 26.05 page-role rules Move Omni smoke-test note out of the chart admonition; use neutral link label to helm README anchor. --- docs/docs/extraction/prerequisites-support-matrix.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index 64699d3a3..02001bf70 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -13,7 +13,7 @@ Before you begin using [NeMo Retriever Library](overview.md), confirm your softw `ffmpeg-python` and `nemo-retriever[multimedia]` do not install these binaries. On Helm with package-repo access, set `service.installFfmpeg=true`. On OpenShift restricted-v2, use a prebuilt service image instead — see - [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-and-video-ffmpeg-on-restricted-openshift) + [Audio and video on restricted OpenShift](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-and-video-ffmpeg-on-restricted-openshift) in the Helm chart README. For air-gapped clusters, see [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment). - For PDF extraction with `extract_method="nemotron_parse"`, install the Nemotron Parse @@ -121,7 +121,7 @@ For 26.05, use **`nemotron_3_nano_omni_30b_a3b_reasoning`** when you enable the To validate caption traffic during ingest, inspect metadata such as `page_elements_v3_counts_by_label`. If the figure is labeled `chart`, expect no Omni chat-completions requests for that region even when captioning is enabled. - For direct `/v1/chat/completions` smoke tests against the Omni NIM (outside the retriever service), set `chat_template_kwargs.enable_thinking=false` so the caption appears in `message.content`. The service caption profile already applies this during ingest. See [OpenShift deployment — Omni caption manual smoke testing](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#omni-caption-manual-smoke-testing) for a request example. +For direct `/v1/chat/completions` smoke tests against the Omni NIM (outside the retriever service), set `chat_template_kwargs.enable_thinking=false` so the caption appears in `message.content`. The service caption profile already applies this during ingest. See [Omni caption manual smoke testing](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#omni-caption-manual-smoke-testing) for a request example. Optional features listed in the table above require additional GPU support, disk space, and feature-specific system dependencies beyond the four default NIMs. From 70fb39ca2fff5fbecf36fb78e0f36e89f7b63bd7 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Fri, 29 May 2026 09:30:19 -0700 Subject: [PATCH 03/17] docs(helm): fix audit findings for OpenShift PR Add explicit #1-service-image anchor and Parakeet step-4 OpenShift caveat for installFfmpeg. --- nemo_retriever/helm/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index d71d2a613..7bbba405b 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -78,7 +78,7 @@ nemo_retriever/helm/ ## Quick start -### 1. Service image +### 1. Service image { #1-service-image } The chart defaults to the staging image published to NGC: @@ -308,7 +308,7 @@ To run self-hosted Parakeet for [audio and video extraction](https://github.com/ 1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install (26.05)](#recommended-minimal-install-2605)). 2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (see [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). 3. Confirm the GPU SKU in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) (footnote ⁴ lists Blackwell limitations). -4. Set `service.installFfmpeg=true` when the retriever service will process audio or video (see `service.installFfmpeg` above). +4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (see `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](#audio-and-video-ffmpeg-on-restricted-openshift) instead. The retriever service picks up the in-cluster ASR endpoint when `nimOperator.audio` is enabled; see [NIM Operator sub-stack](#nim-operator-sub-stack). From da879219f80d617908f4b55f7c3071df6d16b2d9 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Tue, 2 Jun 2026 14:27:33 -0700 Subject: [PATCH 04/17] docs: address ::a audit findings for OpenShift PR 2166 Replace matrix smoke-test prose with a one-line helm link per page roles. Show full nimOperator env lists in OpenShift LD_LIBRARY_PATH examples so Helm list overrides do not drop chart defaults. --- .../extraction/prerequisites-support-matrix.md | 2 +- nemo_retriever/helm/README.md | 16 +++++++++++----- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index 02001bf70..463b0bcbf 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -121,7 +121,7 @@ For 26.05, use **`nemotron_3_nano_omni_30b_a3b_reasoning`** when you enable the To validate caption traffic during ingest, inspect metadata such as `page_elements_v3_counts_by_label`. If the figure is labeled `chart`, expect no Omni chat-completions requests for that region even when captioning is enabled. -For direct `/v1/chat/completions` smoke tests against the Omni NIM (outside the retriever service), set `chat_template_kwargs.enable_thinking=false` so the caption appears in `message.content`. The service caption profile already applies this during ingest. See [Omni caption manual smoke testing](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#omni-caption-manual-smoke-testing) for a request example. +For direct Omni NIM smoke tests, see [Omni caption manual smoke testing](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#omni-caption-manual-smoke-testing) in the Helm chart README. Optional features listed in the table above require additional GPU support, disk space, and feature-specific system dependencies beyond the four default NIMs. diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 7bbba405b..8cc299b4a 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -1100,20 +1100,26 @@ so CUDA/driver libraries from the GPU Operator stack were visible at runtime. Symptoms are `CrashLoopBackOff` and log lines referencing a missing `.so` (for example `libcudart` or `libcudnn`). -Merge additional `env` entries under the relevant `nimOperator..env` list in -your values file (retain the chart's existing `env` entries for that NIM and -append `LD_LIBRARY_PATH`): +Helm **replaces** the whole `env` list when you override `nimOperator..env` +in a values file. Copy the chart defaults from `values.yaml` for that NIM, then +append `LD_LIBRARY_PATH`: ```yaml nimOperator: audio: env: - # ... chart defaults (NIM_TAGS_SELECTOR, NIM_TRITON_LOG_VERBOSE, etc.) ... + - name: NIM_TAGS_SELECTOR + value: "name=parakeet-1-1b-ctc-en-us,mode=ofl,vad=default,diarizer=disabled" + - name: NIM_TRITON_LOG_VERBOSE + value: "1" - name: LD_LIBRARY_PATH value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" nemotron_3_nano_omni_30b_a3b_reasoning: env: - # ... chart defaults (NIM_HTTP_API_PORT, etc.) ... + - name: NIM_HTTP_API_PORT + value: "8000" + - name: NIM_TRITON_LOG_VERBOSE + value: "1" - name: LD_LIBRARY_PATH value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" ``` From 8b6c30e4e3ffed49e47b03a287197719b422671f Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Wed, 3 Jun 2026 15:09:53 -0700 Subject: [PATCH 05/17] docs: remove deploy leakage from prerequisites matrix Move Helm, OpenShift, and Omni smoke-test guidance out of the support matrix so deployment details stay on audio-video.md and the Helm README. --- docs/docs/extraction/prerequisites-support-matrix.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index 463b0bcbf..a505f75a5 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -11,11 +11,7 @@ Before you begin using [NeMo Retriever Library](overview.md), confirm your softw - For audio and video, `ffmpeg` and `ffprobe` must be on `PATH` (for example `sudo apt-get install -y --no-install-recommends ffmpeg` on Debian/Ubuntu). `ffmpeg-python` and `nemo-retriever[multimedia]` do not install these binaries. - On Helm with package-repo access, set `service.installFfmpeg=true`. On - OpenShift restricted-v2, use a prebuilt service image instead — see - [Audio and video on restricted OpenShift](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-and-video-ffmpeg-on-restricted-openshift) - in the Helm chart README. For air-gapped clusters, see - [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment). + For container and Kubernetes guidance, see [Audio and video](audio-video.md). - For PDF extraction with `extract_method="nemotron_parse"`, install the Nemotron Parse client dependencies with `pip install "nemo-retriever[nemotron-parse]"` (pulls `open-clip-torch`, which provides the `open_clip` module required by the Nemotron Parse @@ -121,8 +117,6 @@ For 26.05, use **`nemotron_3_nano_omni_30b_a3b_reasoning`** when you enable the To validate caption traffic during ingest, inspect metadata such as `page_elements_v3_counts_by_label`. If the figure is labeled `chart`, expect no Omni chat-completions requests for that region even when captioning is enabled. -For direct Omni NIM smoke tests, see [Omni caption manual smoke testing](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#omni-caption-manual-smoke-testing) in the Helm chart README. - Optional features listed in the table above require additional GPU support, disk space, and feature-specific system dependencies beyond the four default NIMs. For published NIM model IDs and deployment-specific constraints, use the product support matrices linked under [Related Topics](#related-topics) below. From 710c82e929bb26888713a3e285c057173a40244d Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Wed, 3 Jun 2026 15:30:30 -0700 Subject: [PATCH 06/17] docs: move OpenShift guidance to dedicated openshift.md Extract restricted-v2 install procedures from the Helm README into docs/docs/extraction/openshift.md per review feedback. Helm README keeps a stub with cross-links; deployment-options and mkdocs nav point to the new page. Remove QA phrasing and fix ngcImagePullSecret name clearing. --- docs/docs/extraction/deployment-options.md | 3 +- docs/docs/extraction/openshift.md | 248 +++++++++++++++++ docs/mkdocs.yml | 1 + nemo_retriever/helm/README.md | 309 +-------------------- 4 files changed, 254 insertions(+), 307 deletions(-) create mode 100644 docs/docs/extraction/openshift.md diff --git a/docs/docs/extraction/deployment-options.md b/docs/docs/extraction/deployment-options.md index c01a6f3bd..cd16232fc 100644 --- a/docs/docs/extraction/deployment-options.md +++ b/docs/docs/extraction/deployment-options.md @@ -33,8 +33,7 @@ your cluster blocks startup package installation (for example air-gapped environments or **OpenShift restricted-v2**), use a custom service image that already contains `ffmpeg` and `ffprobe`, then set `service.image.repository` and `service.image.tag`. For OpenShift overrides, see -[OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#openshift-deployment) -in the Helm chart README. +[OpenShift deployment](openshift.md). ### I want examples and notebooks diff --git a/docs/docs/extraction/openshift.md b/docs/docs/extraction/openshift.md new file mode 100644 index 000000000..35d3c3acf --- /dev/null +++ b/docs/docs/extraction/openshift.md @@ -0,0 +1,248 @@ +# OpenShift deployment + +Use this page when you install the [NeMo Retriever Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) on **OpenShift 4.x** with the default **restricted-v2** Security Context Constraint (SCC) and **Pod Security Admission (PSA) `restricted`** profile. + +For general Kubernetes and Helm deployment choices, see [Deployment options](deployment-options.md). For chart values and NIM wiring, see the [Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md). + +## Overview { #openshift-deployment } + +The chart defaults target generic Kubernetes clusters that allow fixed numeric UIDs (`runAsUser` / `runAsGroup` / `fsGroup` **1000**). OpenShift namespaces under **restricted-v2** assign a per-namespace UID/GID range instead. A stock `helm install` without overrides therefore fails SCC validation, emits PSA warnings, or crashes on log paths the random UID cannot write. + +We do **not** change chart defaults for OpenShift-only behavior (that would affect other platforms). Use the overrides below, or save the YAML block into a local values file and pass `-f ` on every `helm install` / `helm upgrade`. + +### Cluster posture (typical hardened namespaces) + +| Control | Typical default on a new OpenShift project | +| --- | --- | +| SCC | **restricted-v2** (first match in priority order) | +| PSA | `pod-security.kubernetes.io/warn=restricted` (and often `audit=restricted`; `enforce` may be unset on dev clusters) | +| UID assignment | SCC injects `runAsUser` / `fsGroup` from the namespace range (for example `1000750000–1000759999`) | + +On clusters with **PSA `enforce=restricted`**, missing container `securityContext` fields become hard rejections, not warnings. + +### Override reference (maps to chart limitations) + +| Symptom on stock install | Cause | Helm override | +| --- | --- | --- | +| `FailedCreate`: UID/GID **1000** not in namespace range | Hardcoded `service.podSecurityContext` UID/GID/fsGroup | Omit `runAsUser`, `runAsGroup`, and `fsGroup`; keep only `runAsNonRoot: true` | +| PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (see sample below) | +| `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | +| `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Disable vectordb for smoke tests, or patch the vectordb Deployment after install (below) | +| PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | +| Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (see [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | +| `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart `imagePullSecrets` lists only `ngc-secret`; the namespace ServiceAccount `dockercfg` secret is not merged automatically | Add the SA pull secret under `imagePullSecrets` (see [Internal registry pull secrets](#internal-registry-pull-secrets)) | +| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths via `nimOperator..env` (see [Optional NIM runtime environment](#optional-nim-runtime-environment)) | + +### Recommended value overrides + +Save the block below as a local values file (for example `openshift-restricted.yaml`) and pass `-f openshift-restricted.yaml` on every `helm install` / `helm upgrade`. Use this profile for **restricted-v2** / PSA **restricted** namespaces with the NIM Operator, a prebuilt `ffmpeg` service image when you need [audio and video](audio-video.md), and optional NIM env overrides from later sections. + +```yaml +# OpenShift overrides for nemo-retriever Helm chart (restricted-v2 / PSA restricted). +# helm install retriever ./nemo_retriever/helm -f openshift-restricted.yaml ... + +service: + podSecurityContext: + runAsNonRoot: true + # Do NOT set runAsUser, runAsGroup, or fsGroup — OpenShift SCC assigns them. + securityContext: + runAsNonRoot: true + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + seccompProfile: + type: RuntimeDefault + # Leave installFfmpeg false on restricted-v2; use a custom image for audio/video. + installFfmpeg: false + +serviceConfig: + logging: + # Writable without persistence PVC (chart always mounts emptyDir at /tmp). + file: /tmp/retriever-service.log + vectordb: + # Set false for minimal service-only validation; see vectordb patch below if enabled. + enabled: false + +topology: + otel: + enabled: false +``` + +When **`persistence.enabled=true`**, you can keep the default log path under `persistence.mountPath` (`/var/lib/nemo-retriever`) because the PVC is mounted and SCC-assigned `fsGroup` applies. When persistence is off, always relocate logs to `/tmp` (or another path backed by `service.extraVolumes`). + +### Audio and video (`ffmpeg`) on restricted OpenShift { #audio-and-video-ffmpeg-on-restricted-openshift } + +The Helm chart supports `service.installFfmpeg=true`, which installs `ffmpeg`/`ffprobe` at container startup via passwordless `sudo`. On OpenShift **restricted-v2** SCC, that path is blocked: PSA restricted sets `allowPrivilegeEscalation: false` and the SCC enforces **no-new-privileges**, so the entrypoint cannot elevate to install packages. + +For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. Instead, extend the service image on a connected build host and point the chart at that tag (same pattern as [air-gapped custom service images](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#1-service-image)): + +```dockerfile +FROM nvcr.io/nvstaging/nim/nrl-service: +USER root +RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \ + && rm -rf /var/lib/apt/lists/* +USER nemo +``` + +Push the result to NGC, your private registry, or the [OpenShift internal registry](#internal-registry-pull-secrets), then set `service.image.repository` / `service.image.tag` accordingly. + +### Internal registry pull secrets { #internal-registry-pull-secrets } + +When you rebuild the service image into the OpenShift internal registry (`image-registry.openshift-image-registry.svc:5000/...`), pods pull through the namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart renders an explicit `imagePullSecrets` list on every Pod; that list **replaces** the implicit SA default rather than merging with it. If you set only `ngc-secret` (for NGC NIM images) but point `service.image` at the internal registry, the service Pod can fail with `ImagePullBackOff`. + +**Internal-registry service image only** — clear the chart-managed NGC pull secret name so the helper does not inject `ngc-secret` alone: + +```yaml +ngcImagePullSecret: + create: false + name: "" # Explicitly empty — clears the default "ngc-secret" + +imagePullSecrets: + - name: default-dockercfg-xxxxx # replace with your SA secret (see below) +``` + +**Both NGC NIMs and an internal-registry service image** — list every secret the pods need: + +```yaml +ngcImagePullSecret: + create: false + name: ngc-secret + +imagePullSecrets: + - name: ngc-secret + - name: default-dockercfg-xxxxx +``` + +Look up the ServiceAccount pull secret name: + +```bash +oc get sa default -n nemo-retriever -o jsonpath='{.imagePullSecrets[*].name}{"\n"}' +``` + +Pre-create `ngc-secret` / `ngc-api` with `oc create secret` when `ngcImagePullSecret.create=false` and `ngcApiSecret.create=false`, as in the install examples below. + +### Optional NIM runtime environment { #optional-nim-runtime-environment } + +On some OpenShift clusters with the NIM Operator, **optional** NIM pods (for example Parakeet ASR or Omni caption) need an explicit `LD_LIBRARY_PATH` so CUDA/driver libraries from the GPU Operator stack are visible at runtime. Symptoms are `CrashLoopBackOff` and log lines referencing a missing `.so` (for example `libcudart` or `libcudnn`). + +Helm **replaces** the whole `env` list when you override `nimOperator..env` in a values file. Copy the chart defaults from `values.yaml` for that NIM, then append `LD_LIBRARY_PATH`: + +```yaml +nimOperator: + audio: + env: + - name: NIM_TAGS_SELECTOR + value: "name=parakeet-1-1b-ctc-en-us,mode=ofl,vad=default,diarizer=disabled" + - name: NIM_TRITON_LOG_VERBOSE + value: "1" + - name: LD_LIBRARY_PATH + value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" + nemotron_3_nano_omni_30b_a3b_reasoning: + env: + - name: NIM_HTTP_API_PORT + value: "8000" + - name: NIM_TRITON_LOG_VERBOSE + value: "1" + - name: LD_LIBRARY_PATH + value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" +``` + +Paths vary with GPU Operator version and node image. Inspect a healthy GPU workload on the same cluster (`oc exec` into a CUDA sample pod) or the failing NIM pod's filesystem before pinning production values. + +### Omni caption manual smoke testing { #omni-caption-manual-smoke-testing } + +The retriever service caption profile already sends `chat_template_kwargs.enable_thinking=false` to the Omni NIM during ingest. When you call the Omni NIM **directly** at `/v1/chat/completions` (for example with `curl` against the in-cluster `NIMService`), include the same flag so the caption text lands in `message.content` instead of reasoning-only fields: + +```json +{ + "model": "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning", + "messages": [{"role": "user", "content": "..."}], + "chat_template_kwargs": {"enable_thinking": false} +} +``` + +For pipeline scope (PDF chart regions are not captioned), see [Image captioning](prerequisites-support-matrix.md#image-captioning-2605) and [Image captioning](multimodal-extraction.md#image-captioning) in the extraction docs. + +### Example install (service-only validation) + +Minimal validation with external NIMs disabled, no persistence, and no results PVC: + +```bash +oc new-project nemo-retriever + +oc create secret docker-registry ngc-secret -n nemo-retriever \ + --docker-server=nvcr.io --docker-username='$oauthtoken' \ + --docker-password="$NGC_API_KEY" + +oc create secret generic ngc-api -n nemo-retriever \ + --from-literal=NGC_API_KEY="$NGC_API_KEY" \ + --from-literal=NGC_CLI_API_KEY="$NGC_API_KEY" + +helm install retriever ./nemo_retriever/helm -n nemo-retriever \ + -f openshift-restricted.yaml \ + --set ngcImagePullSecret.create=false \ + --set ngcApiSecret.create=false \ + --set nims.enabled=false \ + --set persistence.enabled=false \ + --set retrieverResults.enabled=false \ + --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ + --set service.image.tag= +``` + +Verify pods: + +```bash +oc get pods -n nemo-retriever +oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever +``` + +You should see SCC-assigned numeric `runAsUser` on containers that declare a `securityContext` block, and no PSA warnings once overrides are applied. + +### Example install with NIM Operator (full pipeline) + +After NIM Operator and GPU Operator are installed, reuse `openshift-restricted.yaml` and pre-created NGC secrets. Point `service.image` at a **ffmpeg-enabled** build when validating [audio and video](#audio-and-video-ffmpeg-on-restricted-openshift). Add [optional NIM `LD_LIBRARY_PATH`](#optional-nim-runtime-environment) overrides if ASR or Omni pods crash on missing libraries. + +```bash +helm install retriever ./nemo_retriever/helm -n nemo-retriever \ + -f openshift-restricted.yaml \ + --set ngcImagePullSecret.create=false \ + --set ngcApiSecret.create=false \ + --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ + --set service.image.tag= +``` + +### Enabling the vectordb Deployment on OpenShift + +`serviceConfig.vectordb.enabled=true` renders a **vectordb** container from the same image (`USER nemo`, non-numeric). The chart does not yet expose a `securityContext` value for that container. After `helm install`, patch the Deployment so OpenShift can inject a numeric UID into the container spec: + +```bash +RELEASE=retriever +NS=nemo-retriever +VDB_DEPLOY="${RELEASE}-nemo-retriever-vectordb" + +oc patch deployment "$VDB_DEPLOY" -n "$NS" --type=json -p='[ + {"op": "add", "path": "/spec/template/spec/containers/0/securityContext", "value": { + "allowPrivilegeEscalation": false, + "capabilities": {"drop": ["ALL"]}, + "runAsNonRoot": true, + "seccompProfile": {"type": "RuntimeDefault"} + }} +]' +``` + +Re-apply the patch after `helm upgrade` if the Deployment is recreated. A future chart release may add first-class `topology.vectordb.securityContext` values. + +### Enabling the OpenTelemetry collector on OpenShift + +The chart's otel-collector Deployment likewise lacks `securityContext` fields. Prefer `topology.otel.enabled=false` (as in the sample values) unless you operate your own collector or patch `*-otel` the same way as vectordb. + +### What we intentionally do not require on OpenShift + +Do **not** bind the namespace to **anyuid** SCC or set PSA `enforce=privileged` unless your security team explicitly approves it. The overrides above are intended to keep **restricted-v2** / PSA **restricted** posture. + +## Related topics + +- [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md) +- [Deployment options](deployment-options.md) +- [Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) +- [Audio and video](audio-video.md) diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 6300e6c41..d88532158 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -84,6 +84,7 @@ nav: - "Authentication and API keys": extraction/api-keys.md - "3. Deployment options": - "Compare deployment options": extraction/deployment-options.md + - "OpenShift deployment": extraction/openshift.md - "Helm chart (Kubernetes)": https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/helm - "Docker service image": https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/docker.md - "4. Core workflows": diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 8cc299b4a..a5cf17d90 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -297,7 +297,7 @@ short list of knobs you'll touch first. For audio and video extraction, set `service.installFfmpeg=true` when your cluster allows runtime package installation. **OpenShift restricted-v2** blocks that path — use a prebuilt service image instead; see -[Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift). +[Audio and video on restricted OpenShift](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift). For air-gapped clusters, see [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment). @@ -308,7 +308,7 @@ To run self-hosted Parakeet for [audio and video extraction](https://github.com/ 1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install (26.05)](#recommended-minimal-install-2605)). 2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (see [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). 3. Confirm the GPU SKU in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) (footnote ⁴ lists Blackwell limitations). -4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (see `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](#audio-and-video-ffmpeg-on-restricted-openshift) instead. +4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (see `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead. The retriever service picks up the in-cluster ASR endpoint when `nimOperator.audio` is enabled; see [NIM Operator sub-stack](#nim-operator-sub-stack). @@ -942,310 +942,9 @@ sanity check before opening Grafana. ## OpenShift deployment { #openshift-deployment } -The chart defaults target generic Kubernetes clusters that allow fixed numeric -UIDs (`runAsUser` / `runAsGroup` / `fsGroup` **1000**). **OpenShift 4.x** -namespaces under the default **restricted-v2** Security Context Constraint (SCC) -and **Pod Security Admission (PSA) `restricted`** profile assign a per-namespace -UID/GID range instead. A stock `helm install` without overrides therefore fails -SCC validation, emits PSA warnings, or crashes on log paths the random UID cannot -write. - -We do **not** change chart defaults for OpenShift-only behavior (that would affect -other platforms). Use the overrides below on OpenShift, or save the YAML block -into a local values file and pass `-f `. - -### Cluster posture (typical QA / hardened namespaces) - -| Control | Typical default on a new OpenShift project | -| --- | --- | -| SCC | **restricted-v2** (first match in priority order) | -| PSA | `pod-security.kubernetes.io/warn=restricted` (and often `audit=restricted`; `enforce` may be unset on dev clusters) | -| UID assignment | SCC injects `runAsUser` / `fsGroup` from the namespace range (for example `1000750000–1000759999`) | - -On clusters with **PSA `enforce=restricted`**, missing container `securityContext` -fields become hard rejections, not warnings. - -### Override reference (maps to chart limitations) - -| Symptom on stock install | Cause | Helm override | -| --- | --- | --- | -| `FailedCreate`: UID/GID **1000** not in namespace range | Hardcoded `service.podSecurityContext` UID/GID/fsGroup | Omit `runAsUser`, `runAsGroup`, and `fsGroup`; keep only `runAsNonRoot: true` | -| PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (see sample below) | -| `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | -| `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Disable vectordb for smoke tests, or patch the vectordb Deployment after install (below) | -| PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | -| Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (see [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | -| `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart `imagePullSecrets` lists only `ngc-secret`; the namespace ServiceAccount `dockercfg` secret is not merged automatically | Add the SA pull secret under `imagePullSecrets` (see [Internal registry pull secrets](#internal-registry-pull-secrets)) | -| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths via `nimOperator..env` (see [Optional NIM runtime environment](#optional-nim-runtime-environment)) | - -### Recommended value overrides - -Save the block below as a local values file (for example `openshift-restricted.yaml`) -and pass `-f openshift-restricted.yaml` on every `helm install` / `helm upgrade`. -This is the profile QA used on OpenShift 4.20 with the NIM Operator: core PDF -pipeline, Parakeet ASR (with a prebuilt `ffmpeg` service image), and Omni captioning -all reached Ready when combined with the NGC secrets and any NIM-specific env -overrides noted in later subsections. +OpenShift install procedures, **restricted-v2** / PSA **restricted** value overrides, prebuilt `ffmpeg` images, internal registry pull secrets, optional NIM `LD_LIBRARY_PATH` tuning, and install examples are in **[OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md)** (published extraction docs). -```yaml -# OpenShift overrides for nemo-retriever Helm chart (restricted-v2 / PSA restricted). -# helm install retriever ./nemo_retriever/helm -f openshift-restricted.yaml ... - -service: - podSecurityContext: - runAsNonRoot: true - # Do NOT set runAsUser, runAsGroup, or fsGroup — OpenShift SCC assigns them. - securityContext: - runAsNonRoot: true - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - seccompProfile: - type: RuntimeDefault - # Leave installFfmpeg false on restricted-v2; use a custom image for audio/video. - installFfmpeg: false - -serviceConfig: - logging: - # Writable without persistence PVC (chart always mounts emptyDir at /tmp). - file: /tmp/retriever-service.log - vectordb: - # Set false for minimal service-only validation; see vectordb patch below if enabled. - enabled: false - -topology: - otel: - enabled: false -``` - -When **`persistence.enabled=true`**, you can keep the default log path under -`persistence.mountPath` (`/var/lib/nemo-retriever`) because the PVC is mounted and -SCC-assigned `fsGroup` applies. When persistence is off, always relocate logs to -`/tmp` (or another path backed by `service.extraVolumes`). - -### Audio and video (`ffmpeg`) on restricted OpenShift { #audio-and-video-ffmpeg-on-restricted-openshift } - -Elsewhere this README documents `service.installFfmpeg=true`, which installs -`ffmpeg`/`ffprobe` at container startup via passwordless `sudo`. On OpenShift -**restricted-v2** SCC, that path is blocked: PSA restricted sets -`allowPrivilegeEscalation: false` and the SCC enforces **no-new-privileges**, so -the entrypoint cannot elevate to install packages. - -For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. -Instead, extend the service image on a connected build host and point the chart at -that tag (same pattern as [air-gapped custom images](#1-service-image)): - -```dockerfile -FROM nvcr.io/nvstaging/nim/nrl-service: -USER root -RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \ - && rm -rf /var/lib/apt/lists/* -USER nemo -``` - -Push the result to NGC, your private registry, or the -[OpenShift internal registry](#internal-registry-pull-secrets), then set -`service.image.repository` / `service.image.tag` accordingly. - -### Internal registry pull secrets { #internal-registry-pull-secrets } - -When you rebuild the service image into the OpenShift internal registry -(`image-registry.openshift-image-registry.svc:5000/...`), pods pull through the -namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart -renders an explicit `imagePullSecrets` list on every Pod; that list **replaces** -the implicit SA default rather than merging with it. If you set only `ngc-secret` -(for NGC NIM images) but point `service.image` at the internal registry, the -service Pod can fail with `ImagePullBackOff`. - -**Internal-registry service image only** — omit the chart-managed NGC pull secret -name so the helper does not inject `ngc-secret` alone: - -```yaml -ngcImagePullSecret: - create: false - # Do not set name — leaves ngc-secret out of the rendered Pod spec. - -imagePullSecrets: - - name: default-dockercfg-xxxxx # replace with your SA secret (see below) -``` - -**Both NGC NIMs and an internal-registry service image** — list every secret the -pods need: - -```yaml -ngcImagePullSecret: - create: false - name: ngc-secret - -imagePullSecrets: - - name: ngc-secret - - name: default-dockercfg-xxxxx -``` - -Look up the ServiceAccount pull secret name: - -```bash -oc get sa default -n nemo-retriever -o jsonpath='{.imagePullSecrets[*].name}{"\n"}' -``` - -Pre-create `ngc-secret` / `ngc-api` with `oc create secret` when -`ngcImagePullSecret.create=false` and `ngcApiSecret.create=false`, as in the -install examples below. - -### Optional NIM runtime environment { #optional-nim-runtime-environment } - -On OpenShift 4.20 with the NIM Operator, QA found that some **optional** NIM pods -(for example Parakeet ASR or Omni caption) needed an explicit `LD_LIBRARY_PATH` -so CUDA/driver libraries from the GPU Operator stack were visible at runtime. -Symptoms are `CrashLoopBackOff` and log lines referencing a missing `.so` (for -example `libcudart` or `libcudnn`). - -Helm **replaces** the whole `env` list when you override `nimOperator..env` -in a values file. Copy the chart defaults from `values.yaml` for that NIM, then -append `LD_LIBRARY_PATH`: - -```yaml -nimOperator: - audio: - env: - - name: NIM_TAGS_SELECTOR - value: "name=parakeet-1-1b-ctc-en-us,mode=ofl,vad=default,diarizer=disabled" - - name: NIM_TRITON_LOG_VERBOSE - value: "1" - - name: LD_LIBRARY_PATH - value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" - nemotron_3_nano_omni_30b_a3b_reasoning: - env: - - name: NIM_HTTP_API_PORT - value: "8000" - - name: NIM_TRITON_LOG_VERBOSE - value: "1" - - name: LD_LIBRARY_PATH - value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" -``` - -Paths vary with GPU Operator version and node image. Inspect a healthy GPU -workload on the same cluster (`oc exec` into a CUDA sample pod) or the failing -NIM pod's filesystem before pinning production values. - -### Omni caption manual smoke testing { #omni-caption-manual-smoke-testing } - -The retriever service caption profile already sends -`chat_template_kwargs.enable_thinking=false` to the Omni NIM during ingest. When -you call the Omni NIM **directly** at `/v1/chat/completions` (for example with -`curl` against the in-cluster `NIMService`), include the same flag so the -caption text lands in `message.content` instead of reasoning-only fields: - -```json -{ - "model": "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning", - "messages": [{"role": "user", "content": "..."}], - "chat_template_kwargs": {"enable_thinking": false} -} -``` - -See [Image captioning](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#image-captioning-2605) -for pipeline scope (chart regions are not captioned). - -### Example install on OpenShift 4.20 (service-only smoke test) - -Matches early QA repro with external NIMs disabled, no persistence, and no -results PVC: - -```bash -oc new-project nemo-retriever - -oc create secret docker-registry ngc-secret -n nemo-retriever \ - --docker-server=nvcr.io --docker-username='$oauthtoken' \ - --docker-password="$NGC_API_KEY" - -oc create secret generic ngc-api -n nemo-retriever \ - --from-literal=NGC_API_KEY="$NGC_API_KEY" \ - --from-literal=NGC_CLI_API_KEY="$NGC_API_KEY" - -helm install retriever ./nemo_retriever/helm -n nemo-retriever \ - -f openshift-restricted.yaml \ - --set ngcImagePullSecret.create=false \ - --set ngcApiSecret.create=false \ - --set nims.enabled=false \ - --set persistence.enabled=false \ - --set retrieverResults.enabled=false \ - --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ - --set service.image.tag= -``` - -Verify pods: - -```bash -oc get pods -n nemo-retriever -oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever -``` - -You should see SCC-assigned numeric `runAsUser` on containers that declare a -`securityContext` block, and no PSA warnings once overrides are applied. - -### Example install with NIM Operator (full pipeline validation) - -After NIM Operator and GPU Operator are installed, reuse `openshift-restricted.yaml` -and pre-created NGC secrets. Point `service.image` at a **ffmpeg-enabled** build -when validating [audio and video](#audio-and-video-ffmpeg-on-restricted-openshift). -Add [optional NIM `LD_LIBRARY_PATH`](#optional-nim-runtime-environment) overrides -if ASR or Omni pods crash on missing libraries. - -```bash -helm install retriever ./nemo_retriever/helm -n nemo-retriever \ - -f openshift-restricted.yaml \ - --set ngcImagePullSecret.create=false \ - --set ngcApiSecret.create=false \ - --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ - --set service.image.tag= -``` - -QA validated this posture on OpenShift 4.20 with default PDF, Parakeet ASR, and -Omni caption smoke tests passing when the overrides above and any per-NIM env -fixes were applied. - -### Enabling the vectordb Deployment on OpenShift - -`serviceConfig.vectordb.enabled=true` renders a **vectordb** container from the -same image (`USER nemo`, non-numeric). The chart does not yet expose a -`securityContext` value for that container. After `helm install`, patch the -Deployment so OpenShift can inject a numeric UID into the container spec: - -```bash -RELEASE=retriever -NS=nemo-retriever -VDB_DEPLOY="${RELEASE}-nemo-retriever-vectordb" - -oc patch deployment "$VDB_DEPLOY" -n "$NS" --type=json -p='[ - {"op": "add", "path": "/spec/template/spec/containers/0/securityContext", "value": { - "allowPrivilegeEscalation": false, - "capabilities": {"drop": ["ALL"]}, - "runAsNonRoot": true, - "seccompProfile": {"type": "RuntimeDefault"} - }} -]' -``` - -Re-apply the patch after `helm upgrade` if the Deployment is recreated. A future -chart release may add first-class `topology.vectordb.securityContext` values. - -### Enabling the OpenTelemetry collector on OpenShift - -The chart’s otel-collector Deployment likewise lacks `securityContext` fields. -Prefer `topology.otel.enabled=false` (as in the sample values) unless you operate -your own collector or patch `*-otel` the same way as vectordb. - -### What we intentionally do not require on OpenShift - -Do **not** bind the namespace to **anyuid** SCC or set PSA `enforce=privileged` -unless your security team explicitly approves it. The overrides above are intended -to keep **restricted-v2** / PSA **restricted** posture. - -### Related documentation - -- [Pre-Requisites & Support Matrix](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md) -- [Deployment options](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) +This README keeps chart values, NIM wiring, and generic Kubernetes guidance. Pass `-f openshift-restricted.yaml` from that guide when you install on OpenShift. --- From af119bb4a577b3b10fd715bb0b4f54759db6dd1a Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Wed, 3 Jun 2026 15:33:16 -0700 Subject: [PATCH 07/17] docs: replace QA phrasing in OpenShift install examples Rewrite service-only and NIM Operator install intros as user actions instead of validation or QA repro language, per review on PR 2166. --- docs/docs/extraction/openshift.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/docs/extraction/openshift.md b/docs/docs/extraction/openshift.md index 35d3c3acf..e22e3d6c7 100644 --- a/docs/docs/extraction/openshift.md +++ b/docs/docs/extraction/openshift.md @@ -27,7 +27,7 @@ On clusters with **PSA `enforce=restricted`**, missing container `securityContex | `FailedCreate`: UID/GID **1000** not in namespace range | Hardcoded `service.podSecurityContext` UID/GID/fsGroup | Omit `runAsUser`, `runAsGroup`, and `fsGroup`; keep only `runAsNonRoot: true` | | PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (see sample below) | | `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | -| `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Disable vectordb for smoke tests, or patch the vectordb Deployment after install (below) | +| `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Set `serviceConfig.vectordb.enabled=false` for a minimal service-only install, or patch the vectordb Deployment after install (below) | | PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | | Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (see [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | | `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart `imagePullSecrets` lists only `ngc-secret`; the namespace ServiceAccount `dockercfg` secret is not merged automatically | Add the SA pull secret under `imagePullSecrets` (see [Internal registry pull secrets](#internal-registry-pull-secrets)) | @@ -60,7 +60,7 @@ serviceConfig: # Writable without persistence PVC (chart always mounts emptyDir at /tmp). file: /tmp/retriever-service.log vectordb: - # Set false for minimal service-only validation; see vectordb patch below if enabled. + # Set false for a minimal service-only install; see vectordb patch below if you enable it. enabled: false topology: @@ -149,9 +149,9 @@ nimOperator: Paths vary with GPU Operator version and node image. Inspect a healthy GPU workload on the same cluster (`oc exec` into a CUDA sample pod) or the failing NIM pod's filesystem before pinning production values. -### Omni caption manual smoke testing { #omni-caption-manual-smoke-testing } +### Call the Omni NIM directly { #omni-caption-manual-smoke-testing } -The retriever service caption profile already sends `chat_template_kwargs.enable_thinking=false` to the Omni NIM during ingest. When you call the Omni NIM **directly** at `/v1/chat/completions` (for example with `curl` against the in-cluster `NIMService`), include the same flag so the caption text lands in `message.content` instead of reasoning-only fields: +The retriever service caption profile already sends `chat_template_kwargs.enable_thinking=false` to the Omni NIM during ingest. To confirm the in-cluster Omni NIM returns caption text in `message.content`, call `/v1/chat/completions` directly (for example with `curl` against the `NIMService`) and include the same flag: ```json { @@ -163,9 +163,9 @@ The retriever service caption profile already sends `chat_template_kwargs.enable For pipeline scope (PDF chart regions are not captioned), see [Image captioning](prerequisites-support-matrix.md#image-captioning-2605) and [Image captioning](multimodal-extraction.md#image-captioning) in the extraction docs. -### Example install (service-only validation) +### Example install (service only, no in-cluster NIMs) -Minimal validation with external NIMs disabled, no persistence, and no results PVC: +Use this flow when you want only the retriever service pod: disable the in-cluster NIM Operator stack (`nims.enabled=false`), skip the persistence PVC, and skip the results PVC. Pre-create NGC pull/API secrets, then install with the restricted OpenShift values file: ```bash oc new-project nemo-retriever @@ -198,9 +198,9 @@ oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever You should see SCC-assigned numeric `runAsUser` on containers that declare a `securityContext` block, and no PSA warnings once overrides are applied. -### Example install with NIM Operator (full pipeline) +### Example install with NIM Operator (in-cluster NIMs) -After NIM Operator and GPU Operator are installed, reuse `openshift-restricted.yaml` and pre-created NGC secrets. Point `service.image` at a **ffmpeg-enabled** build when validating [audio and video](#audio-and-video-ffmpeg-on-restricted-openshift). Add [optional NIM `LD_LIBRARY_PATH`](#optional-nim-runtime-environment) overrides if ASR or Omni pods crash on missing libraries. +After you install the NIM Operator and GPU Operator, reuse `openshift-restricted.yaml` and the NGC secrets from the service-only example. Point `service.image` at a **ffmpeg-enabled** build when you use [audio and video](#audio-and-video-ffmpeg-on-restricted-openshift). Add [optional NIM `LD_LIBRARY_PATH`](#optional-nim-runtime-environment) overrides if ASR or Omni pods crash on missing libraries. ```bash helm install retriever ./nemo_retriever/helm -n nemo-retriever \ From b434aff68b346e7716e4a619715c0ccff6d7e447 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Wed, 3 Jun 2026 15:35:58 -0700 Subject: [PATCH 08/17] docs: add post-install checks after OpenShift NIM Operator example Replace the removed QA validation paragraph with user-facing pod and NIMService checks plus guidance when optional ASR or Omni NIMs fail. --- docs/docs/extraction/openshift.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/docs/extraction/openshift.md b/docs/docs/extraction/openshift.md index e22e3d6c7..c08a80efd 100644 --- a/docs/docs/extraction/openshift.md +++ b/docs/docs/extraction/openshift.md @@ -211,6 +211,15 @@ helm install retriever ./nemo_retriever/helm -n nemo-retriever \ --set service.image.tag= ``` +After install, confirm workloads reach Ready before you run ingest: + +```bash +oc get pods -n nemo-retriever +oc get nimservice -n nemo-retriever +``` + +When you enable optional NIMs, wait for Parakeet ASR and Omni caption pods to become Ready before you run [audio and video](audio-video.md) or caption extraction. If ASR or Omni pods crash with missing `.so` errors, apply the [LD_LIBRARY_PATH overrides](#optional-nim-runtime-environment) above and upgrade the release. + ### Enabling the vectordb Deployment on OpenShift `serviceConfig.vectordb.enabled=true` renders a **vectordb** container from the same image (`USER nemo`, non-numeric). The chart does not yet expose a `securityContext` value for that container. After `helm install`, patch the Deployment so OpenShift can inject a numeric UID into the container spec: From 2e8b16f12798b27b6ba73b54c94a55b31519826c Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Fri, 5 Jun 2026 16:10:32 -0700 Subject: [PATCH 09/17] docs: polish OpenShift guide after doc audit Clarify internal-registry pull secret wording and replace see-with-link CTAs with refer to so check-nrl-doc-leakage passes on PR 2166 files. --- docs/docs/extraction/deployment-options.md | 4 +-- docs/docs/extraction/openshift.md | 16 +++++----- .../prerequisites-support-matrix.md | 12 ++++---- nemo_retriever/helm/README.md | 30 +++++++++---------- 4 files changed, 31 insertions(+), 31 deletions(-) diff --git a/docs/docs/extraction/deployment-options.md b/docs/docs/extraction/deployment-options.md index cd16232fc..16635948e 100644 --- a/docs/docs/extraction/deployment-options.md +++ b/docs/docs/extraction/deployment-options.md @@ -22,7 +22,7 @@ Build and run the NeMo Retriever service image with the [Docker service image gu 3. **Published Library Helm charts (supported):** cluster install and upgrade procedures are covered in the [NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) — use alongside the NeMo Retriever chart README for your release 4. [Environment variables](environment-config.md) and [Troubleshoot](troubleshoot.md) as needed -**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (see [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). See [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). +**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). For audio and video extraction in Kubernetes, set `service.installFfmpeg=true` @@ -32,7 +32,7 @@ filesystem, and security policy that allows the image's scoped sudo use. If your cluster blocks startup package installation (for example air-gapped environments or **OpenShift restricted-v2**), use a custom service image that already contains `ffmpeg` and `ffprobe`, then set `service.image.repository` -and `service.image.tag`. For OpenShift overrides, see +and `service.image.tag`. For OpenShift overrides, refer to [OpenShift deployment](openshift.md). ### I want examples and notebooks diff --git a/docs/docs/extraction/openshift.md b/docs/docs/extraction/openshift.md index c08a80efd..4cf2df653 100644 --- a/docs/docs/extraction/openshift.md +++ b/docs/docs/extraction/openshift.md @@ -2,7 +2,7 @@ Use this page when you install the [NeMo Retriever Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) on **OpenShift 4.x** with the default **restricted-v2** Security Context Constraint (SCC) and **Pod Security Admission (PSA) `restricted`** profile. -For general Kubernetes and Helm deployment choices, see [Deployment options](deployment-options.md). For chart values and NIM wiring, see the [Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md). +For general Kubernetes and Helm deployment choices, refer to [Deployment options](deployment-options.md). For chart values and NIM wiring, refer to the [Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md). ## Overview { #openshift-deployment } @@ -25,13 +25,13 @@ On clusters with **PSA `enforce=restricted`**, missing container `securityContex | Symptom on stock install | Cause | Helm override | | --- | --- | --- | | `FailedCreate`: UID/GID **1000** not in namespace range | Hardcoded `service.podSecurityContext` UID/GID/fsGroup | Omit `runAsUser`, `runAsGroup`, and `fsGroup`; keep only `runAsNonRoot: true` | -| PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (see sample below) | +| PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (refer to the sample below) | | `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | | `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Set `serviceConfig.vectordb.enabled=false` for a minimal service-only install, or patch the vectordb Deployment after install (below) | | PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | -| Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (see [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | -| `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart `imagePullSecrets` lists only `ngc-secret`; the namespace ServiceAccount `dockercfg` secret is not merged automatically | Add the SA pull secret under `imagePullSecrets` (see [Internal registry pull secrets](#internal-registry-pull-secrets)) | -| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths via `nimOperator..env` (see [Optional NIM runtime environment](#optional-nim-runtime-environment)) | +| Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (refer to [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | +| `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart-rendered `imagePullSecrets` may omit the namespace SA `dockercfg` secret required for internal-registry pulls | List every required pull secret under `imagePullSecrets` (refer to [Internal registry pull secrets](#internal-registry-pull-secrets)) | +| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths via `nimOperator..env` (refer to [Optional NIM runtime environment](#optional-nim-runtime-environment)) | ### Recommended value overrides @@ -60,7 +60,7 @@ serviceConfig: # Writable without persistence PVC (chart always mounts emptyDir at /tmp). file: /tmp/retriever-service.log vectordb: - # Set false for a minimal service-only install; see vectordb patch below if you enable it. + # Set false for a minimal service-only install; refer to the vectordb patch below if you enable it. enabled: false topology: @@ -88,7 +88,7 @@ Push the result to NGC, your private registry, or the [OpenShift internal regist ### Internal registry pull secrets { #internal-registry-pull-secrets } -When you rebuild the service image into the OpenShift internal registry (`image-registry.openshift-image-registry.svc:5000/...`), pods pull through the namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart renders an explicit `imagePullSecrets` list on every Pod; that list **replaces** the implicit SA default rather than merging with it. If you set only `ngc-secret` (for NGC NIM images) but point `service.image` at the internal registry, the service Pod can fail with `ImagePullBackOff`. +When you rebuild the service image into the OpenShift internal registry (`image-registry.openshift-image-registry.svc:5000/...`), pods normally pull through the namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart renders an explicit `imagePullSecrets` list on every Pod, so you must list **every** secret each Pod needs—including the SA `dockercfg` secret when `service.image` points at the internal registry. If the rendered list contains only `ngc-secret` (for NGC NIM images) while `service.image` uses the internal registry, the service Pod can fail with `ImagePullBackOff`. **Internal-registry service image only** — clear the chart-managed NGC pull secret name so the helper does not inject `ngc-secret` alone: @@ -161,7 +161,7 @@ The retriever service caption profile already sends `chat_template_kwargs.enable } ``` -For pipeline scope (PDF chart regions are not captioned), see [Image captioning](prerequisites-support-matrix.md#image-captioning-2605) and [Image captioning](multimodal-extraction.md#image-captioning) in the extraction docs. +For pipeline scope (PDF chart regions are not captioned), refer to [Image captioning](prerequisites-support-matrix.md#image-captioning-2605) and [Image captioning](multimodal-extraction.md#image-captioning) in the extraction docs. ### Example install (service only, no in-cluster NIMs) diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index a505f75a5..bc24f417f 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -11,7 +11,7 @@ Before you begin using [NeMo Retriever Library](overview.md), confirm your softw - For audio and video, `ffmpeg` and `ffprobe` must be on `PATH` (for example `sudo apt-get install -y --no-install-recommends ffmpeg` on Debian/Ubuntu). `ffmpeg-python` and `nemo-retriever[multimedia]` do not install these binaries. - For container and Kubernetes guidance, see [Audio and video](audio-video.md). + For container and Kubernetes guidance, refer to [Audio and video](audio-video.md). - For PDF extraction with `extract_method="nemotron_parse"`, install the Nemotron Parse client dependencies with `pip install "nemo-retriever[nemotron-parse]"` (pulls `open-clip-torch`, which provides the `open_clip` module required by the Nemotron Parse @@ -78,9 +78,9 @@ The production Helm chart enables these NIM microservices **by default** (for ex !!! note - **Helm / NIM:** The production chart deploys **Nemotron OCR v1** under `nimOperator.ocr` (`nvcr.io/nim/nvidia/nemotron-ocr-v1:1.3.0`). For image defaults and upgrade notes, see [OCR NIM configuration](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#ocr-nim-configuration) in the Helm chart README. + **Helm / NIM:** The production chart deploys **Nemotron OCR v1** under `nimOperator.ocr` (`nvcr.io/nim/nvidia/nemotron-ocr-v1:1.3.0`). For image defaults and upgrade notes, refer to [OCR NIM configuration](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#ocr-nim-configuration) in the Helm chart README. - **Local Hugging Face inference:** When you deploy locally with HuggingFace model weights (for example `pip install "nemo-retriever[local]"` and GPU inference without remote OCR NIM URLs), the default OCR engine is **Nemotron OCR v2**, which runs in **multilingual** mode by default. For CLI flags and API parameters, see [Nemotron OCR v2 — language mode](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/docs/cli/README.md#nemotron-ocr-v2-language-mode). Remote OCR NIM endpoints use their own model and language behavior; local OCR language selectors are not sent on remote requests. + **Local Hugging Face inference:** When you deploy locally with HuggingFace model weights (for example `pip install "nemo-retriever[local]"` and GPU inference without remote OCR NIM URLs), the default OCR engine is **Nemotron OCR v2**, which runs in **multilingual** mode by default. For CLI flags and API parameters, refer to [Nemotron OCR v2 — language mode](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/docs/cli/README.md#nemotron-ocr-v2-language-mode). Remote OCR NIM endpoints use their own model and language behavior; local OCR language selectors are not sent on remote requests. Default OCR NIM container for release Helm deployments: @@ -93,7 +93,7 @@ Default VL embedder container and model for release deployments: ### Optional Helm NIMs (not auto-wired) { #optional-helm-nims-not-auto-wired-by-default } -These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For **26.05 production**, disable keys you do not need (see [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator..enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#nim-operator-sub-stack). +These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For **26.05 production**, disable keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator..enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#nim-operator-sub-stack). | Helm flag | NIM | Role | |-----------|-----|------| @@ -126,7 +126,7 @@ For published NIM model IDs and deployment-specific constraints, use the product NeMo Retriever Library supports the following GPU hardware given system constraints in the table. - **HF model weights** — approximate Hugging Face checkpoint footprint (files such as `model*.safetensors`, `weights.pth`, or other published weight bundles in the model repository). Values are rounded from the current public file listing and can change when the repository is updated. -- **NIM disk space** — approximate container and on-disk model cache for self-hosted NIM microservices (not the same as HF download size). For Nemotron 3 Nano Omni captioning, see the [NVIDIA NIM for Vision Language Models support matrix](https://docs.nvidia.com/nim/vision-language-models/latest/support-matrix.html#nemotron-3-nano-omni-30b-a3b-reasoning). +- **NIM disk space** — approximate container and on-disk model cache for self-hosted NIM microservices (not the same as HF download size). For Nemotron 3 Nano Omni captioning, refer to the [NVIDIA NIM for Vision Language Models support matrix](https://docs.nvidia.com/nim/vision-language-models/latest/support-matrix.html#nemotron-3-nano-omni-30b-a3b-reasoning). Model repositories and NIM references are linked in [Core and Advanced Pipeline Features](#core-and-advanced-pipeline-features) above. @@ -153,7 +153,7 @@ Model repositories and NIM references are linked in [Core and Advanced Pipeline ² Nemotron Parse fails to start on 32GB. -³ Opt-in Omni captioning uses the [nemotron-3-nano-omni-30b-a3b-reasoning](https://docs.api.nvidia.com/nim/reference/nvidia-nemotron-3-nano-omni-30b-a3b-reasoning) NIM (`nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:1.7.0-variant`). BF16 requires at least 80 GB total GPU memory; see the [VLM NIM support matrix](https://docs.nvidia.com/nim/vision-language-models/latest/support-matrix.html#nemotron-3-nano-omni-30b-a3b-reasoning). L40S requires two GPUs. A100 40GB, A10G, and RTX PRO 4500 are below the minimum. +³ Opt-in Omni captioning uses the [nemotron-3-nano-omni-30b-a3b-reasoning](https://docs.api.nvidia.com/nim/reference/nvidia-nemotron-3-nano-omni-30b-a3b-reasoning) NIM (`nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:1.7.0-variant`). BF16 requires at least 80 GB total GPU memory; refer to the [VLM NIM support matrix](https://docs.nvidia.com/nim/vision-language-models/latest/support-matrix.html#nemotron-3-nano-omni-30b-a3b-reasoning). L40S requires two GPUs. A100 40GB, A10G, and RTX PRO 4500 are below the minimum. \* GPUs with less than 80GB VRAM cannot run the reranker concurrently with the core pipeline. To perform recall testing with the reranker on these GPUs, shut down the core pipeline NIM microservices diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index a5cf17d90..dbc314571 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -188,7 +188,7 @@ NIM (the VL reranker `rerankqa`, Nemotron Parse, Omni 30B, and the Parakeet `audio` ASR NIM) is **disabled by default** to honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md); -see [Recommended minimal install (26.05)](#recommended-minimal-install-2605) +refer to [Recommended minimal install (26.05)](#recommended-minimal-install-2605) for the opt-in `--set` flags that turn any of them on. ```bash @@ -218,7 +218,7 @@ helm install retriever ./nemo_retriever/helm \ > * Omni 30B captioner — `--set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true` > * Parakeet ASR — `--set nimOperator.audio.enabled=true` (also set `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` to wire ASR into the service, plus `service.installFfmpeg=true` if your image does not bundle ffmpeg) > -> This matches the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and avoids silently pulling ≈ 62 GiB of Omni weights or claiming a second dedicated GPU on a "default" install. See the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table for per-NIM GPU and disk costs. +> This matches the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and avoids silently pulling ≈ 62 GiB of Omni weights or claiming a second dedicated GPU on a "default" install. Refer to the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table for per-NIM GPU and disk costs. The chart auto-wires the operator-managed in-cluster URLs of the four "core" NIMs into the service's `nim_endpoints` block: @@ -306,11 +306,11 @@ For air-gapped clusters, see To run self-hosted Parakeet for [audio and video extraction](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md): 1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install (26.05)](#recommended-minimal-install-2605)). -2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (see [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). +2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (refer to [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). 3. Confirm the GPU SKU in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) (footnote ⁴ lists Blackwell limitations). -4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (see `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead. +4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (refer to `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead. -The retriever service picks up the in-cluster ASR endpoint when `nimOperator.audio` is enabled; see [NIM Operator sub-stack](#nim-operator-sub-stack). +The retriever service picks up the in-cluster ASR endpoint when `nimOperator.audio` is enabled; refer to [NIM Operator sub-stack](#nim-operator-sub-stack). ### Service configuration (rendered into `retriever-service.yaml`) @@ -318,10 +318,10 @@ The retriever service picks up the in-cluster ASR endpoint when `nimOperator.aud |---------------------------------------------------|---------|-------| | `serviceConfig.server.port` | `7670` | Container + Service port. | | `serviceConfig.pipeline.realtimeWorkers` | `24` | Per-pod realtime worker count. | -| `serviceConfig.pipeline.batchWorkers` | `48` | Per-pod batch worker count. See [Timeouts and alleviating ingest failures](#timeouts-and-alleviating-ingest-failures) if embed or pool errors appear under load. | -| `serviceConfig.nimEndpoints.*InvokeUrl` | `""` | Override the auto-resolved NIM Operator URL. Available knobs: `pageElementsInvokeUrl`, `tableStructureInvokeUrl`, `ocrInvokeUrl`, `embedInvokeUrl`, and `captionInvokeUrl` (see [Image captioning (Omni 30B)](#image-captioning-omni-30b)). | +| `serviceConfig.pipeline.batchWorkers` | `48` | Per-pod batch worker count. Refer to [Timeouts and alleviating ingest failures](#timeouts-and-alleviating-ingest-failures) if embed or pool errors appear under load. | +| `serviceConfig.nimEndpoints.*InvokeUrl` | `""` | Override the auto-resolved NIM Operator URL. Available knobs: `pageElementsInvokeUrl`, `tableStructureInvokeUrl`, `ocrInvokeUrl`, `embedInvokeUrl`, and `captionInvokeUrl` (refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b)). | | `serviceConfig.nimEndpoints.captionModelName` | `""` | Model id sent to the remote VLM. Auto-set to `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` whenever a caption URL is resolved. | -| `serviceConfig.vectordb.enabled` | `true` | Deploy the LanceDB vectordb Pod. When `true` the chart **requires** a resolvable embed endpoint (see [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint)); `helm install` / `helm upgrade` fails fast otherwise. | +| `serviceConfig.vectordb.enabled` | `true` | Deploy the LanceDB vectordb Pod. When `true` the chart **requires** a resolvable embed endpoint (refer to [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint)); `helm install` / `helm upgrade` fails fast otherwise. | | `serviceConfig.vectordb.lancedbUri` | `/data/vectordb` | LanceDB on the vectordb Pod's PVC. | | `serviceConfig.vectordb.embedModel` | `nvidia/llama-nemotron-embed-vl-1b-v2` | Passed to vectordb + worker `embed_model_name`. | @@ -348,7 +348,7 @@ resolved. Pick one of: 3. --set serviceConfig.vectordb.enabled=false ``` -Resolution order matches the rest of the chart (see [Mix and match NIM +Resolution order matches the rest of the chart (refer to [Mix and match NIM sources](#3-install-with-the-nim-operator-in-cluster-nims)): 1. Explicit `serviceConfig.nimEndpoints.embedInvokeUrl` always wins. @@ -379,17 +379,17 @@ pair gated on three conditions ALL holding: | `nimOperator.vlm_embed.image` | `nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:1.12.0` | Default VLM embed NIM image. | | `nimOperator.rerankqa.enabled` | `false` | VL reranker NIM (optional; not auto-wired). Set `true` to opt in. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the **VL** SKU (`llama-nemotron-rerank-vl-1b-v2`) per [prerequisites-support-matrix.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#default-helm-nims) — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR. | | `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `extract_method="nemotron_parse"`. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). Image tag follows the [image tag conventions](#image-tag-conventions). | -| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — see [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so 26.05 installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | +| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so 26.05 installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | | `nimOperator.audio.enabled` | `false` | Parakeet ASR NIM (optional). Set `true` for audio/video transcription; pair with `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` so the retriever-service can reach it. | | `nimOperator..image.repository` | `nvcr.io/nim/nvidia/...` | Per-NIM image. | | `nimOperator..image.pullSecrets` | `[ngc-secret]` | Referenced by the NIMService CR. | | `nimOperator..authSecret` | `ngc-api` | NIM auth Secret name. | | `nimOperator..storage.pvc.size` | `25Gi` (50Gi for vlm_embed/rerankqa, 100Gi parse, 300Gi VL) | NIMCache PVC size. | | `nimOperator..replicas` | `1` | Per-NIMService replica count. | -| `nimOperator.nimServiceGpuLimit` | `1` | Default `nvidia.com/gpu` limit on every NIMService when per-NIM `resources` is `{}`. Set to `null` for operator-only reconciliation (not reliable on all NIM Operator versions — see [GPU limits and `helm upgrade`](#gpu-limits-and-helm-upgrade)). | +| `nimOperator.nimServiceGpuLimit` | `1` | Default `nvidia.com/gpu` limit on every NIMService when per-NIM `resources` is `{}`. Set to `null` for operator-only reconciliation (not reliable on all NIM Operator versions — refer to [GPU limits and `helm upgrade`](#gpu-limits-and-helm-upgrade)). | | `nimOperator..resources` | `{}` | Per-NIM override of the whole `resources` block. Empty uses `nimServiceGpuLimit`; non-empty replaces the chart default (may require `--force-conflicts` on later `helm upgrade`). | -| `nimOperator.modelProfile` | `{}` | Chart-wide NIMCache GPU/profile filter. Applied to every NIMCache that does not have its own override. See [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | -| `nimOperator..modelProfile` | `{}` | Per-NIM NIMCache GPU/profile filter. Non-empty values REPLACE the chart-wide default (no merge). See [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | +| `nimOperator.modelProfile` | `{}` | Chart-wide NIMCache GPU/profile filter. Applied to every NIMCache that does not have its own override. Refer to [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | +| `nimOperator..modelProfile` | `{}` | Per-NIM NIMCache GPU/profile filter. Non-empty values REPLACE the chart-wide default (no merge). Refer to [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | | `nimOperator..expose.service.port` | `8000` (9000 for audio) | HTTP port. | | `nimOperator..expose.service.grpcPort` | `8001` (50051 for audio) | gRPC port. | @@ -950,7 +950,7 @@ This README keeps chart values, NIM wiring, and generic Kubernetes guidance. Pas ## Air-gapped deployment { #air-gapped-deployment } -See [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment) for overview and workflow. Chart-specific reference for mirroring: +Refer to [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment) for overview and workflow. Chart-specific reference for mirroring: ### Container images to mirror (26.05 chart defaults) @@ -1071,7 +1071,7 @@ helm template r nemo_retriever/helm \ ``` Both renders should succeed cleanly and parse as valid Kubernetes manifests -(`kubectl apply --dry-run=client -f /tmp/r.yaml`). See [VectorDB and the +(`kubectl apply --dry-run=client -f /tmp/r.yaml`). Refer to [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint) for why `helm template r nemo_retriever/helm` without flags is rejected as a misconfiguration. From 164b8a2118a2a9f32805b7d65c900c2ef2b6f557 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Fri, 5 Jun 2026 16:12:37 -0700 Subject: [PATCH 10/17] docs: apply NVIDIA style fixes to OpenShift PR prose Replace via/once wording, drop code from headings, disambiguate caption links, and fix split refer-to link CTAs in the Helm README. --- docs/docs/extraction/openshift.md | 16 +++++++++------- nemo_retriever/helm/README.md | 6 ++---- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/docs/extraction/openshift.md b/docs/docs/extraction/openshift.md index 4cf2df653..cd427642f 100644 --- a/docs/docs/extraction/openshift.md +++ b/docs/docs/extraction/openshift.md @@ -31,7 +31,7 @@ On clusters with **PSA `enforce=restricted`**, missing container `securityContex | PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | | Audio/video fails or pod never gets `ffmpeg` | `service.installFfmpeg=true` runs sudo at startup; **restricted-v2** blocks privilege escalation (`no-new-privileges`) | Prebuild a service image with `ffmpeg`/`ffprobe` baked in (refer to [Audio and video on restricted OpenShift](#audio-and-video-ffmpeg-on-restricted-openshift)); leave `service.installFfmpeg=false` | | `ImagePullBackOff` for a service image in the **internal OpenShift registry** | Chart-rendered `imagePullSecrets` may omit the namespace SA `dockercfg` secret required for internal-registry pulls | List every required pull secret under `imagePullSecrets` (refer to [Internal registry pull secrets](#internal-registry-pull-secrets)) | -| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths via `nimOperator..env` (refer to [Optional NIM runtime environment](#optional-nim-runtime-environment)) | +| Optional NIM `CrashLoopBackOff` with missing `.so` in logs | GPU/CUDA libraries not on `LD_LIBRARY_PATH` for some NIM Operator stacks on OCP | Append paths through `nimOperator..env` (refer to [Optional NIM runtime environment](#optional-nim-runtime-environment)) | ### Recommended value overrides @@ -70,9 +70,9 @@ topology: When **`persistence.enabled=true`**, you can keep the default log path under `persistence.mountPath` (`/var/lib/nemo-retriever`) because the PVC is mounted and SCC-assigned `fsGroup` applies. When persistence is off, always relocate logs to `/tmp` (or another path backed by `service.extraVolumes`). -### Audio and video (`ffmpeg`) on restricted OpenShift { #audio-and-video-ffmpeg-on-restricted-openshift } +### Audio and video (ffmpeg) on restricted OpenShift { #audio-and-video-ffmpeg-on-restricted-openshift } -The Helm chart supports `service.installFfmpeg=true`, which installs `ffmpeg`/`ffprobe` at container startup via passwordless `sudo`. On OpenShift **restricted-v2** SCC, that path is blocked: PSA restricted sets `allowPrivilegeEscalation: false` and the SCC enforces **no-new-privileges**, so the entrypoint cannot elevate to install packages. +The Helm chart supports `service.installFfmpeg=true`, which installs `ffmpeg`/`ffprobe` at container startup through passwordless `sudo`. On OpenShift **restricted-v2** SCC, that path is blocked: PSA restricted sets `allowPrivilegeEscalation: false` and the SCC enforces **no-new-privileges**, so the entrypoint cannot elevate to install packages. For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. Instead, extend the service image on a connected build host and point the chart at that tag (same pattern as [air-gapped custom service images](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#1-service-image)): @@ -88,7 +88,9 @@ Push the result to NGC, your private registry, or the [OpenShift internal regist ### Internal registry pull secrets { #internal-registry-pull-secrets } -When you rebuild the service image into the OpenShift internal registry (`image-registry.openshift-image-registry.svc:5000/...`), pods normally pull through the namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart renders an explicit `imagePullSecrets` list on every Pod, so you must list **every** secret each Pod needs—including the SA `dockercfg` secret when `service.image` points at the internal registry. If the rendered list contains only `ngc-secret` (for NGC NIM images) while `service.image` uses the internal registry, the service Pod can fail with `ImagePullBackOff`. +When you rebuild the service image into the OpenShift internal registry (`image-registry.openshift-image-registry.svc:5000/...`), pods normally pull through the namespace ServiceAccount's automatic `kubernetes.io/dockercfg` secret. The chart renders an explicit `imagePullSecrets` list on every Pod, so you must list **every** secret each Pod needs—including the SA `dockercfg` secret when `service.image` points at the internal registry. + +If the rendered list contains only `ngc-secret` (for NGC NIM images) while `service.image` uses the internal registry, the service Pod can fail with `ImagePullBackOff`. **Internal-registry service image only** — clear the chart-managed NGC pull secret name so the helper does not inject `ngc-secret` alone: @@ -98,7 +100,7 @@ ngcImagePullSecret: name: "" # Explicitly empty — clears the default "ngc-secret" imagePullSecrets: - - name: default-dockercfg-xxxxx # replace with your SA secret (see below) + - name: default-dockercfg-xxxxx # replace with your SA secret (section below) ``` **Both NGC NIMs and an internal-registry service image** — list every secret the pods need: @@ -161,7 +163,7 @@ The retriever service caption profile already sends `chat_template_kwargs.enable } ``` -For pipeline scope (PDF chart regions are not captioned), refer to [Image captioning](prerequisites-support-matrix.md#image-captioning-2605) and [Image captioning](multimodal-extraction.md#image-captioning) in the extraction docs. +For pipeline scope (PDF chart regions are not captioned), refer to [Image captioning (support matrix)](prerequisites-support-matrix.md#image-captioning-2605) and [Image captioning (pipeline scope)](multimodal-extraction.md#image-captioning) in the extraction docs. ### Example install (service only, no in-cluster NIMs) @@ -196,7 +198,7 @@ oc get pods -n nemo-retriever oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever ``` -You should see SCC-assigned numeric `runAsUser` on containers that declare a `securityContext` block, and no PSA warnings once overrides are applied. +You should see SCC-assigned numeric `runAsUser` on containers that declare a `securityContext` block, and no PSA warnings after overrides are applied. ### Example install with NIM Operator (in-cluster NIMs) diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index dbc314571..c6d538bef 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -296,10 +296,8 @@ short list of knobs you'll touch first. For audio and video extraction, set `service.installFfmpeg=true` when your cluster allows runtime package installation. **OpenShift restricted-v2** blocks -that path — use a prebuilt service image instead; see -[Audio and video on restricted OpenShift](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift). -For air-gapped clusters, see -[Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment). +that path — use a prebuilt service image instead; refer to [Audio and video on restricted OpenShift](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift). +For air-gapped clusters, refer to [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment). ### Audio and video (Parakeet ASR) { #audio-video-parakeet } From 8e99048f654b57654d81a35b8592cf707961e485 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Mon, 8 Jun 2026 14:41:16 -0700 Subject: [PATCH 11/17] docs: address Greptile review on pull secrets and NIM env overrides Clarify air-gap ngcImagePullSecret.name clearing and add copy-paste-safe LD_LIBRARY_PATH env examples with explicit chart defaults. --- docs/docs/extraction/openshift.md | 2 ++ nemo_retriever/helm/README.md | 7 ++++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/docs/extraction/openshift.md b/docs/docs/extraction/openshift.md index cd427642f..717531466 100644 --- a/docs/docs/extraction/openshift.md +++ b/docs/docs/extraction/openshift.md @@ -133,6 +133,7 @@ Helm **replaces** the whole `env` list when you override `nimOperator..env` nimOperator: audio: env: + # Retain chart defaults, then append LD_LIBRARY_PATH. - name: NIM_TAGS_SELECTOR value: "name=parakeet-1-1b-ctc-en-us,mode=ofl,vad=default,diarizer=disabled" - name: NIM_TRITON_LOG_VERBOSE @@ -141,6 +142,7 @@ nimOperator: value: "/usr/local/nvidia/lib64:/usr/local/cuda/lib64" nemotron_3_nano_omni_30b_a3b_reasoning: env: + # Retain chart defaults, then append LD_LIBRARY_PATH. - name: NIM_HTTP_API_PORT value: "8000" - name: NIM_TRITON_LOG_VERBOSE diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index c6d538bef..81444c071 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -995,7 +995,8 @@ imagePullSecrets: - name: my-private-registry ngcImagePullSecret: - create: false # use secrets that authenticate to YOUR mirror + create: false + name: "" # Explicitly empty — clears the default "ngc-secret" nimOperator: page_elements: @@ -1006,8 +1007,8 @@ nimOperator: # Repeat for table_structure, ocr, vlm_embed, and any optional keys you enable. ``` -- Set `nimOperator..image.pullSecrets` to the Secret name your - `NIMService` resources should use (defaults to `ngc-secret`). +- Set `nimOperator..image.pullSecrets` to your mirror pull secret + (for example `my-private-registry`; chart default is `ngc-secret`). - Leave `serviceConfig.nimEndpoints.*` empty when operator-managed NIMs are in-cluster; set explicit URLs only for external or mirrored services outside the chart. From 061c8ec72903636cd1b17be89942240c0c7f7214 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Tue, 16 Jun 2026 14:13:50 -0700 Subject: [PATCH 12/17] docs: drop 26.05 labels from minimal-install links on main Rename the Helm README minimal-install section, keep a legacy blob/main, and use version-neutral prose in chart default notes. --- docs/docs/extraction/deployment-options.md | 2 +- .../prerequisites-support-matrix.md | 2 +- nemo_retriever/helm/README.md | 22 ++++++++++--------- 3 files changed, 14 insertions(+), 12 deletions(-) diff --git a/docs/docs/extraction/deployment-options.md b/docs/docs/extraction/deployment-options.md index 16635948e..9a8df5098 100644 --- a/docs/docs/extraction/deployment-options.md +++ b/docs/docs/extraction/deployment-options.md @@ -22,7 +22,7 @@ Build and run the NeMo Retriever service image with the [Docker service image gu 3. **Published Library Helm charts (supported):** cluster install and upgrade procedures are covered in the [NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) — use alongside the NeMo Retriever chart README for your release 4. [Environment variables](environment-config.md) and [Troubleshoot](troubleshoot.md) as needed -**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). +**Core NIMs for the default extraction pipeline:** `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). For audio and video extraction in Kubernetes, set `service.installFfmpeg=true` diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index 01f4f1b07..f9bf96a93 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -93,7 +93,7 @@ Default VL embedder container and model for release deployments: ### Optional Helm NIMs (not auto-wired) { #optional-helm-nims-not-auto-wired-by-default } -These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For **26.05 production**, disable keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator..enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#nim-operator-sub-stack). +These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For production deployments, disable keys you do not need (refer to [Recommended minimal install](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator..enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#nim-operator-sub-stack). | Helm flag | NIM | Role | |-----------|-----|------| diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 81444c071..4c604cb76 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -188,7 +188,7 @@ NIM (the VL reranker `rerankqa`, Nemotron Parse, Omni 30B, and the Parakeet `audio` ASR NIM) is **disabled by default** to honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md); -refer to [Recommended minimal install (26.05)](#recommended-minimal-install-2605) +refer to [Recommended minimal install](#recommended-minimal-install-2605) for the opt-in `--set` flags that turn any of them on. ```bash @@ -199,7 +199,9 @@ helm install retriever ./nemo_retriever/helm \ --set ngcApiSecret.password=$NGC_API_KEY ``` -### Recommended minimal install (26.05) { #recommended-minimal-install-2605 } +### Recommended minimal install { #recommended-minimal-install } + + Deploy only the four core NIMs that the retriever service auto-wires (`page_elements`, `table_structure`, `ocr`, `vlm_embed`): @@ -211,7 +213,7 @@ helm install retriever ./nemo_retriever/helm \ --set ngcApiSecret.password=$NGC_API_KEY ``` -> The VL reranker (`rerankqa`), Nemotron Parse, the Nemotron 3 Nano Omni 30B caption NIM, and the Parakeet `audio` ASR NIM are **all off by default** in 26.05 — they only reconcile when you explicitly opt in. Opt-in flags: +> The VL reranker (`rerankqa`), Nemotron Parse, the Nemotron 3 Nano Omni 30B caption NIM, and the Parakeet `audio` ASR NIM are **all off by default** — they only reconcile when you explicitly opt in. Opt-in flags: > > * VL reranker — `--set nimOperator.rerankqa.enabled=true` > * Nemotron Parse — `--set nimOperator.nemotron_parse.enabled=true` @@ -303,7 +305,7 @@ For air-gapped clusters, refer to [Deployment options — Air-gapped and disconn To run self-hosted Parakeet for [audio and video extraction](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md): -1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install (26.05)](#recommended-minimal-install-2605)). +1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install](#recommended-minimal-install-2605)). 2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (refer to [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). 3. Confirm the GPU SKU in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) (footnote ⁴ lists Blackwell limitations). 4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (refer to `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead. @@ -375,9 +377,9 @@ pair gated on three conditions ALL holding: | `nimOperator.vlm_embed.enabled` | `true` | Multimodal embedding NIM (also used by the vectordb Pod). | | `nimOperator.vlm_embed.nimServiceName` | `llama-nemotron-embed-vl-1b-v2` | NIMService / in-cluster DNS name. | | `nimOperator.vlm_embed.image` | `nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:1.12.0` | Default VLM embed NIM image. | -| `nimOperator.rerankqa.enabled` | `false` | VL reranker NIM (optional; not auto-wired). Set `true` to opt in. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the **VL** SKU (`llama-nemotron-rerank-vl-1b-v2`) per [prerequisites-support-matrix.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#default-helm-nims) — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR. | -| `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `extract_method="nemotron_parse"`. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). Image tag follows the [image tag conventions](#image-tag-conventions). | -| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so 26.05 installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | +| `nimOperator.rerankqa.enabled` | `false` | VL reranker NIM (optional; not auto-wired). Set `true` to opt in. Default `false` so default installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the **VL** SKU (`llama-nemotron-rerank-vl-1b-v2`) per [prerequisites-support-matrix.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#default-helm-nims) — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR. | +| `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `extract_method="nemotron_parse"`. Default `false` so default installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). Image tag follows the [image tag conventions](#image-tag-conventions). | +| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so default installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | | `nimOperator.audio.enabled` | `false` | Parakeet ASR NIM (optional). Set `true` for audio/video transcription; pair with `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` so the retriever-service can reach it. | | `nimOperator..image.repository` | `nvcr.io/nim/nvidia/...` | Per-NIM image. | | `nimOperator..image.pullSecrets` | `[ngc-secret]` | Referenced by the NIMService CR. | @@ -395,7 +397,7 @@ pair gated on three conditions ALL holding: > are auto-wired into the retriever-service config. Optional NIMs may reconcile > when `nimOperator..enabled` is `true` in `values.yaml`, but the > retriever-service won't call them unless you wire your pipeline to use them. -> For 26.05, prefer the [minimal install](#recommended-minimal-install-2605) overrides. +> Prefer the [recommended minimal install](#recommended-minimal-install-2605) overrides. #### Filtering cached GPU profiles { #filtering-cached-gpu-profiles } @@ -489,7 +491,7 @@ and **ocr** (no `graphic_elements` operator NIM in this chart). For image captioning, set `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true` — see [Image captioning (Omni 30B)](#image-captioning-omni-30b) for the chart-side wiring and -[Image captioning (26.05)](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-2605) +[Image captioning — NIM and hardware](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-nim-hardware) for the product matrix. #### Image captioning (Omni 30B) { #image-captioning-omni-30b } @@ -1015,7 +1017,7 @@ nimOperator: - For **offline captioning**, enable `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning` and point the pipeline caption endpoint at the in-cluster NIM URL (see - [Image captioning (26.05)](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-2605)). + [Image captioning — NIM and hardware](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-nim-hardware)). ### Mirroring pattern From 32446d9f5106be61a4c37e0229bd5541af53b7fa Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Tue, 16 Jun 2026 14:19:19 -0700 Subject: [PATCH 13/17] Revert "docs: drop 26.05 labels from minimal-install links on main" This reverts commit 061c8ec72903636cd1b17be89942240c0c7f7214. --- docs/docs/extraction/deployment-options.md | 2 +- .../prerequisites-support-matrix.md | 2 +- nemo_retriever/helm/README.md | 22 +++++++++---------- 3 files changed, 12 insertions(+), 14 deletions(-) diff --git a/docs/docs/extraction/deployment-options.md b/docs/docs/extraction/deployment-options.md index 9a8df5098..16635948e 100644 --- a/docs/docs/extraction/deployment-options.md +++ b/docs/docs/extraction/deployment-options.md @@ -22,7 +22,7 @@ Build and run the NeMo Retriever service image with the [Docker service image gu 3. **Published Library Helm charts (supported):** cluster install and upgrade procedures are covered in the [NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) — use alongside the NeMo Retriever chart README for your release 4. [Environment variables](environment-config.md) and [Troubleshoot](troubleshoot.md) as needed -**Core NIMs for the default extraction pipeline:** `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). +**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). For audio and video extraction in Kubernetes, set `service.installFfmpeg=true` diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index f9bf96a93..01f4f1b07 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -93,7 +93,7 @@ Default VL embedder container and model for release deployments: ### Optional Helm NIMs (not auto-wired) { #optional-helm-nims-not-auto-wired-by-default } -These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For production deployments, disable keys you do not need (refer to [Recommended minimal install](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator..enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#nim-operator-sub-stack). +These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For **26.05 production**, disable keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator..enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#nim-operator-sub-stack). | Helm flag | NIM | Role | |-----------|-----|------| diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 4c604cb76..81444c071 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -188,7 +188,7 @@ NIM (the VL reranker `rerankqa`, Nemotron Parse, Omni 30B, and the Parakeet `audio` ASR NIM) is **disabled by default** to honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md); -refer to [Recommended minimal install](#recommended-minimal-install-2605) +refer to [Recommended minimal install (26.05)](#recommended-minimal-install-2605) for the opt-in `--set` flags that turn any of them on. ```bash @@ -199,9 +199,7 @@ helm install retriever ./nemo_retriever/helm \ --set ngcApiSecret.password=$NGC_API_KEY ``` -### Recommended minimal install { #recommended-minimal-install } - - +### Recommended minimal install (26.05) { #recommended-minimal-install-2605 } Deploy only the four core NIMs that the retriever service auto-wires (`page_elements`, `table_structure`, `ocr`, `vlm_embed`): @@ -213,7 +211,7 @@ helm install retriever ./nemo_retriever/helm \ --set ngcApiSecret.password=$NGC_API_KEY ``` -> The VL reranker (`rerankqa`), Nemotron Parse, the Nemotron 3 Nano Omni 30B caption NIM, and the Parakeet `audio` ASR NIM are **all off by default** — they only reconcile when you explicitly opt in. Opt-in flags: +> The VL reranker (`rerankqa`), Nemotron Parse, the Nemotron 3 Nano Omni 30B caption NIM, and the Parakeet `audio` ASR NIM are **all off by default** in 26.05 — they only reconcile when you explicitly opt in. Opt-in flags: > > * VL reranker — `--set nimOperator.rerankqa.enabled=true` > * Nemotron Parse — `--set nimOperator.nemotron_parse.enabled=true` @@ -305,7 +303,7 @@ For air-gapped clusters, refer to [Deployment options — Air-gapped and disconn To run self-hosted Parakeet for [audio and video extraction](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md): -1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install](#recommended-minimal-install-2605)). +1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install (26.05)](#recommended-minimal-install-2605)). 2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (refer to [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). 3. Confirm the GPU SKU in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) (footnote ⁴ lists Blackwell limitations). 4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (refer to `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead. @@ -377,9 +375,9 @@ pair gated on three conditions ALL holding: | `nimOperator.vlm_embed.enabled` | `true` | Multimodal embedding NIM (also used by the vectordb Pod). | | `nimOperator.vlm_embed.nimServiceName` | `llama-nemotron-embed-vl-1b-v2` | NIMService / in-cluster DNS name. | | `nimOperator.vlm_embed.image` | `nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:1.12.0` | Default VLM embed NIM image. | -| `nimOperator.rerankqa.enabled` | `false` | VL reranker NIM (optional; not auto-wired). Set `true` to opt in. Default `false` so default installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the **VL** SKU (`llama-nemotron-rerank-vl-1b-v2`) per [prerequisites-support-matrix.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#default-helm-nims) — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR. | -| `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `extract_method="nemotron_parse"`. Default `false` so default installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). Image tag follows the [image tag conventions](#image-tag-conventions). | -| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so default installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | +| `nimOperator.rerankqa.enabled` | `false` | VL reranker NIM (optional; not auto-wired). Set `true` to opt in. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the **VL** SKU (`llama-nemotron-rerank-vl-1b-v2`) per [prerequisites-support-matrix.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#default-helm-nims) — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR. | +| `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `extract_method="nemotron_parse"`. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). Image tag follows the [image tag conventions](#image-tag-conventions). | +| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so 26.05 installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | | `nimOperator.audio.enabled` | `false` | Parakeet ASR NIM (optional). Set `true` for audio/video transcription; pair with `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` so the retriever-service can reach it. | | `nimOperator..image.repository` | `nvcr.io/nim/nvidia/...` | Per-NIM image. | | `nimOperator..image.pullSecrets` | `[ngc-secret]` | Referenced by the NIMService CR. | @@ -397,7 +395,7 @@ pair gated on three conditions ALL holding: > are auto-wired into the retriever-service config. Optional NIMs may reconcile > when `nimOperator..enabled` is `true` in `values.yaml`, but the > retriever-service won't call them unless you wire your pipeline to use them. -> Prefer the [recommended minimal install](#recommended-minimal-install-2605) overrides. +> For 26.05, prefer the [minimal install](#recommended-minimal-install-2605) overrides. #### Filtering cached GPU profiles { #filtering-cached-gpu-profiles } @@ -491,7 +489,7 @@ and **ocr** (no `graphic_elements` operator NIM in this chart). For image captioning, set `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true` — see [Image captioning (Omni 30B)](#image-captioning-omni-30b) for the chart-side wiring and -[Image captioning — NIM and hardware](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-nim-hardware) +[Image captioning (26.05)](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-2605) for the product matrix. #### Image captioning (Omni 30B) { #image-captioning-omni-30b } @@ -1017,7 +1015,7 @@ nimOperator: - For **offline captioning**, enable `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning` and point the pipeline caption endpoint at the in-cluster NIM URL (see - [Image captioning — NIM and hardware](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-nim-hardware)). + [Image captioning (26.05)](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-2605)). ### Mirroring pattern From 71eaf8671c44503dd129f78179b58e7d5cc354fd Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Mon, 22 Jun 2026 10:17:59 -0700 Subject: [PATCH 14/17] docs: defer Helm audio/video and OpenShift config to chart README Replace inline Helm value guidance in deployment-options with links to the audio-video and OpenShift sections per PR review. --- docs/docs/extraction/deployment-options.md | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/docs/docs/extraction/deployment-options.md b/docs/docs/extraction/deployment-options.md index 16635948e..8ac654502 100644 --- a/docs/docs/extraction/deployment-options.md +++ b/docs/docs/extraction/deployment-options.md @@ -24,16 +24,7 @@ Build and run the NeMo Retriever service image with the [Docker service image gu **Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). - -For audio and video extraction in Kubernetes, set `service.installFfmpeg=true` -so the service container installs `ffmpeg` and `ffprobe` at startup. This -runtime install requires package-repository network egress, a writable root -filesystem, and security policy that allows the image's scoped sudo use. If -your cluster blocks startup package installation (for example air-gapped -environments or **OpenShift restricted-v2**), use a custom service image that -already contains `ffmpeg` and `ffprobe`, then set `service.image.repository` -and `service.image.tag`. For OpenShift overrides, refer to -[OpenShift deployment](openshift.md). +For audio and video dependencies and OpenShift-specific Helm configuration, refer to [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-video-parakeet) and [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#openshift-deployment) in the Helm chart README. ### I want examples and notebooks From c48738e4529f5e331870376de28340c114953227 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Mon, 22 Jun 2026 10:23:58 -0700 Subject: [PATCH 15/17] docs: move OpenShift guide into nemo_retriever/helm/openshift.md Relocate OpenShift install content from the extraction doc site into the Helm chart directory per review. Update README, deployment-options, and mkdocs nav to link to the Helm-local guide. --- docs/docs/extraction/deployment-options.md | 2 +- docs/mkdocs.yml | 2 +- nemo_retriever/helm/README.md | 9 ++++----- .../helm}/openshift.md | 20 +++++++++---------- 4 files changed, 16 insertions(+), 17 deletions(-) rename {docs/docs/extraction => nemo_retriever/helm}/openshift.md (87%) diff --git a/docs/docs/extraction/deployment-options.md b/docs/docs/extraction/deployment-options.md index 8ac654502..d1c0d05f7 100644 --- a/docs/docs/extraction/deployment-options.md +++ b/docs/docs/extraction/deployment-options.md @@ -24,7 +24,7 @@ Build and run the NeMo Retriever service image with the [Docker service image gu **Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). -For audio and video dependencies and OpenShift-specific Helm configuration, refer to [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-video-parakeet) and [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#openshift-deployment) in the Helm chart README. +For audio and video dependencies and OpenShift-specific Helm configuration, refer to [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-video-parakeet) and [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/openshift.md) in the Helm chart directory. ### I want examples and notebooks diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index d88532158..3f59f0a8f 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -84,7 +84,7 @@ nav: - "Authentication and API keys": extraction/api-keys.md - "3. Deployment options": - "Compare deployment options": extraction/deployment-options.md - - "OpenShift deployment": extraction/openshift.md + - "OpenShift deployment (Helm)": https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/openshift.md - "Helm chart (Kubernetes)": https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/helm - "Docker service image": https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/docker.md - "4. Core workflows": diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 81444c071..31c31a80a 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -50,6 +50,7 @@ nemo_retriever/helm/ ├── Chart.yaml ├── values.yaml ├── README.md <-- this file +├── openshift.md <-- OpenShift restricted-v2 install guide ├── .helmignore └── templates/ ├── _helpers.tpl @@ -296,7 +297,7 @@ short list of knobs you'll touch first. For audio and video extraction, set `service.installFfmpeg=true` when your cluster allows runtime package installation. **OpenShift restricted-v2** blocks -that path — use a prebuilt service image instead; refer to [Audio and video on restricted OpenShift](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift). +that path — use a prebuilt service image instead; refer to [Audio and video on restricted OpenShift](./openshift.md#audio-and-video-ffmpeg-on-restricted-openshift). For air-gapped clusters, refer to [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment). ### Audio and video (Parakeet ASR) { #audio-video-parakeet } @@ -306,7 +307,7 @@ To run self-hosted Parakeet for [audio and video extraction](https://github.com/ 1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install (26.05)](#recommended-minimal-install-2605)). 2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (refer to [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). 3. Confirm the GPU SKU in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) (footnote ⁴ lists Blackwell limitations). -4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (refer to `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead. +4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (refer to `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](./openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead. The retriever service picks up the in-cluster ASR endpoint when `nimOperator.audio` is enabled; refer to [NIM Operator sub-stack](#nim-operator-sub-stack). @@ -940,9 +941,7 @@ sanity check before opening Grafana. ## OpenShift deployment { #openshift-deployment } -OpenShift install procedures, **restricted-v2** / PSA **restricted** value overrides, prebuilt `ffmpeg` images, internal registry pull secrets, optional NIM `LD_LIBRARY_PATH` tuning, and install examples are in **[OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/openshift.md)** (published extraction docs). - -This README keeps chart values, NIM wiring, and generic Kubernetes guidance. Pass `-f openshift-restricted.yaml` from that guide when you install on OpenShift. +OpenShift install procedures, **restricted-v2** / PSA **restricted** value overrides, prebuilt `ffmpeg` images, internal registry pull secrets, optional NIM `LD_LIBRARY_PATH` tuning, and install examples are in **[OpenShift deployment](./openshift.md)**. Pass `-f openshift-restricted.yaml` from that guide when you install on OpenShift. --- diff --git a/docs/docs/extraction/openshift.md b/nemo_retriever/helm/openshift.md similarity index 87% rename from docs/docs/extraction/openshift.md rename to nemo_retriever/helm/openshift.md index 717531466..5fbc283d4 100644 --- a/docs/docs/extraction/openshift.md +++ b/nemo_retriever/helm/openshift.md @@ -1,8 +1,8 @@ # OpenShift deployment -Use this page when you install the [NeMo Retriever Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) on **OpenShift 4.x** with the default **restricted-v2** Security Context Constraint (SCC) and **Pod Security Admission (PSA) `restricted`** profile. +Use this guide when you install the [NeMo Retriever Helm chart](./README.md) on **OpenShift 4.x** with the default **restricted-v2** Security Context Constraint (SCC) and **Pod Security Admission (PSA) `restricted`** profile. -For general Kubernetes and Helm deployment choices, refer to [Deployment options](deployment-options.md). For chart values and NIM wiring, refer to the [Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md). +For general Kubernetes and Helm deployment choices, refer to [Deployment options](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/). For chart values and NIM wiring, refer to the [Helm chart README](./README.md). ## Overview { #openshift-deployment } @@ -35,7 +35,7 @@ On clusters with **PSA `enforce=restricted`**, missing container `securityContex ### Recommended value overrides -Save the block below as a local values file (for example `openshift-restricted.yaml`) and pass `-f openshift-restricted.yaml` on every `helm install` / `helm upgrade`. Use this profile for **restricted-v2** / PSA **restricted** namespaces with the NIM Operator, a prebuilt `ffmpeg` service image when you need [audio and video](audio-video.md), and optional NIM env overrides from later sections. +Save the block below as a local values file (for example `openshift-restricted.yaml`) and pass `-f openshift-restricted.yaml` on every `helm install` / `helm upgrade`. Use this profile for **restricted-v2** / PSA **restricted** namespaces with the NIM Operator, a prebuilt `ffmpeg` service image when you need [audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md), and optional NIM env overrides from later sections. ```yaml # OpenShift overrides for nemo-retriever Helm chart (restricted-v2 / PSA restricted). @@ -74,7 +74,7 @@ When **`persistence.enabled=true`**, you can keep the default log path under `pe The Helm chart supports `service.installFfmpeg=true`, which installs `ffmpeg`/`ffprobe` at container startup through passwordless `sudo`. On OpenShift **restricted-v2** SCC, that path is blocked: PSA restricted sets `allowPrivilegeEscalation: false` and the SCC enforces **no-new-privileges**, so the entrypoint cannot elevate to install packages. -For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. Instead, extend the service image on a connected build host and point the chart at that tag (same pattern as [air-gapped custom service images](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#1-service-image)): +For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. Instead, extend the service image on a connected build host and point the chart at that tag (same pattern as [air-gapped custom service images](./README.md#1-service-image)): ```dockerfile FROM nvcr.io/nvstaging/nim/nrl-service: @@ -165,7 +165,7 @@ The retriever service caption profile already sends `chat_template_kwargs.enable } ``` -For pipeline scope (PDF chart regions are not captioned), refer to [Image captioning (support matrix)](prerequisites-support-matrix.md#image-captioning-2605) and [Image captioning (pipeline scope)](multimodal-extraction.md#image-captioning) in the extraction docs. +For pipeline scope (PDF chart regions are not captioned), refer to [Image captioning (support matrix)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#image-captioning-2605) and [Image captioning (pipeline scope)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/multimodal-extraction.md#image-captioning) in the extraction docs. ### Example install (service only, no in-cluster NIMs) @@ -222,7 +222,7 @@ oc get pods -n nemo-retriever oc get nimservice -n nemo-retriever ``` -When you enable optional NIMs, wait for Parakeet ASR and Omni caption pods to become Ready before you run [audio and video](audio-video.md) or caption extraction. If ASR or Omni pods crash with missing `.so` errors, apply the [LD_LIBRARY_PATH overrides](#optional-nim-runtime-environment) above and upgrade the release. +When you enable optional NIMs, wait for Parakeet ASR and Omni caption pods to become Ready before you run [audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md) or caption extraction. If ASR or Omni pods crash with missing `.so` errors, apply the [LD_LIBRARY_PATH overrides](#optional-nim-runtime-environment) above and upgrade the release. ### Enabling the vectordb Deployment on OpenShift @@ -255,7 +255,7 @@ Do **not** bind the namespace to **anyuid** SCC or set PSA `enforce=privileged` ## Related topics -- [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md) -- [Deployment options](deployment-options.md) -- [Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) -- [Audio and video](audio-video.md) +- [Pre-Requisites & Support Matrix](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md) +- [Deployment options](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/) +- [Helm chart README](./README.md) +- [Audio and video](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md) From 65d63f5d218beb2d75995e1678fc7d8e9a31b66a Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Mon, 22 Jun 2026 13:37:14 -0700 Subject: [PATCH 16/17] docs: update prerequisites matrix OCR defaults to v2 (Randy review) Address PR #2166 review: main branch Helm chart defaults to Nemotron OCR v2 (nemotron-ocr-v2:1.4.0). Remaining OCR documentation and chart code in #2259. --- docs/docs/extraction/prerequisites-support-matrix.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/docs/extraction/prerequisites-support-matrix.md b/docs/docs/extraction/prerequisites-support-matrix.md index 01f4f1b07..1649d2bcf 100644 --- a/docs/docs/extraction/prerequisites-support-matrix.md +++ b/docs/docs/extraction/prerequisites-support-matrix.md @@ -71,20 +71,20 @@ The production Helm chart enables these NIM microservices **by default** (for ex |-----------|-----|------| | `page_elements` | [nemotron-page-elements-v3](https://huggingface.co/nvidia/nemotron-page-elements-v3) | Page layout and element detection | | `table_structure` | [nemotron-table-structure-v1](https://huggingface.co/nvidia/nemotron-table-structure-v1) | Table structure extraction | -| `ocr` | [nemotron-ocr-v1](https://huggingface.co/nvidia/nemotron-ocr-v1) | Image OCR | +| `ocr` | [nemotron-ocr-v2](https://huggingface.co/nvidia/nemotron-ocr-v2) | Image OCR | | `vlm_embed` | [llama-nemotron-embed-vl-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-vl-1b-v2) | Multimodal (VL) embedding | ### OCR artifacts (Helm vs local Hugging Face) { #nemotron-ocr-v2-language-mode } !!! note - **Helm / NIM:** The production chart deploys **Nemotron OCR v1** under `nimOperator.ocr` (`nvcr.io/nim/nvidia/nemotron-ocr-v1:1.3.0`). For image defaults and upgrade notes, refer to [OCR NIM configuration](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#ocr-nim-configuration) in the Helm chart README. + **Helm / NIM:** The production chart deploys **Nemotron OCR v2** under `nimOperator.ocr` (`nvcr.io/nim/nvidia/nemotron-ocr-v2:1.4.0`). For image defaults and upgrade notes, refer to [OCR NIM configuration](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#ocr-nim-configuration) in the Helm chart README. **Local Hugging Face inference:** When you deploy locally with HuggingFace model weights (for example `pip install "nemo-retriever[local]"` and GPU inference without remote OCR NIM URLs), the default OCR engine is **Nemotron OCR v2**, which runs in **multilingual** mode by default. For CLI flags and API parameters, refer to [Nemotron OCR v2 — language mode](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/docs/cli/README.md#nemotron-ocr-v2-language-mode). Remote OCR NIM endpoints use their own model and language behavior; local OCR language selectors are not sent on remote requests. Default OCR NIM container for release Helm deployments: -- **Image:** `nvcr.io/nim/nvidia/nemotron-ocr-v1:1.3.0` +- **Image:** `nvcr.io/nim/nvidia/nemotron-ocr-v2:1.4.0` Default VL embedder container and model for release deployments: From e799395978d9287b5ac9f2084d52d9959e301074 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Mon, 22 Jun 2026 13:45:13 -0700 Subject: [PATCH 17/17] docs(helm): point service image docs at GA nrl-service:26.5.0 Replace pre-release nvstaging references with the official NGC image that matches values.yaml. Update OpenShift install examples to match. --- nemo_retriever/helm/README.md | 10 +++++----- nemo_retriever/helm/openshift.md | 10 +++++----- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 31c31a80a..c41f836b0 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -81,13 +81,13 @@ nemo_retriever/helm/ ### 1. Service image { #1-service-image } -The chart defaults to the staging image published to NGC: +The chart defaults to the GA image published to NGC: ``` -nvcr.io/nvstaging/nim/nemo-retriever-service:043020205-001 +nvcr.io/nvidia/nemo-microservices/nrl-service:26.5.0 ``` -Pulling from `nvcr.io/nvstaging` requires an NGC pull secret — either set +Pulling from `nvcr.io` requires an NGC pull secret — either set `ngcImagePullSecret.create=true` (see below) or pre-create one in the namespace named `ngc-secret`. @@ -287,8 +287,8 @@ short list of knobs you'll touch first. | Path | Default | Notes | |-------------------------------|------------------------------------|-------| -| `service.image.repository` | `localhost:32000/nemo-retriever-service` | Override to a published image. | -| `service.image.tag` | `latest` | | +| `service.image.repository` | `nvcr.io/nvidia/nemo-microservices/nrl-service` | GA NGC image; override to pin a different build or use a local registry. | +| `service.image.tag` | `26.5.0` | | | `service.replicas` | `1` | Hard cap = 1 while SQLite is the backend. | | `service.installFfmpeg` | `false` | Install `ffmpeg`/`ffprobe` at container startup by setting `INSTALL_FFMPEG=true`. Requires network egress, writable root filesystem, and sudo/setuid allowed. Not for air-gapped clusters — use a custom image instead. | | `service.resources.requests` | `16 / 16Gi` | Tune in tandem with `serviceConfig.pipeline.*Workers`. | diff --git a/nemo_retriever/helm/openshift.md b/nemo_retriever/helm/openshift.md index 5fbc283d4..cda227314 100644 --- a/nemo_retriever/helm/openshift.md +++ b/nemo_retriever/helm/openshift.md @@ -77,7 +77,7 @@ The Helm chart supports `service.installFfmpeg=true`, which installs `ffmpeg`/`f For audio and video extraction on OpenShift, **do not** set `service.installFfmpeg=true`. Instead, extend the service image on a connected build host and point the chart at that tag (same pattern as [air-gapped custom service images](./README.md#1-service-image)): ```dockerfile -FROM nvcr.io/nvstaging/nim/nrl-service: +FROM nvcr.io/nvidia/nemo-microservices/nrl-service: USER root RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \ && rm -rf /var/lib/apt/lists/* @@ -189,8 +189,8 @@ helm install retriever ./nemo_retriever/helm -n nemo-retriever \ --set nims.enabled=false \ --set persistence.enabled=false \ --set retrieverResults.enabled=false \ - --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ - --set service.image.tag= + --set service.image.repository=nvcr.io/nvidia/nemo-microservices/nrl-service \ + --set service.image.tag=26.5.0 ``` Verify pods: @@ -211,8 +211,8 @@ helm install retriever ./nemo_retriever/helm -n nemo-retriever \ -f openshift-restricted.yaml \ --set ngcImagePullSecret.create=false \ --set ngcApiSecret.create=false \ - --set service.image.repository=nvcr.io/nvstaging/nim/nrl-service \ - --set service.image.tag= + --set service.image.repository=nvcr.io/nvidia/nemo-microservices/nrl-service \ + --set service.image.tag=26.5.0 ``` After install, confirm workloads reach Ready before you run ingest: