Repository navigation
feat(helm,guardrails): self-check logs by default, output check off - #133
Merged
Merged
Conversation
The self-check model answered a free-text Yes/No, and NeMo's parser treats
an unclear answer as a block, so default installs refused ordinary
requests (SQL tool calls, a caller's own balances).
- selfCheck.input (default log), selfCheck.output (default none),
selfCheck.timeoutMs (default 1500); each side is log, block or none.
- zelkor_self_check action: the model answers {"block": true|false};
an unclear, slow or failed answer lets the request through and logs
guardrail_unparseable / guardrail_unavailable. Verdicts log
guardrail_verdict without prompt bodies.
- Set input/output to block to refuse with safetyRefusal, as before.
- Tests: render tests per mode, unit tests for the action; provider tests
set output=log where they assert the output model; stale OpenTelemetry
0.66b1 pin assertions updated to 0.66b2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
devopssquaddev
added a commit
that referenced
this pull request
Nov 17, 2026
…elemetry 1.45.2) (#134) * fix(guardrails): keep guardrail_verdict logs after the NeMo CLI import Importing nemoguardrails.cli sets the root logger to WARNING, after boot.py configured Zelkor logging. Every zelkor-nemo INFO line after startup was dropped, including the guardrail_verdict that log mode promises (#133); warnings still printed. Pin the zelkor-nemo logger to ZELKOR_LOG_LEVEL after the import; the root, and NeMo's own INFO logs that carry user text, stay at WARNING. Verified on a live cluster: verdict lines appear, no prompt text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(platform): release 2.5.3 (guardrails self-check log mode, OpenTelemetry 1.45.2) PATCH 2.5.3, released together with the 2.5.2 changes (2.5.2 was not tagged): - #133: NeMo self-check input defaults to log (record, never refuse), output off; selfCheck.input/output: block keeps refusing. - #127: OpenTelemetry 1.45.2 / 0.66b2 for CVE-2026-126360. - charts, first-party image tags, Zelkor Dockerfile bases and example charts at 2.5.3; example chart dependencies rebuilt - profiles/values-production.yaml at tag 2.5.3, digests empty until the signed release run - docs/security/cve-log.md: CVE-2026-126360 fixed in v2.5.3-ce - THIRD_PARTY_NOTICES: 2.5.3 row Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * release: S-gate evidence for v2.5.3-ce S-gate against the 2.5.3 candidate on the production-shaped lab: 103 tests, 0 failures, 11 environment skips. Default self-check mode (log) for everything except the guardrail-refusal cases (S-12, S-13, S-81, UAT-12, drop-in harmful), which ran under the selfCheck.input: block opt-in. Tool-policy cases (S-71/72/73) ran under a temporary overlay. File paths removed. Fail-closed matrix: PASS 6, GAP 1 (DNS egress). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The default NeMo self-check asked the model for a free-text Yes/No, and NeMo's parser treats an unclear answer as a block. Default installs refused ordinary requests (read-only SQL tool calls, a caller's own balances), depending on which model answered.
What changes
workspace.policies.nemo.selfCheck.inputloglog(record the verdict, never refuse),block(refuse withsafetyRefusal),noneworkspace.policies.nemo.selfCheck.outputnoneworkspace.policies.nemo.selfCheck.timeoutMs1500zelkor_self_checkaction in the guardrails image: the model answers{"block": true|false}. An unclear, slow or failed answer lets the request through and logsguardrail_unparseable/guardrail_unavailable; every verdict logsguardrail_verdict(no prompt bodies).selfCheck.input: blockandselfCheck.output: block.docs/nemo-guardrails.mddocuments the new keys.Tests
tests/test_helm_nemo_self_check_flows.py(render per mode, invalid mode fails) andtests/test_guardrails_self_check.py(JSON parsing, log never refuses, unclear/slow/failed answers pass through). 19 pass.output=logwhere they assert the output model; stale OpenTelemetry0.66b1pin assertions →0.66b2(already failing onmain).test_llms_txtNode ESM errors that also fail onmain.logandblockmode.Release note
Behaviour change for the default install: harmful prompts are now logged, not refused, unless
selfCheck.input: blockis set. New Helm keys → MINOR bump.🤖 Generated with Claude Code