Skip to content

feat(helm,guardrails): self-check logs by default, output check off - #133

Merged
devopssquaddev merged 1 commit into
mainfrom
feat/platform-guardrails-defaults
Nov 17, 2026
Merged

devopssquaddev merged 1 commit into
mainfrom
feat/platform-guardrails-defaults

Conversation

@devopssquaddev

Copy link
Copy Markdown
Owner

Why

The default NeMo self-check asked the model for a free-text Yes/No, and NeMo's parser treats an unclear answer as a block. Default installs refused ordinary requests (read-only SQL tool calls, a caller's own balances), depending on which model answered.

What changes

Value Default Options
workspace.policies.nemo.selfCheck.input log log (record the verdict, never refuse), block (refuse with safetyRefusal), none
workspace.policies.nemo.selfCheck.output none same; off by default because it adds a second model call per request
workspace.policies.nemo.selfCheck.timeoutMs 1500 per check
  • New zelkor_self_check action in the guardrails image: the model answers {"block": true|false}. An unclear, slow or failed answer lets the request through and logs guardrail_unparseable / guardrail_unavailable; every verdict logs guardrail_verdict (no prompt bodies).
  • To keep the previous refusing behaviour, set selfCheck.input: block and selfCheck.output: block.
  • docs/nemo-guardrails.md documents the new keys.

Tests

  • New: tests/test_helm_nemo_self_check_flows.py (render per mode, invalid mode fails) and tests/test_guardrails_self_check.py (JSON parsing, log never refuses, unclear/slow/failed answers pass through). 19 pass.
  • Updated: provider tests set output=log where they assert the output model; stale OpenTelemetry 0.66b1 pin assertions → 0.66b2 (already failing on main).
  • Full offline suite: 1169 passed. Remaining failures need a live cluster (connection refused) or are the test_llms_txt Node ESM errors that also fail on main.
  • Not yet run live: guardrails image build, benign/harmful prompts against a cluster in log and block mode.

Release note

Behaviour change for the default install: harmful prompts are now logged, not refused, unless selfCheck.input: block is set. New Helm keys → MINOR bump.

🤖 Generated with Claude Code

The self-check model answered a free-text Yes/No, and NeMo's parser treats
an unclear answer as a block, so default installs refused ordinary
requests (SQL tool calls, a caller's own balances).

- selfCheck.input (default log), selfCheck.output (default none),
  selfCheck.timeoutMs (default 1500); each side is log, block or none.
- zelkor_self_check action: the model answers {"block": true|false};
  an unclear, slow or failed answer lets the request through and logs
  guardrail_unparseable / guardrail_unavailable. Verdicts log
  guardrail_verdict without prompt bodies.
- Set input/output to block to refuse with safetyRefusal, as before.
- Tests: render tests per mode, unit tests for the action; provider tests
  set output=log where they assert the output model; stale OpenTelemetry
  0.66b1 pin assertions updated to 0.66b2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devopssquaddev
devopssquaddev merged commit fb893d6 into main Nov 17, 2026
11 checks passed
devopssquaddev added a commit that referenced this pull request Nov 17, 2026
…elemetry 1.45.2) (#134)

* fix(guardrails): keep guardrail_verdict logs after the NeMo CLI import

Importing nemoguardrails.cli sets the root logger to WARNING, after
boot.py configured Zelkor logging. Every zelkor-nemo INFO line after
startup was dropped, including the guardrail_verdict that log mode
promises (#133); warnings still printed. Pin the zelkor-nemo logger to
ZELKOR_LOG_LEVEL after the import; the root, and NeMo's own INFO logs
that carry user text, stay at WARNING. Verified on a live cluster:
verdict lines appear, no prompt text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(platform): release 2.5.3 (guardrails self-check log mode, OpenTelemetry 1.45.2)

PATCH 2.5.3, released together with the 2.5.2 changes (2.5.2 was not
tagged):
- #133: NeMo self-check input defaults to log (record, never refuse),
  output off; selfCheck.input/output: block keeps refusing.
- #127: OpenTelemetry 1.45.2 / 0.66b2 for CVE-2026-126360.
- charts, first-party image tags, Zelkor Dockerfile bases and example
  charts at 2.5.3; example chart dependencies rebuilt
- profiles/values-production.yaml at tag 2.5.3, digests empty until the
  signed release run
- docs/security/cve-log.md: CVE-2026-126360 fixed in v2.5.3-ce
- THIRD_PARTY_NOTICES: 2.5.3 row

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* release: S-gate evidence for v2.5.3-ce

S-gate against the 2.5.3 candidate on the production-shaped lab: 103
tests, 0 failures, 11 environment skips. Default self-check mode (log)
for everything except the guardrail-refusal cases (S-12, S-13, S-81,
UAT-12, drop-in harmful), which ran under the selfCheck.input: block
opt-in. Tool-policy cases (S-71/72/73) ran under a temporary overlay.
File paths removed. Fail-closed matrix: PASS 6, GAP 1 (DNS egress).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant