Repository navigation
chore(platform): release 2.5.5 (trace error levels, guardrail fields on the model call, CLI fixes) - #138
Merged
Conversation
…rail result on the model call; sandbox denied status
A trace review of the 2.5.4 gate run found failures that Langfuse showed at
the normal level, and guardrail fields that were hard to find.
- a self-check with no verdict (timeout, error, unparseable) makes the
check's model call and the guardrail span ERROR spans with a message
such as "self-check timeout; request allowed", plus error.type
- the check result (zelkor.guardrail.*, Langfuse guardrail_* metadata) is
on NeMo's self_check_<side> model call and, per side, on the NeMo
request span; the separate guardrail span stays
- a 4xx answer from NeMo is an ERROR span ("HTTP 429"); OTel left 4xx
server spans unset
- sandbox execute results report status "denied" when the sandbox refused
a probe and the script carried on, instead of "error"; the Deep Agents
backend still maps it to exit code 1
Level comes from the OTel span status, as the uniform trace contract
requires; no langfuse.observation.level.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…k timeouts are known trace errors On the production-shaped lab 3 of 8 FinServe quant self-checks stopped at 5 s while the same checks finished in about 2 s under a 15 s limit (the reasoning model sometimes thinks long). Now that a timed-out check is an ERROR span, a 5 s default would fail runs that are otherwise healthy. - selfCheck.timeoutMs default 5000 -> 15000 - trace hygiene in tests skips a log-mode self-check timeout (the request went through); it stays an ERROR in Langfuse. Block mode and other statuses still count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r too A FinServe quant check ran 6.9 s and returned an empty answer (the reasoning model spent its budget thinking); in log mode the request went through, as with a timeout. A failed check call (error) still counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a cosign timeout as FAIL - `zelkor status --env staging` (the documented form, six places in docs) failed with "unrecognized arguments"; every subcommand now takes --env, overriding the global flag - `zelkor verify`: a cosign call that hits the 120 s limit is a FAIL line with "timed out after 120s" and the summary still prints, instead of a traceback Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on the model call, CLI fixes) - charts, first-party image tags, Zelkor Dockerfile bases and example charts at 2.5.5; example chart dependencies rebuilt - profiles/values-production.yaml at tag 2.5.5, digests empty until the signed release run - THIRD_PARTY_NOTICES: 2.5.5 row (no third-party changes) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
S-gate against the 2.5.5 candidate on the production-shaped lab: default self-check mode (log), the guardrail-refusal cases (S-12, S-13, S-81, UAT-12, drop-in harmful) under the selfCheck.input: block opt-in, S-71/72/73 under a temporary tool-policy overlay, and the Qdrant tenant tests re-run after a test-env label fix. 103 tests, 0 failures, 11 skipped. File paths removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Release 2.5.5 (PATCH): fixes from the trace review of the 2.5.4 gate run, plus two
zelkorCLI bugs.self-check timeout; request allowed;HTTP 429).zelkor.guardrail.*/ Langfuseguardrail_*metadata is now on NeMo'sself_check_<side> <model>call, and per side (guardrail_input_mode, …) on the NeMo request span. The 2.5.4guardrail self_check <side>span stays.status: "denied"(not"error") when the sandbox refused a probe and the script carried on.selfCheck.timeoutMsdefault 5000 → 15000. FinServe quant checks sometimes ran past 5 s (the reasoning model thinks long); at 15 s every check finished in about 2 s.--envnow works after the command (zelkor status --env staging, as the docs show);zelkor verifyreports a cosign timeout as FAIL instead of a traceback.Verification on the production-shaped lab
self-check timeout; request allowedHTTP 422mknodprobestatus: denied; benign codesuccessrelease-evidence/v2.5.5-ce)🤖 Generated with Claude Code