fix: tier-1 defcon special-casing is a dead branch in digest_to_tbsa#1958
Open
chuenchen309 wants to merge 1 commit into
Open
fix: tier-1 defcon special-casing is a dead branch in digest_to_tbsa#1958chuenchen309 wants to merge 1 commit into
chuenchen309 wants to merge 1 commit into
Conversation
`if probe_detector in tiers[1]:` compares the joined "probe+detector" key against tiers[1], which only ever contains bare probe names (built via `tiers[Tier(...)].append(probename)` a few lines above) -- so this condition is never true, and every probe:detector pair falls through to the `else: pd_defcon = dc_scores["relative"]` branch regardless of tier. The correct pattern already exists a few lines below, in the t1_dc/t2_dc list comprehensions: `pd.split(PROBE_DETECTOR_SEP)[0] in tiers[1]`. A second, stacked dead check compounds this: `isinstance(dc_scores ["relative"], float)` is also always False, since defcon values are always ints (1-5 integer scale) -- so even after fixing the key comparison, the intended `min(relative, absolute)` computation for tier-1 pairs would still be unreachable, silently using only `absolute_defcon` instead. Fix both: split the joined key before the tiers[1] membership check, and check `isinstance(..., int)` instead of `float`. Verified real behavioral impact: a synthetic tier-1 pair with absolute_defcon=2, relative_defcon=5 produces tbsa=5 (buggy, using relative_defcon unconditionally) vs tbsa=2 (fixed, using the intended min(5, 2)=2) -- a significant swing in this security-severity score. Signed-off-by: Andrew Chen <48723787+chuenchen309@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Andrew Chen <48723787+chuenchen309@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
digest_to_tbsa's tier-1 special-casing never actually triggers, due to two stacked dead conditions:if probe_detector in tiers[1]:compares the joined"probe:detector"key againsttiers[1], which only ever contains bare probe names (built viatiers[Tier(...)].append(probename)a few lines above) — so this is never true, and every probe:detector pair falls through toelse: pd_defcon = dc_scores["relative"]regardless of tier.isinstance(dc_scores["relative"], float)is also always False, since defcon values are always ints (1-5 scale) — so the intendedmin(relative, absolute)computation for tier-1 pairs would still be unreachable, silently using onlyabsolute_defcon.The correct pattern already exists a few lines below in this same function, in the
t1_dc/t2_dclist comprehensions:pd.split(PROBE_DETECTOR_SEP)[0] in tiers[1].Fix
Split the joined key before the
tiers[1]membership check, and checkisinstance(..., int)instead offloat.Verification
absolute_defcon=2,relative_defcon=5) and confirmedtbsacomputed as5before the fix,2after — demonstrating the dead branch has a real, observable effect on the final score.tests/analyze/test_tbsa.pycovering this case.tests/analyze/suite: 142 passed.garak -t <target_type> -n <model_name>— not run: this is a unit-level fix and I have no live target configured; verified via the tests above rather than ticking a box I didn't exercise.I used AI assistance (Claude) to help investigate and draft this fix, but I personally constructed the repro, reviewed the root cause, and reviewed the final diff before submitting.
Signed-off-by: Andrew Chen 48723787+chuenchen309@users.noreply.github.com