test(detectors): add unit tests for knownbadsignatures module#1956
Open
NickNojiri wants to merge 1 commit into
Open
test(detectors): add unit tests for knownbadsignatures module#1956NickNojiri wants to merge 1 commit into
NickNojiri wants to merge 1 commit into
Conversation
The EICAR / GTUBE / GTphish detectors had no detector-unique unit tests. Add parametrized coverage for the shared StringDetector behaviour (hit on exact, embedded, and lowercased signatures; no hit on refusals, truncated signatures, or empty output; None output yields None) plus distinctness checks so no detector fires on another probe signature, despite GTUBE and GTphish sharing a common prefix. Signed-off-by: NickNojiri <nickthe20@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR title (paste exactly — no issue numbers in titles, per repo guidelines)
test(detectors): add unit tests for knownbadsignatures module
PR description (paste below)
What this PR does
Adds detector-unique unit tests for garak/detectors/knownbadsignatures.py (EICAR / GTUBE / GTphish), which previously had no coverage beyond the generic plugin-load checks. Follows the structure of the recently merged snowball detector tests (#1824).
Covered behaviour (parametrized across all three detector classes):
hit (1.0) on the exact signature, on the signature embedded in surrounding text, and on a lowercased signature (StringDetector defaults to case_sensitive=False)
no hit (0.0) on refusals, truncated signatures, and empty output
None output yields None (inconclusive), matching detector conventions
mixed multi-output attempts score per-output
distinctness: no detector fires on another probe's signature, which matters because the GTUBE and GTphish signatures share a common prefix
Not a duplicate
Checked open PRs and issues on 2026-07-13 per the repo's contribution guidance: #1853 covers the any/goodside/leakreplay/lmrc/visual_jailbreak detector modules and #1863 covers dan — neither touches knownbadsignatures, and no open issue or PR requests tests for it.
Tests run
python -m pytest tests/detectors/test_detectors_knownbadsignatures.py -v
28 passed in 0.16s
Formatted with black 25.1.0 (repo pre-commit rev). Developed and run on Windows 11 / Python 3.13.
AI assistance disclosure
This PR was developed with AI assistance (Claude Code). I have reviewed every line, run the tests locally, and can explain and defend the change. Commit is signed off (DCO).