fix(scanner): prevent is_code_example() from hard-dropping SKILL.md findings - #381
Open
glatinone wants to merge 1 commit into
Open
fix(scanner): prevent is_code_example() from hard-dropping SKILL.md findings#381glatinone wants to merge 1 commit into
glatinone wants to merge 1 commit into
Conversation
…_code_example (NVIDIA#373) is_code_example() had no SKILL.md exclusion, unlike _is_documentation_context() which already special-cases it. Since SKILL.md's file_type ("markdown") is non-executable, any finding within 3 lines of an indicator like "for example" or a backtick fence was silently dropped in static_runner._scan_path - including HIGH-confidence prompt-injection findings on the primary attack surface. Add the same SKILL.md guard to is_code_example() via an optional path kwarg, and pass the file path at the one call site that hard-drops on it. A pre-existing anti_refusal test asserted the vulnerable behavior for AR1 on SKILL.md; it is split into a generic-markdown case (still downgraded) and a SKILL.md case (now correctly preserved). Signed-off-by: glatinone <93207632+glatinone@users.noreply.github.com>
glatinone
force-pushed
the
fix/issue-373-skillmd-code-example-guard
branch
from
August 16, 2026 12:34
0699945 to
6141366
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
is_code_example()incommon.pynow takes an optionalpathkeyword argument and returnsFalsewheneverpathends withSKILL.md— mirroring the guard_is_documentation_context()already has. The one call site that can hard-drop a finding (static_runner._scan_path,static_runner.py:358) now passes the file path through, so SKILL.md findings are never suppressed by nearby documentation-style phrasing.Root Cause Analysis
_scan_pathhard-drops (continue) any non-PE3 finding whose 3-line context matches an indicator likefor example,e.g., or a backtick fence, when the file's type is in_NON_EXECUTABLE_FILE_TYPES(markdown, text, json, yaml, toml).SKILL.mdis classified asmarkdown, so it hit this non-executable hard-drop path — even thoughSKILL.mdis the primary instruction/attack surface for a skill, not incidental documentation.The codebase already recognized this class of problem for
_is_documentation_context(), which explicitly excludesSKILL.md(static_runner.py:259).is_code_example()had no equivalent guard, so an attacker could place an innocuous phrase such as "For example" a few lines from an injected instruction (e.g.Ignore all previous instructions...) to have the finding dropped entirely, regardless of rule ID (all rules exceptPE3, which is separately excluded).Test Coverage & Verification
TestSkillMdCodeExampleGuardintest_static_runner_filtering.py, reproducing the issue's PoC almost verbatim: aP1(Instruction Override) finding a few lines below "For example" inSKILL.mdnow survives, while the same content in a plaindocs/*.mdfile is still filtered (existing suppression behavior for genuine documentation is preserved).git stash) and passes after the fix.test_code_example_is_downgradedintest_static_patterns_anti_refusal.py, asserted the same vulnerable behavior forAR1onSKILL.md(silently relying on the bug to suppress anever refusefinding). Split it into: the original case moved to a genericdocs/example.mdpath (still correctly downgraded there), plus a newtest_code_example_is_not_downgraded_in_skill_mdasserting theAR1finding now correctly survives onSKILL.md.uv run pytest— 2141 passed, 21 skipped, 4 xfailed (excluding pre-existing Windows-only failures intest_build_context.py,test_create_github_release.py,test_input_handler.pythat are present identically onmainbefore this change, unrelated to this fix).ruff check,ruff format --check, andmypyall pass clean on the changed files.Closes #373