Skip to content

fix(scanner): prevent is_code_example() from hard-dropping SKILL.md findings - #381

Open
glatinone wants to merge 1 commit into
NVIDIA:mainfrom
glatinone:fix/issue-373-skillmd-code-example-guard
Open

fix(scanner): prevent is_code_example() from hard-dropping SKILL.md findings#381
glatinone wants to merge 1 commit into
NVIDIA:mainfrom
glatinone:fix/issue-373-skillmd-code-example-guard

Conversation

@glatinone

Copy link
Copy Markdown

Summary of Changes

is_code_example() in common.py now takes an optional path keyword argument and returns False whenever path ends with SKILL.md — mirroring the guard _is_documentation_context() already has. The one call site that can hard-drop a finding (static_runner._scan_path, static_runner.py:358) now passes the file path through, so SKILL.md findings are never suppressed by nearby documentation-style phrasing.

Root Cause Analysis

_scan_path hard-drops (continue) any non-PE3 finding whose 3-line context matches an indicator like for example, e.g., or a backtick fence, when the file's type is in _NON_EXECUTABLE_FILE_TYPES (markdown, text, json, yaml, toml). SKILL.md is classified as markdown, so it hit this non-executable hard-drop path — even though SKILL.md is the primary instruction/attack surface for a skill, not incidental documentation.

The codebase already recognized this class of problem for _is_documentation_context(), which explicitly excludes SKILL.md (static_runner.py:259). is_code_example() had no equivalent guard, so an attacker could place an innocuous phrase such as "For example" a few lines from an injected instruction (e.g. Ignore all previous instructions...) to have the finding dropped entirely, regardless of rule ID (all rules except PE3, which is separately excluded).

Test Coverage & Verification

  • Added TestSkillMdCodeExampleGuard in test_static_runner_filtering.py, reproducing the issue's PoC almost verbatim: a P1 (Instruction Override) finding a few lines below "For example" in SKILL.md now survives, while the same content in a plain docs/*.md file is still filtered (existing suppression behavior for genuine documentation is preserved).
  • Confirmed test fails against the pre-fix code (verified via git stash) and passes after the fix.
  • One pre-existing test, test_code_example_is_downgraded in test_static_patterns_anti_refusal.py, asserted the same vulnerable behavior for AR1 on SKILL.md (silently relying on the bug to suppress a never refuse finding). Split it into: the original case moved to a generic docs/example.md path (still correctly downgraded there), plus a new test_code_example_is_not_downgraded_in_skill_md asserting the AR1 finding now correctly survives on SKILL.md.
  • Full suite: uv run pytest — 2141 passed, 21 skipped, 4 xfailed (excluding pre-existing Windows-only failures in test_build_context.py, test_create_github_release.py, test_input_handler.py that are present identically on main before this change, unrelated to this fix).
  • ruff check, ruff format --check, and mypy all pass clean on the changed files.

Closes #373

…_code_example (NVIDIA#373)

is_code_example() had no SKILL.md exclusion, unlike _is_documentation_context()
which already special-cases it. Since SKILL.md's file_type ("markdown") is
non-executable, any finding within 3 lines of an indicator like "for example"
or a backtick fence was silently dropped in static_runner._scan_path -
including HIGH-confidence prompt-injection findings on the primary attack
surface.

Add the same SKILL.md guard to is_code_example() via an optional path kwarg,
and pass the file path at the one call site that hard-drops on it. A
pre-existing anti_refusal test asserted the vulnerable behavior for AR1 on
SKILL.md; it is split into a generic-markdown case (still downgraded) and a
SKILL.md case (now correctly preserved).

Signed-off-by: glatinone <93207632+glatinone@users.noreply.github.com>
@glatinone
glatinone force-pushed the fix/issue-373-skillmd-code-example-guard branch from 0699945 to 6141366 Compare August 16, 2026 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[SECURITRY] is_code_example() hard-drops findings from .md files

1 participant