Skip to content

fix(interactive): do_probe drops a first-time probe name and can crash with UnboundLocalError#1965

Open
chuenchen309 wants to merge 1 commit into
NVIDIA:mainfrom
chuenchen309:fix/interactive-do-probe-argparser-bugs
Open

fix(interactive): do_probe drops a first-time probe name and can crash with UnboundLocalError#1965
chuenchen309 wants to merge 1 commit into
NVIDIA:mainfrom
chuenchen309:fix/interactive-do-probe-argparser-bugs

Conversation

@chuenchen309

@chuenchen309 chuenchen309 commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Two bugs in GarakCommands.do_probe (garak/interactive.py):

  1. The probe-name branch only covers "overwrite an existing probe" (if args.probe and self._cmd.probe) and "neither given" (elif not args.probe and not self._cmd.probe). The case of a probe name given for the first time (self._cmd.probe was falsy, args.probe is truthy) matches neither branch, so self._cmd.probe is silently never set — probe myprobe run as the very first probe command in a session does nothing.
  2. If loading the generator raises ImportError or AttributeError, the except blocks only log/print and fall through — generator was never assigned, so the next line, harness.run(generator, ...), raises UnboundLocalError instead of the intended graceful message.

Fix

Add an explicit elif args.probe: self._cmd.probe = args.probe branch for the first-time-set case, and return after each except block so a failed generator load stops the command instead of falling through to code that needs a generator that was never created.

Verification

  • Added tests/test_interactive.py (no prior test coverage existed for this module): calls GarakCommands.do_probe.__wrapped__ directly (bypassing the @cmd2.with_argparser decorator, so no full cmd2.Cmd registration or garak config is needed), with ThresholdEvaluator and ProbewiseHarness mocked out to isolate the control-flow bug from unrelated config-loading concerns.
  • Confirmed red→green: reverting only interactive.py reproduces both symptoms exactly (probe stays None; UnboundLocalError on generator); reapplying passes.
  • black --check clean on both changed files.
  • xref: fix(lint): tune pylintrc and fix violations across core modules #1793 ("fix(lint): tune pylintrc and fix violations across core modules") also touches do_probe, but only for an f-string→%s logging conversion and a trailing return None, not either of these two logic bugs — not a duplicate, though a trivial rebase may be needed if it merges first.
  • garak -t <target_type> -n <model_name>not run: this is a unit-level fix and I have no live target configured; verified via the tests above rather than ticking a box I didn't exercise.
  • Supporting configuration such as a generator configuration file — n/a, no config surface changed.

I used AI assistance (Claude) to help investigate and draft this fix, but I personally constructed the repro, reviewed the root cause, and reviewed the final diff before submitting.

Signed-off-by: Andrew Chen 48723787+chuenchen309@users.noreply.github.com

@jmartin-tech

Copy link
Copy Markdown
Collaborator

DCO signoff must be in the commit message

…h with UnboundLocalError

Two bugs in GarakCommands.do_probe (garak/interactive.py):

1. The probe-name branch only covers "overwrite an existing probe"
   (`if args.probe and self._cmd.probe`) and "neither given" (`elif not
   args.probe and not self._cmd.probe`). The case of a probe name given
   for the FIRST TIME (self._cmd.probe was falsy, args.probe is truthy)
   matches neither branch, so self._cmd.probe is silently never set --
   `probe myprobe` run as the very first `probe` command in a session
   does nothing.

2. If loading the generator raises ImportError or AttributeError, the
   except blocks only log/print and fall through -- `generator` was
   never assigned, so the next line, `harness.run(generator, ...)`,
   raises UnboundLocalError instead of the intended graceful message.

Fix: add an explicit `elif args.probe: self._cmd.probe = args.probe`
branch for the first-time-set case, and `return` after each except
block so a failed generator load stops the command instead of falling
through to code that needs a `generator` that was never created.

Added tests/test_interactive.py (no prior test coverage existed for
this module): calls GarakCommands.do_probe.__wrapped__ directly
(bypassing the @cmd2.with_argparser decorator, so no full cmd2.Cmd
registration or garak config is needed), with ThresholdEvaluator and
ProbewiseHarness mocked out to isolate the control-flow bug from
unrelated config-loading. TDD-verified red/green: reverting only
interactive.py reproduces both symptoms exactly (probe stays None;
UnboundLocalError on 'generator'); reapplying passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Andrew Chen <48723787+chuenchen309@users.noreply.github.com>
@chuenchen309
chuenchen309 force-pushed the fix/interactive-do-probe-argparser-bugs branch from 0d2be91 to 75f28a1 Compare July 15, 2026 14:18
@chuenchen309

Copy link
Copy Markdown
Contributor Author

Thanks @jmartin-tech — added the DCO sign-off; the commit now carries Signed-off-by: and the DCO check is passing. Appreciate the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants