Show probe name in detector progress bar#1942
Conversation
The detector progress bar previously showed only `detectors.<name>`, leaving users unable to tell which probe's results were being checked when multiple probes share the same detector in a run. Include the probe name in the description so the current probe is always visible. Adds a regression test that wraps tqdm to capture set_description calls and asserts both names appear in the detector progress bar. Fixes NVIDIA#324 Co-authored-by: Claude Signed-off-by: fei <3303354867@qq.com>
39cef28 to
6f83043
Compare
|
Hi @jmartin-tech — noting that the CI failures on the ubuntu/macos jobs are from The CI log shows this is a HuggingFace The force-push to fix a missing DCO signoff on the rerun-trigger commit set the workflow runs to |
6f83043 to
a1559a9
Compare
jmartin-tech
left a comment
There was a problem hiding this comment.
Testing for land is queued, I removed the empty commit and triggered CI tests. If there is another intermittent failure a maintainer will clear it.
|
Thanks @jmartin-tech for the update and for clearing the empty commit — appreciate the hand on the intermittent CI failure. Standing by if anything else is needed. |
|
Hi @jmartin-tech — gentle nudge on this one. CI has been green (CLEAN state) since 7/16, just confirming whether anything is still needed on my side before this can be merged. Happy to rebase or address any final feedback. Thanks! |
|
Follow-up to my earlier ping — I dug into the two failing build jobs and confirmed they are both HuggingFace download timeouts, not code regressions:
|
What
The detector progress bar previously showed only
detectors.<name>in itsdescription, leaving users unable to tell which probe's results were being
checked when multiple probes share the same detector in a single run.
This change prepends the probe name to the description, producing e.g.
probes.test.Blank/detectors.always.Pass.Fixes #324
Why
Issue #324 reports that the probe name is hidden in the detector tqdm
description. In a multi-probe run with shared detectors, the progress bar
gives no indication of which probe is currently being processed, so the user
cannot track progress through the run.
Approach
probe.probename.replace("garak.", "")once per probe (matchingthe pattern already used in
garak/probes/base.py:325,363) and prepend itto the existing detector description using a
/separator. This separatormatches the precedent in
garak/buffs/base.py:84(
f"📥 Buffing probe: {probename}/{self.fullname}").ProbewiseHarness(harnesses/probewise.py:110) andPxD(
harnesses/pxd.py:60) delegate tobase.Harness.run()viasuper().run(), so a single change inbase.pycovers all harnessvariants.
classes.
Why this is not a duplicate
Verified via GitHub search on 2026-07-11:
repo:NVIDIA/garak is:pr 324→ 0 resultsrepo:NVIDIA/garak is:pr is:open progress bar probe→ 2 results(PR Feat: report technique and intent tabs lists #1879 HTML report UI — no overlap; PR feat: Add probe-level resume (runservice) and report digest fixes #1589 resume feature — does not
touch
garak/harnesses/base.pydespite description mentioning it)PR #1718 (open, 2026-04-27, "Log detector skip info", fixes #1061) also
touches
garak/harnesses/base.pybut modifies theif d.skip:block(lines 162-175) and appends a
_log_detector_skip_info()function. This PRmodifies only the
set_descriptioncall (line 200) and adds one line beforethe
for d in detectors:loop (line 194). The changes are non-overlappingand git auto-merges cleanly.
Verification
python -m pytest tests/harnesses/test_harnesses.py -v→ 4 passedpython -m pytest tests/test_internal_structures.py -v→ 17 passed(adjacent suite — tests harness plugin cache behavior)
python -m black --check garak/harnesses/base.py tests/harnesses/test_harnesses.py→ clean (line-length 88)
python -m garak -m test -p test.Blank -d always.Pass -g 5shows
probes.test.Blank/detectors.always.Pass: 0%|...in the detectorprogress bar
test_harness_detector_progress_shows_probe_namewrapstqdm.tqdmto captureset_descriptioncalls and asserts bothprobes.test.Blankanddetectors.always.Passappear in the descriptionpython -m pytest tests/— full suite hangs at collection time on thismachine (garak's plugin imports require network/optional dependencies
not available locally; PR Log detector skip info #1718 reports the same issue). Targeted
relevant subsets (harnesses, internal_structures) all pass.
AI assistance
This change was developed with AI assistance (Claude). The change has been
reviewed and tested end-to-end by the human submitter. Declared via the
Co-authored-by: Claudetrailer on commita1559a9b, perAGENTS.md.