Skip to content

docs(review): skip the challenger when there are no findings - #1009

Draft
guyoron1 wants to merge 1 commit into
fullsend-ai:mainfrom
guyoron1:feat/review-challenger-skip
Draft

docs(review): skip the challenger when there are no findings#1009
guyoron1 wants to merge 1 commit into
fullsend-ai:mainfrom
guyoron1:feat/review-challenger-skip

Conversation

@guyoron1

@guyoron1 guyoron1 commented Aug 25, 2026

Copy link
Copy Markdown

The challenger is an opus dispatch whose only inputs are the merged
findings; on a clean PR it adjudicates an empty list at the cost of a
serial opus call. Skip it when the set is empty and say so in the
orchestrator's own reasoning. Verdict logic is unchanged — an empty set
approved before and approves now.

What changed

skills/pr-review/SKILL.md only, two hunks:

  1. Step 6d (challenger pass) — before dispatch, added: if the merged
    finding set from steps 6a–6c is empty, skip the dispatch entirely
    and proceed to the verdict with the empty set. Applies regardless
    of why it's empty (a dimension failure already surfaces via
    existing error handling). Nothing else in 6d changes — dispatch
    mechanics, output parsing, and the fallback-on-challenger-failure
    path are untouched.
  2. The re-review dispatch rule (~line 347): **Challenger** — always dispatch (unchanged).**Challenger** — always dispatch when findings exist (step 6d).

Before/after — step 6d intro

Before:

After steps 6a–6c produce a merged finding set, dispatch the
`challenger` sub-agent to adversarially challenge the findings with
fresh context. The challenger has not seen the orchestrator's synthesis
— it receives only the raw findings and the diff, preserving context
isolation.

1. Compose the spawn prompt from:

After:

After steps 6a–6c produce a merged finding set, dispatch the
`challenger` sub-agent to adversarially challenge the findings with
fresh context. The challenger has not seen the orchestrator's synthesis
— it receives only the raw findings and the diff, preserving context
isolation.

**Skip when there is nothing to adjudicate.** If the merged finding set
from steps 6a–6c is empty, skip the challenger dispatch entirely and
proceed straight to the verdict with the empty set — the empty-findings
approval path (step 7) is unchanged; this skip does not add anything to
it. This applies regardless of *why* the set is empty: a dimension
dispatch failure already surfaces via existing error handling (the
`sub-agent-failure` info finding below), and the challenger's job is to
adjudicate findings it is given, not manufacture them from nothing.
(This does forfeit the challenger's secondary, not-owned allowance —
see `sub-agents/challenger.md`'s "Do not own" section — to flag a
genuine issue it happens to notice while checking an empty set against
the diff. Accepted: exercising that allowance would mean re-reading the
whole diff on every clean PR, which is exactly the cost this skip
exists to avoid.) Note `challenger: skipped (no findings to adjudicate)`
in your own reasoning for auditability — there is no field for it in
`agent-result.json` (`schemas/review-result.schema.json` is
`additionalProperties: false`), and it does not belong in the posted
review body.

Otherwise, dispatch the challenger:

1. Compose the spawn prompt from:

Design notes

  • Where the rule sits. The challenger dispatch is step 6d
    ("Challenger pass (dedicated sub-agent)"), a subsection of step 6
    (Synthesis) — step 7 is "Produce the review result" (final output
    formatting), unrelated. The skip rule sits immediately before the 6d
    dispatch, and the "always dispatch" cross-reference at line 347 now
    says 6d.
  • Why the skip note lives in the reasoning trace, not the review
    body.
    Step 7 has an explicit rule: "No footer. Do not
    append any footer, action-hints block, or boilerplate after
    findings," and the empty-findings case has a fixed template ("Looks
    good to me", no findings section). The output JSON schema
    (schemas/review-result.schema.json, referenced from
    agents/review.md) is also additionalProperties: false with no
    free-form metadata field. Forcing the note into body or
    findings[] (the way agents#1008's analogous "excluded-content" disclosure
    does, via an info-level finding) would fire on every clean PR — the
    modal case — and change the empty-findings approval output this PR
    promises to leave untouched. So the note lives in the orchestrator's
    own reasoning trace instead: auditable if someone inspects the
    transcript, invisible to the PR author, no schema/body change.
  • Challenger's role. Read sub-agents/challenger.md in full per
    the task's "read first" list, specifically checking for any role
    beyond adjudication. Its "Own"/"Do not own" section says: "Do not
    own: Generating new findings... If you discover a genuine issue not
    covered by any finding, note it — but your primary job is quality
    control of the existing set." That's a secondary, explicitly
    not-owned allowance, not a primary responsibility — but it does mean
    skipping dispatch on an empty set forfeits that safety net (the rare
    case where the challenger notices something no dimension flagged,
    purely from reading the diff). Judged this as not a blocking
    contradiction (the design's core reasoning — "adjudicates an empty
    list," "a challenger cannot conjure findings from an empty input" —
    holds for the challenger's primary role) but noting it explicitly
    per the task's instruction to report if I found one, rather than
    silently proceeding or silently stopping.

Test evidence

Prompt-only change, no script test (per the task spec). Before/after
text is quoted above. eval/review/cases/006-docs-only and
007-dependency-bump (from open PR agents#1003, read to confirm before
citing) both expect approval with zero-or-near-zero findings — 006
allows an info-level note, 007 allows one low-severity observation — so
neither is a guaranteed-empty-set run, but on the runs that do land
empty, this change removes one Opus dispatch and one serial latency
segment. No other tests exist for this eval harness yet on this branch.

Rebase notes

  • Open PR agents#1004 adds step 6g (finding cap) between the current
    6f and 7 — confirmed 6g does not exist yet on upstream main. Doesn't
    overlap this change (6d), sits further down.
  • Open PR agents#1006 (feat/review-untrusted-text) edits step
    6d's Part 3/Part 4 blocks (untrusted-text fencing) — those are the
    numbered sub-steps under "Otherwise, dispatch the challenger:" below
    my new paragraph. Different hunks, but noting the adjacency since
    whoever lands second will need to rebase past the other.

Scope

The challenger exists only on the claude review path — the pi-runtime
review runs without sub-agents — so this change is complete for the only
path that has a challenger to skip.

The challenger is an opus dispatch whose only inputs are the merged
findings from steps 6a-6c; on a clean PR it adjudicates an empty list
at the cost of a serial opus call and latency segment. Skip the
dispatch when the set is empty and proceed straight to the verdict -
the empty-findings approval path is unchanged. The skip is noted in
the orchestrator's own reasoning for auditability; there is no field
for it in the strict agent-result.json schema and it does not belong
in the posted review body.

Signed-off-by: guy oron <goron@redhat.com>
@github-actions

Copy link
Copy Markdown

Functional tests did not run

Functional tests run automatically for org/repo members and collaborators on pull requests.

For other contributors, a maintainer must add the ok-to-test label after the latest push.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant