test(eval): measure review findings, not just review outcomes - #1003

Open

guyoron1 wants to merge 9 commits into

fullsend-ai:mainfrom

guyoron1:eval/review-precision-cases

Commits on Aug 26, 2026

feat(eval): capture inline PR review comments in fixture state
guyoron1
committed
340aa03
feat(eval): judge review findings, not just review outcomes
guyoron1
committed
f63ceea
test(eval): pin the review findings judges against synthetic payloads
guyoron1
committed
3c3654d
test(eval): add four review precision cases
guyoron1
committed
3df1ac2
fix(eval): make the findings judges falsifiable
guyoron1
committed
d01cd92
test(eval): add an anti-bait case the prompts do not name
guyoron1
committed
7b003e0
fix(eval): fail closed when findings cannot be read
guyoron1
committed
d177dbb
fix(eval): read findings that a 422 pushed into the review body
guyoron1
committed
eb45eae
fix(eval): fail closed on dropped findings and malformed ground truth
guyoron1
committed
072c700