Skip to content

Improve review-anvil finding explanations and impact judgment - #13

Merged
mrshu merged 3 commits into
mainfrom
fix/review-anvil-finding-content
Oct 7, 2026
Merged

mrshu merged 3 commits into
mainfrom
fix/review-anvil-finding-content

Conversation

@mrshu

@mrshu mrshu commented Oct 6, 2026

Copy link
Copy Markdown
Owner

Summary

Implements the two content improvements, not a Codex-specific launcher change:

  • Select the necessary condition-to-consequence explanation (or why existing evidence is insufficient) before freezing diagnostic facts and source fallback. Keep incidental proof details out of diagnostic locks and keep author work unchanged.
  • Ground proof expectations in supported requirements and the applicable PR change. Judge severity by concrete supported consequences, not merely the correctness-bug category; preserve security/data-integrity concerns and supported APIs, workers and tools.

Align normal reviewer context/task, engine scope selection, reproduction and the existing scope auditor: PR/diff reviews require applicable attribution; whole-file/path reviews can report proven existing defects within their declared scope. Newly exposed unchanged helpers remain in scope.

No launcher/backend changes, schema additions, new production audit waves, weakened semantic locks or approval-threshold changes. Updates README, retains sanitized evaluation evidence, and removes the source-text/wording assertion suite while retaining behavioral suites.

Verification

  • Passed: 29 clarity-validator tests, identifier contracts, PR-helper e2e, reviewer-wrapper tests, reproduction-policy examples, skill frontmatter and plugin manifests.
  • Model-stage smoke: original facts/work retained with the close-save explanation; two independent semantic auditors passed all three surfaces. Contract verdicts distinguished static obligation refutation from unknown requirements and unexecuted runtime claims.
  • Normal reviewer whole-file smoke: existing defects remain actionable without PR attribution; the corrected reviewer template returned no findings. The scope auditor upheld the retained existing defect.

Limits and observed failures

  • The 40-call baseline/candidate pilot was mixed/tied. No broad quality or human-comprehension improvement is claimed; the whole-file baseline already detected the scope conflict.
  • One renderer smoke emitted an unsupported section label and was rejected by the unchanged structural validator. A separate source-fallback smoke retained the explanation/work byte-for-byte and forced COMMENT. No invalid bundle was posted.
  • The proof-runner suite passed initially, but its final publication run failed: timeout exit: got '3', want '124'. Retained captures show capability checking exited 143 with empty capability output, returning capability-rejected before the expected run-phase timeout; source/proof unchanged and runtime removed. Runner and suite code are unchanged; no unrelated runtime fix or retry was attempted.
  • No model-authored target probes were executed; no live Hermes deployment or target review posting was performed.

Evidence: evaluations/review-anvil-finding-content.json. Private code, feedback quotations and author identities are excluded.

@mrshu
mrshu merged commit 459a115 into main Oct 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant