Skip to content

Add cheat detection for generated-code evaluation#1

Open
helenxie-bit wants to merge 1 commit into
uccl-project:mainfrom
helenxie-bit:cheat-detect
Open

Add cheat detection for generated-code evaluation#1
helenxie-bit wants to merge 1 commit into
uccl-project:mainfrom
helenxie-bit:cheat-detect

Conversation

@helenxie-bit

Copy link
Copy Markdown
Contributor

Summary

This PR adds cheat detection code to flag generated solutions that appear to modify template code or leave suspicious TODO markers, and integrates that detection into the evaluation pipeline before compile/run.

Main Changes

  • run_eval/cheat_detect.py

    • Implements function todo_remaining to flag generated files with TODO marker counts greater than or equal to those in the empty file.
    • Implements function non_todo_modified to parse comparable functions with tree-sitter and flag generated code that changes non-TODO function bodies, such as benchmark, validation, or test harness logic.
    • Implements function run_all_checks to run the above two checks and combine their results into a CheatResult.
    • Provides debug helpers to list parsed functions and print modified-function diffs.
  • scripts/generate_eval_one.py

    • Inserts cheat detection immediately after each LLM response and before compile/run.
    • On detection, writes summary.json with status: "cheat", cheat_detected: true, and cheat_reasons.
    • Stops the round immediately without retrying and returns passed=False with a CHEAT:-prefixed reason.
  • prompt_llm/prompt.py

    • Adds anti-cheat rules to the initial completion prompts for C++ and Python as well as the retry/fix prompts.
  • requirements.txt

    • Adds tree-sitter, tree-sitter-cpp, and tree-sitter-python for source parsing.

Notes

  • The detector currently only compares functions for modifications. It ignores other declarations, such as constants, structs, and other non-function declarations.
  • When comparing functions, it only compares function bodies. It does not compare function signatures, parameter declarations, or parameter initialization/default values.
  • If the generated code has invalid grammar, parsing may fail or produce unreliable detection results. For example, some generated files started with "```cpp", which is not valid C++ grammar, causing generated-file parsing to fail and all functions to be treated as modified.
  • Some generated code may stop midway due to context window limits, leaving a class or function definition unfinished. In those cases, tree-sitter may not parse the class/function correctly.

Signed-off-by: helenxie-bit <helenxiehz@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant