Support AGENTS.md and CLAUDE.md evaluation targets - #11
Open
aaronfc wants to merge 2 commits into
Open
Conversation
…ction-targets # Conflicts: # README.md # src/unvibe/cli.py # tests/test_cli.py
Owner
Author
|
Needs proper dogfooding before merging. Still not sure if this is a good idea. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #10
Why
unvibe could evaluate only a directory containing SKILL.md, even though its plan and assertion engine operates on plain instruction text. Repositories that keep AGENTS.md or CLAUDE.md could not reuse the same lightweight evaluation workflow, and silently approximating native loading semantics would overstate what the runner measures.
Current main also standardized the suite filename as EVALUATIONS.yaml and added selectable Claude, Codex, and OpenCode harnesses. This branch needs to compose with those changes instead of restoring the old singular filename or Claude-only execution.
What
Testing Steps
Prerequisite: install and authenticate a supported harness. The following uses the suggested Codex models from the README.
Expected: the command reports
created .../EVALUATIONS.yaml; the directory contains the originalAGENTS.mdand newEVALUATIONS.yaml.Cheap ambiguity edge case:
Expected: exit 1, list
AGENTS.mdandCLAUDE.md, and request an explicit file target.Cleanup:
rm -r "$proof_dir"Proofs
Direct deterministic CLI proof with a stub Codex harness:
Automated checks:
uv run pytest -q— 88 passed in 0.35s.tests/smoke.sh—SMOKE OK; packaged flows passed for Claude, Codex, and OpenCode.