Fix malformed <|im_start> token in Phi4Reasoning prompt style#2267
Open
JSap0914 wants to merge 1 commit into
Open
Fix malformed <|im_start> token in Phi4Reasoning prompt style#2267JSap0914 wants to merge 1 commit into
JSap0914 wants to merge 1 commit into
Conversation
The first ChatML turn in Phi4Reasoning.apply emitted '<|im_start>system' (missing the closing pipe) instead of the well-formed '<|im_start|>' special token used by Phi4 and every other ChatML-based template. With the malformed marker the Phi-4-reasoning tokenizer does not recognize the system turn as a single special token, corrupting the chat template. Add a regression test asserting the template uses the well-formed token.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
Phi4Reasoning.applybuilds the first ChatML turn with a malformed special token: it emits<|im_start>system(missing the closing|) instead of the well-formed<|im_start|>token thatPhi4and every other ChatML-based prompt style use.Because
<|im_start>is not the registered special token, the Phi-4-reasoning tokenizer does not collapse the system turn into the single<|im_start|>token, corrupting the chat template for that model.Fix
Use the well-formed
<|im_start|>token for the system turn, matchingPhi4and the two other occurrences in the same template.Verification
Added a regression test in
tests/test_prompts.py:The test asserts the rendered template contains no malformed
<|im_start>marker, starts with<|im_start|>system<|im_sep|>, and uses the well-formed token exactly three times.