Skip to content

Fix malformed <|im_start> token in Phi4Reasoning prompt style#2267

Open
JSap0914 wants to merge 1 commit into
Lightning-AI:mainfrom
JSap0914:fix/phi4-reasoning-im-start-token
Open

Fix malformed <|im_start> token in Phi4Reasoning prompt style#2267
JSap0914 wants to merge 1 commit into
Lightning-AI:mainfrom
JSap0914:fix/phi4-reasoning-im-start-token

Conversation

@JSap0914

Copy link
Copy Markdown

Bug

Phi4Reasoning.apply builds the first ChatML turn with a malformed special token: it emits <|im_start>system (missing the closing |) instead of the well-formed <|im_start|> token that Phi4 and every other ChatML-based prompt style use.

return f"<|im_start>system<|im_sep|>{sys_prompt}<|im_end|><|im_start|>user<|im_sep|>{prompt}<|im_end|><|im_start|>assistant<|im_sep|>"
#        ^^^^^^^^^^^ missing the closing pipe; the other two occurrences are correct

Because <|im_start> is not the registered special token, the Phi-4-reasoning tokenizer does not collapse the system turn into the single <|im_start|> token, corrupting the chat template for that model.

Fix

Use the well-formed <|im_start|> token for the system turn, matching Phi4 and the two other occurrences in the same template.

Verification

Added a regression test in tests/test_prompts.py:

$ pytest tests/test_prompts.py::test_phi4_reasoning_uses_well_formed_im_start_token -v
test_phi4_reasoning_uses_well_formed_im_start_token PASSED

The test asserts the rendered template contains no malformed <|im_start> marker, starts with <|im_start|>system<|im_sep|>, and uses the well-formed token exactly three times.

The first ChatML turn in Phi4Reasoning.apply emitted '<|im_start>system'
(missing the closing pipe) instead of the well-formed '<|im_start|>' special
token used by Phi4 and every other ChatML-based template. With the malformed
marker the Phi-4-reasoning tokenizer does not recognize the system turn as a
single special token, corrupting the chat template.

Add a regression test asserting the template uses the well-formed token.
Copilot AI review requested due to automatic review settings June 16, 2026 08:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants