fix(anthropic): add non-whitespace trailing dummy#1891
fix(anthropic): add non-whitespace trailing dummy#1891rosetta-livekit-bot[bot] wants to merge 1 commit into
Conversation
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| // Claude 4.6+ does not support prefilling (trailing assistant messages). | ||
| if (messages.length > 0 && messages[messages.length - 1]!.role === 'assistant') { | ||
| messages.push({ role: 'user', content: [{ type: 'text', text: '.' }] }); | ||
| } |
There was a problem hiding this comment.
🟡 New trailing-message fix breaks existing unit test for message merging
A dummy user message is unconditionally appended when the last message is from the assistant (messages.push(...) at plugins/anthropic/src/llm.ts:179), but the existing test that ends with an assistant reply was not updated, so it will fail its length assertion.
Impact: The test suite will fail, blocking CI and preventing the PR from merging.
Test failure trace for "merges consecutive same-role messages"
The test at plugins/anthropic/src/llm.test.ts:83-100 builds a chat context with two user messages and one assistant reply (line 89). After merging consecutive user messages, _buildAnthropicContext produces a 2-element array. However, the new code at plugins/anthropic/src/llm.ts:178-180 detects that the last message is from the assistant and appends a dummy { role: 'user', content: [{ type: 'text', text: '.' }] } entry, growing the array to length 3. The assertion at plugins/anthropic/src/llm.test.ts:95 — expect(messages).toHaveLength(2) — therefore fails.
The test file was not modified by this PR, so this is an oversight. The test needs updating to expect the new trailing user message.
Prompt for agents
The new code at plugins/anthropic/src/llm.ts:177-180 appends a dummy user message whenever the conversation ends with an assistant turn. This is the intended behavior for Claude 4.6+ compatibility, but the existing test 'merges consecutive same-role messages' at plugins/anthropic/src/llm.test.ts:83-100 was not updated. That test builds a context ending with an assistant message (line 89) and asserts messages.toHaveLength(2) at line 95, which will now be 3.
The fix is to update the test at plugins/anthropic/src/llm.test.ts:
1. Change the length assertion at line 95 from toHaveLength(2) to toHaveLength(3)
2. Add an assertion for the new trailing dummy user message (e.g. expect(messages[2].role).toBe('user'))
3. Consider also adding a dedicated test case for the new trailing-message behavior, similar to the existing test at line 102 for the leading-message injection.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
|
||
| // Claude 4.6+ does not support prefilling (trailing assistant messages). | ||
| if (messages.length > 0 && messages[messages.length - 1]!.role === 'assistant') { | ||
| messages.push({ role: 'user', content: [{ type: 'text', text: '.' }] }); |
There was a problem hiding this comment.
🚩 Dummy message content '.' differs from existing placeholder style and may influence model output
The new trailing dummy message uses { type: 'text', text: '.' } at plugins/anthropic/src/llm.ts:179, while the existing leading dummy message uses the string '(empty)' at plugins/anthropic/src/llm.ts:174. Beyond the stylistic inconsistency, a single period could be interpreted by Claude as actual user input (e.g., prompting a "How can I help you?" response), whereas '(empty)' more clearly signals a placeholder. This is a design choice rather than a correctness bug, but it may affect response quality in conversations where the assistant's last message needs a follow-up.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
.instead of whitespace to satisfy provider validationTesting
pnpm --filter @livekit/agents build && pnpm --filter @livekit/agents-plugin-anthropic buildStacked on #1588 because the Anthropic plugin is not on
mainyet.Ported from livekit/agents#6216
Original PR description
Summary
One-line fix in
livekit-agents/livekit/agents/llm/_provider_format/anthropic.py: the trailing dummy user message injected when the last message is from the assistant now uses"."instead of" "(a single non-whitespace character instead of whitespace).Why
Anthropic's API rejects messages whose text content is empty or whitespace-only. When
inject_trailing_user_message=Trueappends a placeholder user turn to satisfy Claude 4.6+'s no-prefilling requirement, the current" "(single space) trips Anthropic's server-side validation and the request fails before the model is ever reached. Replacing it with"."keeps the placeholder semantically inert while clearing the validation gate.Relationship to existing issues
This addresses #5254, which was closed as stale. The bug still reproduces verbatim on current
main(1.6.3) — any conversation that ends with an assistant turn and is then re-submitted to the Anthropic provider hits the same validation error described in the original report.Requesting reopen of #5254 and merge of this fix.
Change
That is the entire diff.