Skip to content

chore(compat-matrix): refresh for v1.99.1 + claude-code 2.1.228 - #1218

Open
mateo-berri wants to merge 1 commit into
mainfrom
compat-matrix/v1.99.1-2.1.228-2026-09-05
Open

chore(compat-matrix): refresh for v1.99.1 + claude-code 2.1.228#1218
mateo-berri wants to merge 1 commit into
mainfrom
compat-matrix/v1.99.1-2.1.228-2026-09-05

Conversation

@mateo-berri

Copy link
Copy Markdown
Contributor

Automated daily refresh of the Claude Code compatibility matrix.

Warning

Auto-merge disabled: one or more cells regressed green→red versus the
currently-published matrix. Review the diff before merging.

detected 14 green->red regression(s):
  - Basic messaging (non-streaming) [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Basic messaging (streaming) [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Long context (1M) [bedrock_converse]: pass -> fail ([claude-sonnet-4-6-bedrock-converse] claude CLI failed: exit=1; api_status=500; text=API Error: 500 litellm.APIConnectionError: 'NoneType' object has no attribu)
  - PDF document input [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Prompt caching (1h TTL) [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Prompt caching (5m TTL) [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Structured outputs [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Thinking [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Extended thinking + tool use [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Tool search (MCP discovery) [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] tool_search probe failed: status 500: {"error":{"message":"litellm.APIConnectionError: 'NoneType' object has no attribute 'a)
  - Tool use [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Tool use (streaming / fine-grained) [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Vision [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
  - Web search (server tool) [bedrock_converse]: pass -> fail ([claude-haiku-4-5-bedrock-converse] claude CLI timed out after 120.0s)
Field Value
litellm_version v1.99.1
claude_code_version 2.1.228
generated_at 2026-09-05T06:41:15Z

Per-feature results

  • Basic messaging (non-streaming): anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Basic messaging (streaming): anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Tool use: anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Prompt caching (5m TTL): anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Vision: anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Thinking: anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Tool use (streaming / fine-grained): anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Extended thinking + tool use: anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • PDF document input: anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Prompt caching (1h TTL): anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Web search (server tool): anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Structured outputs: anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • count_tokens endpoint: anthropic=pass, bedrock_invoke=pass, bedrock_converse=pass, vertex_ai=pass, azure=pass
  • Tool search (MCP discovery): anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass
  • Long context (1M): anthropic=pass, bedrock_invoke=pass, bedrock_converse=fail, vertex_ai=pass, azure=pass

Generated by tests/claude_code/cron_vm/run_daily.sh. Close without merging if the diff looks wrong; the next cron run will reopen with fresh results.

litellm_version: v1.99.1
claude_code_version: 2.1.228
generated_at: 2026-09-05T06:41:15Z
@vercel

vercel Bot commented Sep 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
litellm Ready Ready Preview Sep 5, 2026 6:46am UTC

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant