fix(vertex_ai): forward function_call id on Vertex Gemini 3+ tool turns

[ljogeiger]

TLDR

Problem this solves:

• Vertex AI Gemini 3+ tool calls lose their id. LiteLLM strips it from both functionCall and functionResponse on every Vertex request, so Gemini cannot do strict tool-call matching and has to fall back to matching by function name
• The user-visible symptom is an empty assistant message after the final tool result. Gemini 3 returns a unique id on every functionCall and expects it echoed back on the matching functionResponse; when the id is missing, generateContent does not reject the request, it returns an empty response with finish_reason: STOP. A tool-calling agent looks like it silently gave up on the last turn, which is hard to attribute to a missing field several layers down
• The gate that does this was added in fix(vertex_ai): omit function_call id on Vertex Gemini 3.5+ tool turns #28324 on the premise that "Vertex AI rejects id on function_call/function_response parts". Google has since shipped the field to the v1 aiplatform endpoint, so the premise no longer holds and the workaround is now the bug
• Anyone running Gemini 3+ through vertex_ai/ with parallel tool calls is affected. Google AI Studio users are not, because fix(vertex_ai): omit function_call id on Vertex Gemini 3.5+ tool turns #28324 left that path alone

How it solves it:

• Gate the id on model version alone, which is what the code did before fix(vertex_ai): omit function_call id on Vertex Gemini 3.5+ tool turns #28324 and what Google AI Studio already does today
• Drop custom_llm_provider from _forward_gemini_function_call_id and from both Gemini tool-call converters, since the provider branch was its only reader
• Resolve the decision once in _gemini_convert_messages_with_history and pass it down as a bool, instead of re-deriving it independently in each converter

This brings LiteLLM back in line with what Vertex documents for Gemini 3. Google's migration guide covers it under function calling strict response matching: the id, name and response count on every functionResponse must match the functionCall parts that preceded it, and adding id to all function response parts is listed as a required migration step. The Interactions API errors outright on a mismatch. generateContent does not, which is why this surfaces as degraded output rather than a 400, and why it went unnoticed after #28324 landed

Echoing the id is also the only thing that makes parallel tool calls unambiguous. Name-based matching works as long as each turn calls a distinct function, and breaks as soon as a model issues two calls to the same function with different arguments, which is common for search and lookup tools

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy against real Vertex AI, no mocks. Config (vertex_fc_id_test_config.yaml):

model_list:
  - model_name: vertex-gemini-3
    litellm_params:
      model: vertex_ai/gemini-3.6-flash
      vertex_project: os.environ/VERTEX_PROJECT
      vertex_location: global

  - model_name: vertex-gemini-25
    litellm_params:
      model: vertex_ai/gemini-2.5-flash
      vertex_project: os.environ/VERTEX_PROJECT
      vertex_location: global

general_settings:
  master_key: sk-1234

python litellm/proxy/proxy_cli.py --config vertex_fc_id_test_config.yaml --detailed_debug 2>&1 | tee litellm.log

Turn 1, get a real tool call back from Vertex:

curl -s http://localhost:4000/v1/chat/completions \
  -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' \
  -d '{
    "model": "vertex-gemini-3",
    "messages": [{"role": "user", "content": "What is the weather in Boston, MA? Use the tool."}],
    "tools": [{"type": "function", "function": {
      "name": "get_current_weather",
      "description": "Get the current weather in a given location",
      "parameters": {"type": "object",
        "properties": {"location": {"type": "string", "description": "City and state"}},
        "required": ["location"]}}}]
  }' | jq '.choices[0].message.tool_calls[0] | {id, function}'

{
  "id": "ZX9IviSb__thought__AY89a18/oeibAN3n5I8cD...<truncated>",
  "function": {
    "arguments": "{\"location\": \"Boston, MA\"}",
    "name": "get_current_weather"
  }
}

ZX9IviSb is Vertex's own functionCall.id, returned by the v1 endpoint; the __thought__ suffix is LiteLLM's existing thought-signature packing. Vertex returning an id here is the response-side confirmation that v1 now supports the field.

Turn 2, feed that tool call and its result back:

curl -s http://localhost:4000/v1/chat/completions \
  -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' \
  -d @turn2.json | jq -r '.choices[0].message.content'

The current weather in Boston, MA is 42°F and cloudy.

Outbound request LiteLLM built for Google, from litellm.log. Before this PR, on vertex_ai/gemini-3.6-flash, no id on either part:

POST https://aiplatform.googleapis.com/v1/projects/your-project/locations/global/publishers/google/models/gemini-3.6-flash:generateContent

'contents': [
  {'role': 'user',  'parts': [{'text': 'What is the weather in Boston, MA? Use the tool.'}]},
  {'role': 'model', 'parts': [{'function_call': {'name': 'get_current_weather', 'args': {'location': 'Boston, MA'}},
                               'thoughtSignature': '<redacted>'}]},
  {'role': 'user',  'parts': [{'function_response': {'name': 'get_current_weather',
                                                     'response': {'temp_f': 42, 'conditions': 'cloudy'}}}]}
]

After this PR, same request, same model, id present on both parts and stripped of the thought-signature suffix:

POST https://aiplatform.googleapis.com/v1/projects/your-project/locations/global/publishers/google/models/gemini-3.6-flash:generateContent

'contents': [
  {'role': 'user',  'parts': [{'text': 'What is the weather in Boston, MA? Use the tool.'}]},
  {'role': 'model', 'parts': [{'function_call': {'name': 'get_current_weather', 'args': {'location': 'Boston, MA'},
                                                 'id': 'ZX9IviSb'},
                               'thoughtSignature': '<redacted>'}]},
  {'role': 'user',  'parts': [{'function_response': {'name': 'get_current_weather',
                                                     'response': {'temp_f': 42, 'conditions': 'cloudy'},
                                                     'id': 'ZX9IviSb'}}]}
]

Both returned 200 with the correct answer.

Older models are unaffected. Same turn-2 request against vertex-gemini-25 (vertex_ai/gemini-2.5-flash) still sends no id, which is required since those models reject the field:

POST https://aiplatform.googleapis.com/v1/projects/your-project/locations/global/publishers/google/models/gemini-2.5-flash:generateContent

'contents': [
  {'role': 'user',  'parts': [{'text': 'What is the weather in Boston, MA? Use the tool.'}]},
  {'role': 'model', 'parts': [{'function_call': {'name': 'get_current_weather', 'args': {'location': 'Boston, MA'}},
                               'thoughtSignature': '<redacted>'}]},
  {'role': 'user',  'parts': [{'function_response': {'name': 'get_current_weather',
                                                     'response': {'temp_f': 42, 'conditions': 'cloudy'}}}]}
]

The weather in Boston, MA is cloudy with a temperature of 42 degrees Fahrenheit.

Type

🐛 Bug Fix

Changes

VertexGeminiConfig._forward_gemini_function_call_id drops its custom_llm_provider parameter and returns _is_gemini_3_or_newer(model). That parameter existed only to hold the != "gemini" short-circuit, and nothing else read it.

convert_to_gemini_tool_call_invoke and convert_to_gemini_tool_call_result now take forward_function_call_id: bool in place of custom_llm_provider (and, for the result converter, model, whose only reader was the same gate). _gemini_convert_messages_with_history resolves the flag once and passes it to both. This removes the duplicated derivation and lets the result converter drop the function-local VertexGeminiConfig import that existed to dodge a circular import.

The id field comments on FunctionCall, FunctionResponse and HttpxFunctionCall in types/llms/vertex_ai.py came from #28324 and stated that Vertex rejects the field, which is no longer true; they now describe the version gate.

custom_llm_provider on _gemini_convert_messages_with_history itself is left in place. It is unused after this change, but it has three production call sites and a long tail of test callers, none of which have anything to do with this bug. Worth removing as a follow-up.

Out of scope and unchanged: countTokens strips functionResponse.id unconditionally in llms/gemini/count_tokens/handler.py, which shifts token counts slightly but never correctness; the realtime transformation has its own _include_function_response_id returning False and no seam to thread this through.

QA runbook

Point a vertex_ai/ deployment at any Gemini 3+ model and run a two-turn tool-calling exchange, as in the proof of fix above. With --detailed_debug, the outbound generateContent body should carry the same id on the function_call part and on the matching function_response part, and the call should return 200. Repeat against a Gemini 2.5 deployment and confirm neither part carries an id. Google AI Studio (gemini/) behaviour should be byte-identical to before.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Tests live in tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py. The two tests that encoded the old "Vertex must omit the id" contract are rewritten to the new one, and the Google AI Studio test is unchanged in meaning. Coverage: the gate is version-gated only; Gemini 3+ emits the id on both parts across vertex_ai, vertex_ai_beta and gemini; Gemini 2.5 emits the key on neither; a <id>__thought__<sig> tool-call id forwards stripped; and an unpairable tool result raises rather than shipping a half-formed payload.

Mutation-checked against five mutants, all killed: the gate forced to False, to True, and back to the old provider branch, plus each converter call site independently forced to False. That last pair is what makes a one-sided implementation impossible to pass, since functionCall.id without a matching functionResponse.id is exactly the shape that would break strict matching on Google's side.

[greptile-apps]

Greptile Summary

This PR enables strict tool-call ID matching for Gemini 3+ across Vertex AI and Google AI Studio.

• Computes the model-version decision once and applies it to both function calls and responses
• Preserves ID omission for older Gemini models and strips packed thought-signature suffixes
• Updates Vertex wire types and regression tests for both provider paths

Confidence Score: 5/5

The PR appears safe to merge, with call and response IDs forwarded consistently only for recognized Gemini 3+ models.

The changed conversion path applies one model-version decision to both sides of each tool turn, preserves older-model behavior, and includes focused regression coverage for the affected provider variants and ID transformations.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/prompt_templates/factory.py	Accepts the centralized forwarding decision and consistently strips thought-signature suffixes from call and response IDs.
litellm/llms/vertex_ai/gemini/transformation.py	Computes ID-forwarding capability once per conversion and passes it to both tool-turn converters.
litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py	Removes the provider-specific exclusion while retaining the existing Gemini 3+ version gate.
litellm/types/llms/vertex_ai.py	Updates type comments to reflect the revised version-gated provider contract.
tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py	Adds regression coverage for Vertex and AI Studio, older models, packed IDs, and unmatched tool responses.

_{Reviews (1): Last reviewed commit: "fix(vertex_ai): forward function_call id..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing ljogeiger:litellm_vertex_function_call_id (acd414f) with litellm_internal_staging (b9b27c2)}