docs(realtime): describe the lazy Gemini Live setup built from the first session.update

docs/providers/google_ai_studio/realtime.md

@@ -192,8 +192,9 @@ model_list:
       api_key: os.environ/GEMINI_API_KEY
 
 litellm_settings:
-  # Required for tool calling with Gemini Live:
-  # defer setup until client sends session.update (with tools)
+  # Optional. session.update events sent before any content already build the Gemini
+  # setup (with tools); this buffers audio and text sent before session.update so that
+  # update still applies
   gemini_live_defer_setup: true
 ```
 
@@ -205,13 +206,13 @@ python test_realtime_tool_calling.py
 ## Limitations 
 
 - Does not support audio transcription.
-- Session config updates after the first `session.update` are ignored (Gemini setup is one-time per connection).
+- Every `session.update` sent before the first content frame is merged into the Gemini setup; a `session.update` sent after content is ignored (Gemini setup is one-time per connection).
 
 ## Precaution
 
 - Tool calling will not work unless you send `session.update` first with your `tools`.
 - Send it as the first config message for that websocket session.
-- `gemini_live_defer_setup` defaults to `false` for backward compatibility.
+- `gemini_live_defer_setup` defaults to `false`. It is only needed when the client sends audio or text before its `session.update`.
 
 ## Supported OpenAI Realtime Events
 

docs/providers/vertex_realtime.md

@@ -163,8 +163,10 @@ Key settings for audio:
 - Server VAD is enabled by default with 800 ms silence threshold
 
 ```python
-# session.update with server VAD — the proxy ignores this for Vertex AI
-# because VAD is already configured in the initial setup message.
+# Send session.update before any content: LiteLLM merges every session.update it
+# receives before the first conversation item or audio frame into the one Vertex AI
+# setup, so they set the modalities, VAD, and tools for the whole session. A
+# session.update sent after content is ignored.
 await ws.send(json.dumps({
     "type": "session.update",
     "session": {
@@ -320,7 +322,7 @@ python test_realtime_tool_calling.py
 |---|---|
 | `input_audio_buffer.append` | Forwarded as `realtime_input.audio` |
 | `conversation.item.create` | Forwarded as `realtime_input.text` |
-| `session.update` | Silently ignored — Vertex AI does not support mid-session reconfiguration |
+| `session.update` | Every one sent before the first content frame is merged into the Vertex AI `setup` and acknowledged with `session.updated`. Ones sent after content are ignored: Vertex AI accepts one setup per connection |
 | `response.create` | Silently ignored — Vertex AI responds automatically after each turn |
 
 **Vertex AI → Proxy (→ Client)**
@@ -336,11 +338,13 @@ python test_realtime_tool_calling.py
 
 ## Limitations
 
-- `session.update` is not forwarded (Vertex AI only accepts one setup message per connection).
+- Session configuration is fixed once content flows. LiteLLM sends the Vertex AI setup with the client's first content frame (a conversation item, audio, or a tool result): every `session.update` received before it is merged into that setup, a session that sends content first gets the model's default (audio) setup, and a `session.update` sent after content is ignored because Vertex AI accepts one setup per connection.
+- A client that streams audio before its `session.update` gets the default setup. Set `gemini_live_defer_setup: true` to buffer that audio until the `session.update` arrives.
+- `output_modalities: ["text"]` on an audio-only model (for example `gemini-live-2.5-flash-native-audio`) is downgraded to audio, and the proxy logs a warning saying so.
 - Audio transcription requires `outputAudioTranscription: {}` to be set in the initial setup (done automatically by LiteLLM).
 
 ## Precaution
 
 - Tool calling depends on `session.update` with `tools`.
 - If you skip `session.update`, tool calls will not be triggered.
-- `gemini_live_defer_setup` defaults to `false` for backward compatibility.
+- `gemini_live_defer_setup` defaults to `false`. It is only needed when the client sends audio or text before its `session.update`.

docs/proxy/config_settings.md

@@ -1222,7 +1222,7 @@ router_settings:
 | LITELLM_TPM_TOKEN_RESERVATION_ENABLED | Default `true`. Set to `false` to disable pre-request TPM reservation in the v3 rate limiter and apply actual usage after each real-time request completes. This removes one Redis operation per request, but concurrent requests may temporarily exceed the TPM limit. This setting does not apply to `POST /v1/batches`, which uses a [separate input-file limiter](../batches#how-rate-limiting-for-batches-api-works). See [Estimated output tokens](./users#estimated-output-tokens-requests-without-max_tokens). |
 | LITELLM_USE_CHAT_COMPLETIONS_URL_FOR_ANTHROPIC_MESSAGES | When set to "true", routes OpenAI /v1/messages requests through chat/completions instead of the Responses API for Anthropic models. Can also be set via `litellm_settings.use_chat_completions_url_for_anthropic_messages`
 | LITELLM_ROUTE_ALL_CHAT_OPENAI_TO_RESPONSES | When set to "true", routes all OpenAI /chat/completions requests through the Responses API bridge. Recommended for OpenAI models. Can also be set via `litellm_settings.route_all_chat_openai_to_responses`
-| LITELLM_GEMINI_LIVE_DEFER_SETUP | When set to "true", defers Gemini/Vertex Live setup until the client sends `session.update` (required for runtime tool injection). Default is "false" for backwards compatibility, which auto-sends setup on connect. Can also be set via `litellm.gemini_live_defer_setup`
+| LITELLM_GEMINI_LIVE_DEFER_SETUP | When set to "true", buffers audio and text the client sends before its `session.update` so that update still builds the Gemini/Vertex Live setup. Default is "false": the setup is sent with the client's first content frame, built from the `session.update` events received before it and from the model defaults when there were none. Can also be set via `litellm.gemini_live_defer_setup`
 | LITELLM_USE_LEGACY_INTERACTIONS_SCHEMA | When set to "true", uses the legacy Google Interactions API schema (`outputs` array, `2026-05-07` revision) instead of the new schema (`steps` array, `2026-05-20` revision). The legacy schema will be sunset on June 8, 2026. Can also be set via `litellm_settings.use_legacy_interactions_schema`
 | LITELLM_USER_AGENT | Custom user agent string for LiteLLM API requests. Used for partner telemetry attribution
 | LITELLM_WORKER_STARTUP_HOOKS | Comma-separated list of `module.path:function_name` callables to run in each worker process during startup. Runs early in the worker lifecycle (before config/DB loading). Useful for re-initializing per-process state like [gflags](https://github.com/google/python-gflags). See [Worker Startup Hooks](/docs/proxy/worker_startup_hooks) for details