docs(realtime): describe the lazy Gemini Live setup built from the first session.update

docs/providers/google_ai_studio/realtime.md

@@ -192,8 +192,8 @@ model_list:
       api_key: os.environ/GEMINI_API_KEY
 
 litellm_settings:
-  # Required for tool calling with Gemini Live:
-  # defer setup until client sends session.update (with tools)
+  # Optional. A session.update sent first already builds the Gemini setup (with tools);
+  # this buffers audio and text sent before session.update so that update still applies
   gemini_live_defer_setup: true
 ```
 
@@ -211,7 +211,7 @@ python test_realtime_tool_calling.py
 
 - Tool calling will not work unless you send `session.update` first with your `tools`.
 - Send it as the first config message for that websocket session.
-- `gemini_live_defer_setup` defaults to `false` for backward compatibility.
+- `gemini_live_defer_setup` defaults to `false`. It is only needed when the client sends audio or text before its `session.update`.
 
 ## Supported OpenAI Realtime Events
 

docs/providers/vertex_realtime.md

@@ -163,8 +163,9 @@ Key settings for audio:
 - Server VAD is enabled by default with 800 ms silence threshold
 
 ```python
-# session.update with server VAD — the proxy ignores this for Vertex AI
-# because VAD is already configured in the initial setup message.
+# Send session.update first: LiteLLM builds the Vertex AI setup from the first client
+# message, so a session.update sent first sets the modalities, VAD, and tools for the
+# whole session. Later session.update events are ignored.
 await ws.send(json.dumps({
     "type": "session.update",
     "session": {
@@ -320,7 +321,7 @@ python test_realtime_tool_calling.py
 |---|---|
 | `input_audio_buffer.append` | Forwarded as `realtime_input.audio` |
 | `conversation.item.create` | Forwarded as `realtime_input.text` |
-| `session.update` | Silently ignored — Vertex AI does not support mid-session reconfiguration |
+| `session.update` | Becomes the Vertex AI `setup` when it is the first client message. Later ones are ignored: Vertex AI accepts one setup per connection |
 | `response.create` | Silently ignored — Vertex AI responds automatically after each turn |
 
 **Vertex AI → Proxy (→ Client)**
@@ -336,11 +337,13 @@ python test_realtime_tool_calling.py
 
 ## Limitations
 
-- `session.update` is not forwarded (Vertex AI only accepts one setup message per connection).
+- Only the first `session.update` takes effect. LiteLLM sends the Vertex AI setup on the client's first message: a `session.update` sent first becomes that setup, anything else sent first gets the model's default (audio) setup, and later `session.update` events are ignored because Vertex AI accepts one setup per connection.
+- A client that streams audio before its `session.update` gets the default setup. Set `gemini_live_defer_setup: true` to buffer that audio until the `session.update` arrives.
+- `output_modalities: ["text"]` on an audio-only model (for example `gemini-live-2.5-flash-native-audio`) is downgraded to audio, and the proxy logs a warning saying so.
 - Audio transcription requires `outputAudioTranscription: {}` to be set in the initial setup (done automatically by LiteLLM).
 
 ## Precaution
 
 - Tool calling depends on `session.update` with `tools`.
 - If you skip `session.update`, tool calls will not be triggered.
-- `gemini_live_defer_setup` defaults to `false` for backward compatibility.
+- `gemini_live_defer_setup` defaults to `false`. It is only needed when the client sends audio or text before its `session.update`.

docs/proxy/config_settings.md

@@ -1222,7 +1222,7 @@ router_settings:
 | LITELLM_TPM_TOKEN_RESERVATION_ENABLED | Default `true`. Set to `false` to disable pre-request TPM reservation in the v3 rate limiter and apply actual usage after each real-time request completes. This removes one Redis operation per request, but concurrent requests may temporarily exceed the TPM limit. This setting does not apply to `POST /v1/batches`, which uses a [separate input-file limiter](../batches#how-rate-limiting-for-batches-api-works). See [Estimated output tokens](./users#estimated-output-tokens-requests-without-max_tokens). |
 | LITELLM_USE_CHAT_COMPLETIONS_URL_FOR_ANTHROPIC_MESSAGES | When set to "true", routes OpenAI /v1/messages requests through chat/completions instead of the Responses API for Anthropic models. Can also be set via `litellm_settings.use_chat_completions_url_for_anthropic_messages`
 | LITELLM_ROUTE_ALL_CHAT_OPENAI_TO_RESPONSES | When set to "true", routes all OpenAI /chat/completions requests through the Responses API bridge. Recommended for OpenAI models. Can also be set via `litellm_settings.route_all_chat_openai_to_responses`
-| LITELLM_GEMINI_LIVE_DEFER_SETUP | When set to "true", defers Gemini/Vertex Live setup until the client sends `session.update` (required for runtime tool injection). Default is "false" for backwards compatibility, which auto-sends setup on connect. Can also be set via `litellm.gemini_live_defer_setup`
+| LITELLM_GEMINI_LIVE_DEFER_SETUP | When set to "true", buffers audio and text the client sends before its `session.update` so that update still builds the Gemini/Vertex Live setup. Default is "false": the setup is sent on the client's first message, built from it when that message is a `session.update` and from the model defaults otherwise. Can also be set via `litellm.gemini_live_defer_setup`
 | LITELLM_USE_LEGACY_INTERACTIONS_SCHEMA | When set to "true", uses the legacy Google Interactions API schema (`outputs` array, `2026-05-07` revision) instead of the new schema (`steps` array, `2026-05-20` revision). The legacy schema will be sunset on June 8, 2026. Can also be set via `litellm_settings.use_legacy_interactions_schema`
 | LITELLM_USER_AGENT | Custom user agent string for LiteLLM API requests. Used for partner telemetry attribution
 | LITELLM_WORKER_STARTUP_HOOKS | Comma-separated list of `module.path:function_name` callables to run in each worker process during startup. Runs early in the worker lifecycle (before config/DB loading). Useful for re-initializing per-process state like [gflags](https://github.com/google/python-gflags). See [Worker Startup Hooks](/docs/proxy/worker_startup_hooks) for details