Repository navigation
feat: Store and Replay the Provider Output Items of Responses Tool Rounds - #17149
Open
Shabonasar wants to merge 109 commits into
Open
Shabonasar wants to merge 109 commits into
Shabonasar wants to merge 109 commits into
Conversation
The per-request web search context (current date/time with milliseconds) was joined before additional_instructions, which carry mostly static text such as the artifacts prompt. Provider-side prefix caches stopped at the timestamp, so the static text after it was recomputed on every request. Move the dynamic context to the end via buildAdditionalInstructions and cover the ordering with unit tests.
…che-order # Conflicts: # packages/api/src/agents/initialize.ts
…e date changes in history The web search runtime context and temporal special variables were anchored to the turn start with millisecond precision. The system message precedes the whole history, so every turn invalidated the cached history and new conversations could not reuse the system prompt (LibreChat-AI#16877). - Web search runtime context states the conversation start in the user's timezone with minute precision; temporal variables resolve against the same persisted anchor. - A user message sent on a later local date than the model last saw gets a <context_update> block in its prompt copy only, derived from stored createdAt values so it renders identically on later turns; compaction boundaries follow the agents SDK formatter. - Optional endpoints.agents.promptMessageTimestamps adds the send time to every user message; the turn's user message is stored with createdAt = turnStartedAt on every save path. - Provider payload stability tests through createRun for OpenAI, an OpenAI-compatible endpoint, Anthropic, OpenRouter, Bedrock and Google.
… routes The browser resolved promptPrefix with its own clock on every send, so turn 1 of a spec-backed conversation carried the raw template while turn 2 carried resolved text in a different system-prompt position. Only Assistants routes resolve on the client now; the server resolves against the conversation start.
… of a conversation
Adds `endpoints.agents.promptCache.stableContext` (default off). Off, the prompt is
built as before. On:
- prompt time (`{{current_datetime}}` and friends, the web search runtime context) is
anchored to the conversation start, and a date change is announced by a
`<context_update>` block at the start of the first user message sent after it;
- the conversation-stable part of `additional_instructions` is merged into the system
instructions for Anthropic and OpenRouter with prompt caching, while memory, RAG and
one-turn queued file context stay in `additional_instructions`, where the SDK moves
them next to the newest user message;
- dynamic tool context goes after the additional instructions;
- the user message is inserted with the turn start as `createdAt`, so later turns render
the same stamp.
`promptMessageTimestamps` (default off) additionally prefixes each user message with the
time it was sent.
The history notes, the instruction split and the lever live in `packages/api`; the
controller only calls them. `applyContextToAgent` accepts the run context as parts and
records which are volatile, so no text is split by matching.
…assified parts Producers (project, memory guard, artifacts, skills catalog, RAG, memory, agent context files, always-apply skills of isolated subagents) contribute keyed parts through contributeContext instead of concatenating strings into additional_instructions. One registry fixes the canonical order and classifies every part, tool context map entries included, as static, conversation or turn. A runtime key the registry does not know counts as turn, so a new producer cannot break the stable prefix by default. assembleSystemPrompt builds the strings handed to the SDK. Once parts exist they are the only source of the tail; a direct write to additional_instructions is ignored with a warning. Without promptCache.stableContext it reproduces the previous request; with it, createRun moves only the turn parts with the prompt-cache tail and merges the rest into the instructions. The assembled text is byte-identical to string concatenation for both values of the lever, guarded by a snapshot recorded on the previous implementation. Isolated subagents with an always-apply skill place the skill ahead of the tool runtime context when stableContext is on.
…m prompt per segment Behind `endpoints.agents.promptCache.stableContext`; off, nothing here runs and the prompt is built as before. A single send-time step (turnContext) decides what the primary agent's model sees. Context sources (project, artifacts mode, memory tool guard, skills catalog, tool runtime context, memory, date, send time, retrieved context, queued upload files) are sampled once when the user sends a message; only the net difference from what the model already knows along the parent chain is stored on that user message (message.contextUpdates) and rendered as one <context_update> block at the start of its prompt copy. Toggles flipped back and forth before sending leave no trace, and stored values render the same bytes on every later turn, whatever the timezone. A regeneration sends no message: it renders what the message stored and writes nothing. The system prompt is rendered from the segment's anchor instead of the live parts, so it stays byte-identical until the next compaction. The anchor of the first segment is stored on the conversation, the anchor of a segment opened by a compaction on the message carrying the summary (contextAnchor); a conversation without one is anchored to its first turn after the upgrade. A re-anchor keeps the previous order, drops what is off and appends what is new, and dates itself, so the first message after the boundary announces nothing. Memory is a state source: an update lists the entries added, changed and removed (by key where the memory has keys) instead of the whole memory, and no longer rewrites the system prompt; memory switched off is withheld from the whole request. Retrieved context is stored with the user message it was retrieved for, like the file context bound to the message that carried the files; isolated subagents still take it in their system prompt. Queued upload files describe their own message only. A resumed turn (tool approval or an answered question) renders its system prompt from the paused turn's anchor, not from the live state. The planning, the anchor storage and the token accounting live in packages/api; the controller calls them. The conversation anchor is read with the conversation the ingress already loads, and a resume reads the messages once with a projection. contextUpdates and contextAnchor are server-written only, select: false, and stripped from client input, so they stay out of reads, shares and forks.
…d the SDK dispatcher Behind `endpoints.agents.promptCache.stableContext`, which also stays off, with one warning, while the installed @librechat/agents has no tool dispatcher. Providers serialize `tools` ahead of the system prompt, so switching an MCP server, an action or an agent tool mid-conversation reset the prompt cache from position 0. - Tool groups (`tool:mcp:<server>`, `tool:action:<domain>`, `tool:agent:<toolkit>`) are a turn-context source. The segment anchor stores each group's schemas and MCP instructions; `tools` and the MCP instructions are rendered from it until the next compaction. - The initial set orders MCP servers and their tools by name, other groups as the agent lists them, after the ungrouped tools. A compaction keeps the anchored order, drops groups that are off and appends the new ones. - A group enabled mid-segment is deferred and announced once in the user message with its schemas; the model calls it through the SDK dispatcher (`toolDispatch`), which resolves the call before hooks, approval and remembered approvals, so they see the real tool. An anchored group that is off keeps its schemas with `disabled`; an unreachable one is `unavailable` without an update. - A schema a server changes mid-segment is announced once, in the message sent with the change: the changed parameters, tools that are new (reached through the dispatcher) and tools that are gone. `tools` itself changes at the next anchor. - Tools found by `tool_search` stay deferred and are called through the dispatcher; the host search returns their schemas. - The formatter tool set keeps past dispatcher calls; isolated subagent copies bind the tools as loaded. - MCP servers and their tools load in name order, and MCP instructions are listed by server name; both only with the lever. The instructions of the named servers are read in packages/api, so MCPManager keeps its API. - UI: `resolveDisplayToolCall` routes a dispatcher call to the card and the group header of the tool it ran; the stream reducer keeps `dispatched`. The package depends on the tool dispatcher of @librechat/agents, which is not in a release yet; `package.json` moves to the release that has it.
…lare prefix resets Behind `endpoints.agents.promptCache.stableContext`, like the rest of the turn context. Switching web search, the code environment, memory or another built-in capability mid-conversation still changed `tools` and the system prompt, and changes the dispatcher cannot absorb (another model, an edited agent, provider tools, tool options that rewrite many schemas) mutated the anchored prompt silently. - Built-in tools are tool groups (`tool:builtin:<capability>`): web_search, file_search, the whole code environment, programmatic tool calling, skills, memory, ask_user_question, background tasks and tool_search. A new anchor binds them first, in a fixed order. Like other groups they are bound from the anchor, deferred and announced when enabled mid-segment, kept with `disabled` when switched off, and "available again" when switched back. - A built-in's instructions (`toolContextMap`) belong to its group: the system prompt states the anchored ones, a mid-segment group announces its own. - The file tools the code environment and skills both register are owned by the code group when the agent has a code environment, else by skills; an anchored file tool stays enabled while either group has it. - Declared resets: the anchor records the model, a digest of the agent's setup (instructions, handoffs, subagent targets) and the provider tools, serialized with sorted keys so the same setup built in another order never differs. A sent message that changes one of them, gains or loses a schema-profile parameter (tool intents, background runs) on an anchored tool, or enables a group the dispatcher cannot reach (the in-graph ask_user_question) re-anchors the segment on itself, as a compaction would: anchored order kept, deferred groups appended, switched-off ones dropped. Anchors recorded before the profile never reset. A regeneration sends no message, so it neither resets nor rewrites an anchor. - When the provider has a prompt cache (Anthropic, OpenRouter, Bedrock with promptCache), the reply stores the reasons in `metadata.promptCacheReset` and the chat shows a one-line notice that earlier context was reloaded. Nothing is recorded or shown for providers without one. - A switched-off ask_user_question stays bound in the graph and is refused through the registry; a deferred one is not bound. - The ask_user_question group header reads a dispatched call by the tool it ran.
…y them for restored tool rounds An OpenAI Responses tool round carries ids, status and encrypted reasoning that the next turn must send back as they were, or the request stops being a prefix of the previous one. The agents SDK reports them per finished round (`ON_PROVIDER_OUTPUT_ITEMS`) and restores a stored round from `tool_call.providerItems`. This depends on the SDK release that has the event; without it the handler is not registered and rounds are restored as before. - The handler adds the items to the run's content parts. `saveMessage` moves them out of `content` into a `providerItems` field of the message, keyed by tool call id, with `select: false`: no read, share, fork, export or search sees them unless it asks for the field, so there is no list of places that must remember to hide them. The history read of the agent client asks for it and puts the items back on their tool calls before formatting; the stored row is not changed. - Live content (the sync and final stream events, `/chat/status`) is cleaned on the known paths: `resumeState.aggregatedContent`, `aggregatedContent` and `responseMessage.content`. - Only the server writes them: a client message, an import or a top-level field never reaches the stored field. A fork or a copy reads without the field, so a forked conversation, including one forked from a shared link, starts without items and restores its rounds as before. Rounds sent through the OpenAI-compatible and Responses routes of this server keep their previous behavior: those routes have their own handlers and store no items.
With `endpoints.agents.promptCache.stableContext` and Anthropic, OpenRouter or Bedrock with promptCache, everything that is stable within the conversation (the anchored context, tool runtime context, the agent's own guidance) goes to the agents SDK as `conversation_instructions`. The SDK keeps that block right after the instructions with a breakpoint of its own and never moves it, and carries a compaction summary as a stable message of its own. Only the per-turn parts (memory, retrieved context, files queued for this turn) stay in `additional_instructions`, which the SDK relocates next to the newest user message or, on Bedrock, appends after the cached blocks. Before, the stable part was merged into `instructions`. A summary created in a run and the same summary restored by the next run now render identically, so a compaction costs one prefix rebuild instead of two. `conversation_instructions` needs the SDK release that has it; without it the lever stays off with a warning. Bedrock was not covered before and is not verified against a real endpoint.
codegraph-librechat
Bot
added
🗺️ Backend Infra
Shabonasar
marked this pull request as draft
October 16, 2026 06:21
…lare prefix resets Behind `endpoints.agents.promptCache.stableContext`, like the rest of the turn context. Switching web search, the code environment, memory or another built-in capability mid-conversation still changed `tools` and the system prompt, and changes the dispatcher cannot absorb (another model, an edited agent, provider tools, tool options that rewrite many schemas) mutated the anchored prompt silently. - Built-in tools are tool groups (`tool:builtin:<capability>`): web_search, file_search, the whole code environment, programmatic tool calling, skills, memory, ask_user_question, background tasks and tool_search. A new anchor binds them first, in a fixed order. Like other groups they are bound from the anchor, deferred and announced when enabled mid-segment, kept with `disabled` when switched off, and "available again" when switched back. - A built-in's instructions (`toolContextMap`) belong to its group: the system prompt states the anchored ones, a mid-segment group announces its own. - The file tools the code environment and skills both register are owned by the code group when the agent has a code environment, else by skills; an anchored file tool stays enabled while either group has it. - Declared resets: the anchor records the model, a digest of the agent's setup (instructions, handoffs, subagent targets) and the provider tools, serialized with sorted keys so the same setup built in another order never differs. A sent message that changes one of them, gains or loses a schema-profile parameter (tool intents, background runs) on an anchored tool, or enables a group the dispatcher cannot reach (the in-graph ask_user_question) re-anchors the segment on itself, as a compaction would: anchored order kept, deferred groups appended, switched-off ones dropped. Anchors recorded before the profile never reset. A regeneration sends no message, so it neither resets nor rewrites an anchor. - When the provider has a prompt cache (Anthropic, OpenRouter, Bedrock with promptCache), the reply stores the reasons in `metadata.promptCacheReset` and the chat shows a one-line notice that earlier context was reloaded. Nothing is recorded or shown for providers without one. - A switched-off ask_user_question stays bound in the graph and is refused through the registry; a deferred one is not bound. - The ask_user_question group header reads a dispatched call by the tool it ran.
…y them for restored tool rounds An OpenAI Responses tool round carries ids, status and encrypted reasoning that the next turn must send back as they were, or the request stops being a prefix of the previous one. The agents SDK reports them per finished round (`ON_PROVIDER_OUTPUT_ITEMS`) and restores a stored round from `tool_call.providerItems`. This depends on the SDK release that has the event; without it the handler is not registered and rounds are restored as before. - The handler adds the items to the run's content parts. `saveMessage` moves them out of `content` into a `providerItems` field of the message, keyed by tool call id, with `select: false`: no read, share, fork, export or search sees them unless it asks for the field, so there is no list of places that must remember to hide them. The history read of the agent client asks for it and puts the items back on their tool calls before formatting; the stored row is not changed. - Live content (the sync and final stream events, `/chat/status`) is cleaned on the known paths: `resumeState.aggregatedContent`, `aggregatedContent` and `responseMessage.content`. - Only the server writes them: a client message, an import or a top-level field never reaches the stored field. A fork or a copy reads without the field, so a forked conversation, including one forked from a shared link, starts without items and restores its rounds as before. Rounds sent through the OpenAI-compatible and Responses routes of this server keep their previous behavior: those routes have their own handlers and store no items.
With `endpoints.agents.promptCache.stableContext` and Anthropic, OpenRouter or Bedrock with promptCache, everything that is stable within the conversation (the anchored context, tool runtime context, the agent's own guidance) goes to the agents SDK as `conversation_instructions`. The SDK keeps that block right after the instructions with a breakpoint of its own and never moves it, and carries a compaction summary as a stable message of its own. Only the per-turn parts (memory, retrieved context, files queued for this turn) stay in `additional_instructions`, which the SDK relocates next to the newest user message or, on Bedrock, appends after the cached blocks. Before, the stable part was merged into `instructions`. A summary created in a run and the same summary restored by the next run now render identically, so a compaction costs one prefix rebuild instead of two. `conversation_instructions` needs the SDK release that has it; without it the lever stays off with a warning. Bedrock was not covered before and is not verified against a real endpoint.
…assified parts Producers (project, memory guard, artifacts, skills catalog, RAG, memory, agent context files, always-apply skills of isolated subagents) contribute keyed parts through contributeContext instead of concatenating strings into additional_instructions. One registry fixes the canonical order and classifies every part, tool context map entries included, as static, conversation or turn. A runtime key the registry does not know counts as turn, so a new producer cannot break the stable prefix by default. assembleSystemPrompt builds the strings handed to the SDK. Once parts exist they are the only source of the tail; a direct write to additional_instructions is ignored with a warning. Without promptCache.stableContext it reproduces the previous request; with it, createRun moves only the turn parts with the prompt-cache tail and merges the rest into the instructions. The assembled text is byte-identical to string concatenation for both values of the lever, guarded by a snapshot recorded on the previous implementation. Isolated subagents with an always-apply skill place the skill ahead of the tool runtime context when stableContext is on.
With `endpoints.agents.promptCache.stableContext` and Anthropic, OpenRouter or Bedrock with promptCache, everything that is stable within the conversation (the anchored context, tool runtime context, the agent's own guidance) goes to the agents SDK as `conversation_instructions`. The SDK keeps that block right after the instructions with a breakpoint of its own and never moves it, and carries a compaction summary as a stable message of its own. Only the per-turn parts (memory, retrieved context, files queued for this turn) stay in `additional_instructions`, which the SDK relocates next to the newest user message or, on Bedrock, appends after the cached blocks. Before, the stable part was merged into `instructions`. A summary created in a run and the same summary restored by the next run now render identically, so a compaction costs one prefix rebuild instead of two. `conversation_instructions` needs the SDK release that has it; without it the lever stays off with a warning. Bedrock was not covered before and is not verified against a real endpoint.
…ched conversation block
…ed tools and instructions A librechat.yaml MCP server hidden from the acting user by a config profile is an access boundary: its instructions must not reach the model and its tools must not load. An anchored group of such a server stayed bound from its stored schemas, with its instructions in the stable system prompt, until the next anchor. - `LiveToolGroups.hidden` holds the group keys of the servers hidden from the user, read where the groups are sampled through an injectable reader (`getHiddenMCPServerNames`). The raw names are normalized like the group keys. A failed read is not swallowed: hiding fails closed upstream, and a turn that cannot tell what is hidden must not bind its schemas. - The tool plan leaves an anchored MCP group out, schemas and instructions, when its server is hidden and not loaded. A loaded group with a hidden name is a user-sourced server and stays.
…m the user An anchored group of a server hidden from the acting user is left out of the tool plan, but the schemas and instructions stayed in the anchor until the next one. A hidden anchored server that is not loaded now declares a reset (`mcp_hidden`): the sent message opens a new anchor built from the loaded groups and the filtered instructions, so the server's tools and instructions leave the prompt. A loaded group with a hidden name is a user-sourced server and keeps its anchor; a server that is only switched off still stays bound, refusing calls.
…memory Since LibreChat-AI#16301 formatMemoryContext('') returns the bare memory instructions sentence, and prepareRuntimeAgent applied it to the primary agent whenever the user had no saved memories. An agent with no memory tools and no automatic memory extraction can neither read nor write memory, so the sentence told the model nothing it could act on, yet it reached the model as per-turn context. The empty-store notice is now added only when the agent has memory tools or automatic extraction is on. Saved memories are still injected as before.
…m prompt per segment Behind `endpoints.agents.promptCache.stableContext`; off, nothing here runs and the prompt is built as before. A single send-time step (turnContext) decides what the primary agent's model sees. Context sources (project, artifacts mode, memory tool guard, skills catalog, tool runtime context, memory, date, send time, retrieved context, queued upload files) are sampled once when the user sends a message; only the net difference from what the model already knows along the parent chain is stored on that user message (message.contextUpdates) and rendered as one <context_update> block at the start of its prompt copy. Toggles flipped back and forth before sending leave no trace, and stored values render the same bytes on every later turn, whatever the timezone. A regeneration sends no message: it renders what the message stored and writes nothing. The system prompt is rendered from the segment's anchor instead of the live parts, so it stays byte-identical until the next compaction. The anchor of the first segment is stored on the conversation, the anchor of a segment opened by a compaction on the message carrying the summary (contextAnchor); a conversation without one is anchored to its first turn after the upgrade. A re-anchor keeps the previous order, drops what is off and appends what is new, and dates itself, so the first message after the boundary announces nothing. Memory is a state source: an update lists the entries added, changed and removed (by key where the memory has keys) instead of the whole memory, and no longer rewrites the system prompt; memory switched off is withheld from the whole request. Retrieved context is stored with the user message it was retrieved for, like the file context bound to the message that carried the files; isolated subagents still take it in their system prompt. Queued upload files describe their own message only. A resumed turn (tool approval or an answered question) renders its system prompt from the paused turn's anchor, not from the live state. The planning, the anchor storage and the token accounting live in packages/api; the controller calls them. The conversation anchor is read with the conversation the ingress already loads, and a resume reads the messages once with a projection. contextUpdates and contextAnchor are server-written only, select: false, and stripped from client input, so they stay out of reads, shares and forks.
A handoff target that shares the primary agent's memory partition kept a live copy of the memory in its per-turn tail. With prompt caching the SDK relocates that tail next to the newest user message, so the memory sat at a different place on every turn: the target's earlier requests no longer matched its next one, which missed the cache and, for models that bind thinking blocks to the prefix, made the API drop the target's own thinking blocks. The memory the turn context anchors for the segment now goes into the target's conversation block, and its changes reach it as the <context_update> blocks already stored on the history messages, as they reach the primary agent. Targets on another memory partition keep their live copy.
…he runtime note stays byte-stable The code-file note of an attached workspace is runtime context that the turn context anchors and updates. Priming returns files by last update, which moves with usage, so the same files could be listed in another order on a later turn and the note was announced again without having changed.
…se memory The send snapshot formatted an empty memory store as the bare memory instructions sentence whatever the agent could do with it, so the turn context anchored a sentence that tells the model nothing it can act on. The snapshot now applies the rule of prepareRuntimeAgent: an empty store is announced only to an agent with memory tools or when automatic extraction is on. Saved memories are anchored as before.
…d the SDK dispatcher Behind `endpoints.agents.promptCache.stableContext`, which also stays off, with one warning, while the installed @librechat/agents has no tool dispatcher. Providers serialize `tools` ahead of the system prompt, so switching an MCP server, an action or an agent tool mid-conversation reset the prompt cache from position 0. - Tool groups (`tool:mcp:<server>`, `tool:action:<domain>`, `tool:agent:<toolkit>`) are a turn-context source. The segment anchor stores each group's schemas and MCP instructions; `tools` and the MCP instructions are rendered from it until the next compaction. - The initial set orders MCP servers and their tools by name, other groups as the agent lists them, after the ungrouped tools. A compaction keeps the anchored order, drops groups that are off and appends the new ones. - A group enabled mid-segment is deferred and announced once in the user message with its schemas; the model calls it through the SDK dispatcher (`toolDispatch`), which resolves the call before hooks, approval and remembered approvals, so they see the real tool. An anchored group that is off keeps its schemas with `disabled`; an unreachable one is `unavailable` without an update. - A schema a server changes mid-segment is announced once, in the message sent with the change: the changed parameters, tools that are new (reached through the dispatcher) and tools that are gone. `tools` itself changes at the next anchor. - Tools found by `tool_search` stay deferred and are called through the dispatcher; the host search returns their schemas. - The formatter tool set keeps past dispatcher calls; isolated subagent copies bind the tools as loaded. - MCP servers and their tools load in name order, and MCP instructions are listed by server name; both only with the lever. The instructions of the named servers are read in packages/api, so MCPManager keeps its API. - UI: `resolveDisplayToolCall` routes a dispatcher call to the card and the group header of the tool it ran; the stream reducer keeps `dispatched`. The package depends on the tool dispatcher of @librechat/agents, which is not in a release yet; `package.json` moves to the release that has it.
…lare prefix resets Behind `endpoints.agents.promptCache.stableContext`, like the rest of the turn context. Switching web search, the code environment, memory or another built-in capability mid-conversation still changed `tools` and the system prompt, and changes the dispatcher cannot absorb (another model, an edited agent, provider tools, tool options that rewrite many schemas) mutated the anchored prompt silently. - Built-in tools are tool groups (`tool:builtin:<capability>`): web_search, file_search, the whole code environment, programmatic tool calling, skills, memory, ask_user_question, background tasks and tool_search. A new anchor binds them first, in a fixed order. Like other groups they are bound from the anchor, deferred and announced when enabled mid-segment, kept with `disabled` when switched off, and "available again" when switched back. - A built-in's instructions (`toolContextMap`) belong to its group: the system prompt states the anchored ones, a mid-segment group announces its own. - The file tools the code environment and skills both register are owned by the code group when the agent has a code environment, else by skills; an anchored file tool stays enabled while either group has it. - Declared resets: the anchor records the model, a digest of the agent's setup (instructions, handoffs, subagent targets) and the provider tools, serialized with sorted keys so the same setup built in another order never differs. A sent message that changes one of them, gains or loses a schema-profile parameter (tool intents, background runs) on an anchored tool, or enables a group the dispatcher cannot reach (the in-graph ask_user_question) re-anchors the segment on itself, as a compaction would: anchored order kept, deferred groups appended, switched-off ones dropped. Anchors recorded before the profile never reset. A regeneration sends no message, so it neither resets nor rewrites an anchor. - When the provider has a prompt cache (Anthropic, OpenRouter, Bedrock with promptCache), the reply stores the reasons in `metadata.promptCacheReset` and the chat shows a one-line notice that earlier context was reloaded. Nothing is recorded or shown for providers without one. - A switched-off ask_user_question stays bound in the graph and is refused through the registry; a deferred one is not bound. - The ask_user_question group header reads a dispatched call by the tool it ran.
…y them for restored tool rounds An OpenAI Responses tool round carries ids, status and encrypted reasoning that the next turn must send back as they were, or the request stops being a prefix of the previous one. The agents SDK reports them per finished round (`ON_PROVIDER_OUTPUT_ITEMS`) and restores a stored round from `tool_call.providerItems`. This depends on the SDK release that has the event; without it the handler is not registered and rounds are restored as before. - The handler adds the items to the run's content parts. `saveMessage` moves them out of `content` into a `providerItems` field of the message, keyed by tool call id, with `select: false`: no read, share, fork, export or search sees them unless it asks for the field, so there is no list of places that must remember to hide them. The history read of the agent client asks for it and puts the items back on their tool calls before formatting; the stored row is not changed. - Live content (the sync and final stream events, `/chat/status`) is cleaned on the known paths: `resumeState.aggregatedContent`, `aggregatedContent` and `responseMessage.content`. - Only the server writes them: a client message, an import or a top-level field never reaches the stored field. A fork or a copy reads without the field, so a forked conversation, including one forked from a shared link, starts without items and restores its rounds as before. Rounds sent through the OpenAI-compatible and Responses routes of this server keep their previous behavior: those routes have their own handlers and store no items.
With `endpoints.agents.promptCache.stableContext` and Anthropic, OpenRouter or Bedrock with promptCache, everything that is stable within the conversation (the anchored context, tool runtime context, the agent's own guidance) goes to the agents SDK as `conversation_instructions`. The SDK keeps that block right after the instructions with a breakpoint of its own and never moves it, and carries a compaction summary as a stable message of its own. Only the per-turn parts (memory, retrieved context, files queued for this turn) stay in `additional_instructions`, which the SDK relocates next to the newest user message or, on Bedrock, appends after the cached blocks. Before, the stable part was merged into `instructions`. A summary created in a run and the same summary restored by the next run now render identically, so a compaction costs one prefix rebuild instead of two. `conversation_instructions` needs the SDK release that has it; without it the lever stays off with a warning. Bedrock was not covered before and is not verified against a real endpoint.
…ched conversation block
…ed tools and instructions A librechat.yaml MCP server hidden from the acting user by a config profile is an access boundary: its instructions must not reach the model and its tools must not load. An anchored group of such a server stayed bound from its stored schemas, with its instructions in the stable system prompt, until the next anchor. - `LiveToolGroups.hidden` holds the group keys of the servers hidden from the user, read where the groups are sampled through an injectable reader (`getHiddenMCPServerNames`). The raw names are normalized like the group keys. A failed read is not swallowed: hiding fails closed upstream, and a turn that cannot tell what is hidden must not bind its schemas. - The tool plan leaves an anchored MCP group out, schemas and instructions, when its server is hidden and not loaded. A loaded group with a hidden name is a user-sourced server and stays.
…m the user An anchored group of a server hidden from the acting user is left out of the tool plan, but the schemas and instructions stayed in the anchor until the next one. A hidden anchored server that is not loaded now declares a reset (`mcp_hidden`): the sent message opens a new anchor built from the loaded groups and the filtered instructions, so the server's tools and instructions leave the prompt. A loaded group with a hidden name is a user-sourced server and keeps its anchor; a server that is only switched off still stays bound, refusing calls.
…lied context With stableContext, the per-turn parts (memory and RAG context of agents that do not take them from the turn context, queued file notes, runtime keys without a conversation date) stay in additional_instructions. For Anthropic and OpenRouter with prompt caching the SDK sends that text as a plain message in the user role right before the newest user message, with nothing around it, so a model that works on the user's text verbatim (a translator, a summarizer) takes it for part of that text. The tail is now wrapped in a fixed <system_context> frame that says the system supplied it, for every provider whose SDK relocates the tail and for isolated children, whose whole tail is relocated. The frame is the same text every turn; only the tail inside it changes. Without stableContext nothing changes.
…ory and its checkpoints Captures the Converse request before the client sends it over three turns with stableContext and checks it against the prompt caching limits of Bedrock: at most four checkpoints across tools, system and messages, one TTL, each checkpoint closing a content block, the tools and the stable system text closed by a checkpoint with no per-turn text after it, one checkpoint at the end of the history in front of the moved tail and one at the end of the request. The system, the tools and the history in front of the tail are byte-identical between turns, also without stableContext. Bedrock now joins the providers whose SDK moves the per-turn tail (@librechat/agents: the tail goes behind the history instead of after the last system checkpoint).
Contributor
Author
|
Added:
Delta: Shabonasar/LibreChat@21495fb...15ed355 — |
1 task done
No file changes (tree of dev c81e244). Joins earlier snapshots of the series that the branches share, so any two of them have a single merge base and merge cleanly.
… tool rounds OpenAI Responses tool rounds carry ids, status and encrypted reasoning that the next turn has to send back unchanged. They are stored beside the message (server-side only, capped per message) and put back on the restored tool calls, so a restored round equals the request sent live. Active when the installed agents SDK reports them (LibreChat-AI/agents#615); otherwise requests are built as before.
Shabonasar
added a commit
to Shabonasar/LibreChat
that referenced
this pull request
Oct 28, 2026
Shabonasar
changed the title
Contributor
Author
|
Reduced to minimize dependencies and the review size: +12894 -> +633 lines, based on |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An OpenAI Responses tool round carries ids, status and encrypted reasoning that the next turn must send back unchanged. LibreChat restores the round without them, so the restored history differs from the live request and the cache is written again from that round. This PR stores the provider output items beside the message (server only, capped at 2 MB per message) and puts them back on the restored tool calls, so a restored round equals the request sent live. It only makes the restored history equal to the live one, so it is not behind a setting: it is active when the installed SDK emits
ON_PROVIDER_OUTPUT_ITEMS; otherwise nothing is registered or stored and requests are as ondev.Related to #17102. Base:
dev, no other LibreChat PR needed. Depends on LibreChat-AI/agents#615.How it works
/chat/status, abort) is stripped.Merging with #17146 / #17147 / #17514
One known conflict: both sides replace the select in
BaseClient.loadHistory. Resolution, and the same string in the stableContext expectation ofBaseClient.test.js("loadHistory should scope database reads to the current user"):Results
Live this round on the Anthropic API only (Claude Haiku 5.5, all PRs of #17102 together, cache TTL 1h): tools with thinking 87.4%, G2 85.1%, both 0 misses, no API errors. Responses rounds are covered by the guard and unit tests here and by LibreChat-AI/agents#615.
Changes from the previous version
dev+12894 -303 -> +633 -17 (tests +427); based ondev, no longer on feat: Route Built-in Capabilities Through Tool Groups and Declare Prefix Resets #17148.stableContext.conversation_instructions) moved to feat: Pass the Stable Context as the SDK Conversation Block #17514; the frame of the relocated tail removed (devhasframeDynamicInstructions).Type of change
Testing
Run two tool rounds with a Responses reasoning model and check that turn N+1 starts with the bytes of turn N.
Automated tests:
BaseClient.test.js,callbacks.spec.js,client.test.js,abortMiddleware.spec.js,messages-get.spec.js,messages-retention.spec.js,streamTenant.spec.js,importers.spec.js,message.spec.ts(packages/api, data-schemas),share.test.ts,providerItems.spec.ts. Guard:client.test.js"builds the prompt of a tool round as upstream when no provider items are stored" (inline snapshot recorded ondev). Against the npm SDK one test fails (needsON_PROVIDER_OUTPUT_ITEMS).tsc --noEmitin packages/api, packages/data-schemas; eslint and prettier on the touched files.Risk / compatibility
New non-selected
providerItemsfield on messages; nothing is stored without the SDK event.Checklist