Skip to content

feat: support Bedrock prompt cache TTL#123

Open
SharpLu wants to merge 1 commit intodanny-avila:mainfrom
SharpLu:feat/bedrock-cache-ttl
Open

feat: support Bedrock prompt cache TTL#123
SharpLu wants to merge 1 commit intodanny-avila:mainfrom
SharpLu:feat/bedrock-cache-ttl

Conversation

@SharpLu
Copy link
Copy Markdown

@SharpLu SharpLu commented Apr 29, 2026

Summary

Adds Bedrock prompt cache TTL support to the agents Bedrock runtime path. Bedrock cache checkpoints now allow an optional ttl value (5m or 1h), and callers can pass promptCacheTtl so the generated cachePoint blocks include that TTL.

When promptCacheTtl is omitted, no TTL is sent and Bedrock keeps its default 5-minute prompt cache behavior.

Changes

  • Add promptCacheTtl?: "5m" | "1h" to Bedrock Anthropic input/client options
  • Pass the TTL into addBedrockCacheControl() from the graph Bedrock options
  • Preserve cachePoint.ttl through Bedrock message conversion for user, assistant, and system blocks
  • Add tests for default behavior, 5m, 1h, and message conversion

References

Testing

  • npm test -- --runInBand src/messages/cache.test.ts src/llm/bedrock/llm.spec.ts
  • npm run build

@SharpLu SharpLu force-pushed the feat/bedrock-cache-ttl branch from d2c8023 to 5c69ab2 Compare May 8, 2026 08:36
@SharpLu
Copy link
Copy Markdown
Author

SharpLu commented May 8, 2026

Hey @danny-avila, friendly bump — just rebased onto main (v3.1.79), tests pass.

This is the runtime half of danny-avila/LibreChat#12875: it threads the optional promptCacheTtl from BedrockAnthropicClientOptions through addBedrockCacheControl so the inserted cachePoint blocks carry the TTL. Defaults stay unchanged when no TTL is provided, and we strip 1h for models that don't support it (Nova, non-4.5 Claude) to avoid AWS validation errors.

Both PRs need to land together for the feature to work end-to-end.

Thanks for taking a look!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants