fix #20326 - [Feature]: Support TTL(1h) field in prompt caching for Bedrock Claude 4.5 models by Lucky-Lodhi2004 · Pull Request #20338 · BerriAI/litellm

Lucky-Lodhi2004 · 2026-02-03T12:10:53Z

Relevant issues

fixes issue #20326

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

50-55 passing tests: main is stable with minor issues.

45-49 passing tests: acceptable but needs attention

<= 40 passing tests: unstable; be careful with your merges and assess the risk.

Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

added ttl support in claude 4.5 models.

vercel · 2026-02-03T12:10:59Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 5, 2026 11:23am

greptile-apps · 2026-02-03T12:15:28Z

Greptile Overview

Greptile Summary

This PR adds support for TTL (Time-To-Live) field in prompt caching for AWS Bedrock Claude 4.5 models, allowing "5m" and "1h" TTL values to be passed through to the API.

Changes made:

Added ttl field to CachePointBlock TypedDict in litellm/types/llms/bedrock.py
Modified Converse API transformation to extract and validate TTL values from cache_control
Modified Invoke API transformation to conditionally preserve TTL based on model version
Added _is_claude_4_5_on_bedrock() method to detect Claude 4.5 models (Sonnet, Haiku, Opus)

Critical issues found:

Logic error (line 156-157 in anthropic_claude3_transformation.py): The code preserves TTL values for ALL models instead of only Claude 4.5, which will break the existing unit test test_remove_ttl_from_cache_control
No tests added: The PR checklist requires adding at least 1 test, but no tests were added to validate the new TTL support for Claude 4.5 models

Additional concerns:

Converse API implementation doesn't check model version before allowing TTL - needs verification if this is correct behavior
Debug comments left in code (lines 151-154)
Typo in comment ("bedock" instead of "Bedrock")

Confidence Score: 1/5

This PR is NOT safe to merge - it contains a critical logic error that will break existing tests and cause incorrect behavior
Score of 1/5 due to critical logic bug (lines 156-157 preserve TTL for all models instead of only Claude 4.5), missing required tests, and uncertainty about Converse API behavior
Pay immediate attention to litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py lines 156-157 - critical logic error must be fixed before merge

Important Files Changed

Filename	Overview
litellm/types/llms/bedrock.py	Added `ttl` field to `CachePointBlock` TypedDict to support TTL values in prompt caching
litellm/llms/bedrock/chat/converse_transformation.py	Modified `_get_cache_point_block` to extract and validate TTL values ("5m" or "1h") from cache_control before creating cache point blocks
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py	Added Claude 4.5 detection method and modified TTL removal logic, but contains critical logic error that preserves TTL for all models instead of only Claude 4.5

Sequence Diagram

sequenceDiagram
    participant Client
    participant LiteLLM
    participant ConverseAPI as Bedrock Converse API
    participant InvokeAPI as Bedrock Invoke API
    
    Note over Client,InvokeAPI: TTL Prompt Caching Flow
    
    Client->>LiteLLM: messages with cache_control: {type: "ephemeral", ttl: "1h"}
    
    alt Converse API Path
        LiteLLM->>LiteLLM: _get_cache_point_block()
        LiteLLM->>LiteLLM: Extract ttl from cache_control
        alt ttl is "5m" or "1h"
            LiteLLM->>LiteLLM: Add ttl to CachePointBlock
        else other ttl value
            LiteLLM->>LiteLLM: Create CachePointBlock without ttl
        end
        LiteLLM->>ConverseAPI: Request with cachePoint: {type: "default", ttl: "1h"}
        ConverseAPI->>LiteLLM: Response with cache metrics
    else Invoke API Path
        LiteLLM->>LiteLLM: _remove_ttl_from_cache_control(model)
        LiteLLM->>LiteLLM: _is_claude_4_5_on_bedrock(model)
        alt Claude 4.5 model AND ttl is "5m" or "1h"
            LiteLLM->>LiteLLM: Keep ttl in cache_control
            Note over LiteLLM: BUG: Lines 156-157<br/>always keep "5m"/"1h"<br/>for ALL models
        else Other model OR other ttl
            LiteLLM->>LiteLLM: Remove ttl from cache_control
        end
        LiteLLM->>InvokeAPI: Request with modified cache_control
        InvokeAPI->>LiteLLM: Response with cache metrics
    end
    
    LiteLLM->>Client: Return response with cache tokens

greptile-apps

_{3 files reviewed, 4 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-02-03T12:15:31Z

litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py

+                                    if ttl in ["5m", "1h"]:
+                                        continue


Critical logic error: this code preserves TTL values of "5m" and "1h" for ALL models, not just Claude 4.5 models. This will break the existing test at tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py:108-113 which expects TTL to be removed when no model is passed.

The lines 148-149 already handle the Claude 4.5 case, so lines 156-157 should be deleted. Without them, the logic will be:

If Claude 4.5 and TTL is "5m" or "1h" → keep it (line 148-149)

Otherwise, if TTL exists → remove it (line 159)

Suggested change

if ttl in ["5m", "1h"]:

continue

cache_control.pop("ttl", None)

Prompt To Fix With AI

This is a comment left during a code review. Path: litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py Line: 156:157 Comment: Critical logic error: this code preserves TTL values of `"5m"` and `"1h"` for ALL models, not just Claude 4.5 models. This will break the existing test at `tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py:108-113` which expects TTL to be removed when no model is passed. The lines 148-149 already handle the Claude 4.5 case, so lines 156-157 should be deleted. Without them, the logic will be: - If Claude 4.5 and TTL is "5m" or "1h" → keep it (line 148-149) - Otherwise, if TTL exists → remove it (line 159) ```suggestion cache_control.pop("ttl", None) ``` How can I resolve this? If you propose a fix, please make it concise.

litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py

greptile-apps · 2026-02-03T12:15:33Z

litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py

        Remove `ttl` field from cache_control in messages.
        Bedrock doesn't support the ttl field in cache_control.

+        Update: bedock supports `5m` and `1h` for Claude 4.5 models.


Typo: "bedock" should be "Bedrock"

Suggested change

Update: bedock supports `5m` and `1h` for Claude 4.5 models.

Update: Bedrock supports `5m` and `1h` for Claude 4.5 models.

Prompt To Fix With AI

This is a comment left during a code review. Path: litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py Line: 125:125 Comment: Typo: "bedock" should be "Bedrock" ```suggestion Update: Bedrock supports `5m` and `1h` for Claude 4.5 models. ``` How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2026-02-03T12:15:34Z

litellm/llms/bedrock/chat/converse_transformation.py

+        cache_point = CachePointBlock(type="default")
+        if isinstance(cache_control, dict) and "ttl" in cache_control:
+            ttl = cache_control["ttl"]
+            if ttl in ["5m", "1h"]:
+                cache_point["ttl"] = ttl


Verify whether the Converse API supports TTL for all Claude models or only Claude 4.5 models. The Invoke API implementation (in anthropic_claude3_transformation.py) restricts TTL support to Claude 4.5 models. If the Converse API has the same restriction, this code should also check the model version using a method similar to _is_claude_4_5_on_bedrock().

Prompt To Fix With AI

This is a comment left during a code review. Path: litellm/llms/bedrock/chat/converse_transformation.py Line: 957:961 Comment: Verify whether the Converse API supports TTL for all Claude models or only Claude 4.5 models. The Invoke API implementation (in `anthropic_claude3_transformation.py`) restricts TTL support to Claude 4.5 models. If the Converse API has the same restriction, this code should also check the model version using a method similar to `_is_claude_4_5_on_bedrock()`. How can I resolve this? If you propose a fix, please make it concise.

@Lucky-Lodhi2004 Please make fixes based on the reviews

ok, i made the proposed changes, but i have a question, why do mcp tests always fail when i make a pull request? do failing mcp tests may cause my PR to be rejected (not merged)?

…-bedrock

added 1h ttl support for aws bedrock

1aa619d

vercel bot deployed to Preview February 3, 2026 12:12 View deployment

greptile-apps bot reviewed Feb 3, 2026

View reviewed changes

fixed linting

b693458

vercel bot deployed to Preview February 3, 2026 13:07 View deployment

removing thinking lines

c1aa1c3

vercel bot deployed to Preview February 3, 2026 13:53 View deployment

fixed typo

a25289e

vercel bot deployed to Preview February 3, 2026 13:55 View deployment

made changes suggested by agent

1e55e61

vercel bot deployed to Preview February 4, 2026 13:13 View deployment

Lucky-Lodhi2004 added 2 commits February 4, 2026 13:20

fixed linting

31e2e72

Merge branch 'main' into ttl-prompt-caching-bedrock

e4b45dc

vercel bot deployed to Preview February 4, 2026 13:24 View deployment

Merge branch 'main' into ttl-prompt-caching-bedrock

7e8be5f

Sameerlite changed the base branch from main to litellm_oss_staging_02_04_2026 February 5, 2026 11:21

Merge branch 'litellm_oss_staging_02_04_2026' into ttl-prompt-caching…

fa733c2

…-bedrock

Sameerlite merged commit 34e5bb2 into BerriAI:litellm_oss_staging_02_04_2026 Feb 5, 2026
3 of 4 checks passed

vercel bot deployed to Preview February 5, 2026 11:23 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix #20326 - [Feature]: Support TTL(1h) field in prompt caching for Bedrock Claude 4.5 models #20338

fix #20326 - [Feature]: Support TTL(1h) field in prompt caching for Bedrock Claude 4.5 models #20338
Sameerlite merged 9 commits intoBerriAI:litellm_oss_staging_02_04_2026from
Lucky-Lodhi2004:ttl-prompt-caching-bedrock

Lucky-Lodhi2004 commented Feb 3, 2026

Uh oh!

vercel bot commented Feb 3, 2026 •

edited

Loading

Uh oh!

greptile-apps bot commented Feb 3, 2026

Important Files Changed

Uh oh!

greptile-apps bot left a comment

Uh oh!

greptile-apps bot Feb 3, 2026

Uh oh!

Uh oh!

greptile-apps bot Feb 3, 2026

Uh oh!

greptile-apps bot Feb 3, 2026

Uh oh!

Sameerlite Feb 4, 2026

Uh oh!

Lucky-Lodhi2004 Feb 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	if ttl in ["5m", "1h"]:
	continue
	cache_control.pop("ttl", None)

	Update: bedock supports `5m` and `1h` for Claude 4.5 models.
	Update: Bedrock supports `5m` and `1h` for Claude 4.5 models.

Uh oh!

Conversation

Lucky-Lodhi2004 commented Feb 3, 2026

Relevant issues

Pre-Submission checklist

CI (LiteLLM team)

Type

Changes

Uh oh!

vercel bot commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps bot commented Feb 3, 2026

Greptile Overview

Greptile Summary

Confidence Score: 1/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps bot Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Sameerlite Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Lucky-Lodhi2004 Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel bot commented Feb 3, 2026 •

edited

Loading