AWS: Prompt Caching Cut AI Agent Input Token Costs by 62%
An agent sends the model all its tools, instructions, and history again at every step, and you pay for those tokens every time. On OpenAI models in Bedrock, AWS showed how caching stores the unchanged start of a request and charges a fraction of the price for it. In a six-turn test with GPT-6 Astra, input token costs fell by about 62% compared with running without caching. So put tools and instructions at the start of the prompt, and move the date and request ID to the end.
AI-processed from AWS Machine Learning Blog; edited by Hamidun News
Imagine your AI agent waking up every morning like the hero of "Groundhog Day" with no memory of anything. That is effectively the case: a model has no memory between requests. At each step (calling a tool, reading the result, deciding what to do next), the agent resends the entire history to the model: tool descriptions, instructions, every previous call, and every result.
By the twentieth turn of a coding session, the model rereads turns one through nineteen, and you pay for each of those input tokens over and over. In its blog, AWS showed how prompt caching addresses this on OpenAI models in Amazon Bedrock. In a six-turn test on GPT-6 Astra, input token costs came out about 62% lower than without caching.
How the cache works and what to put in it
Before responding, the model processes the entire input and builds a working state from it: a set of keys and values (KV). The cache stores this state. If the next request starts with the same tokens, the service loads the saved state and computes only the new tail. Tokens read from the cache cost noticeably less than regular ones.
An agent request consists of four parts: the tool list, developer instructions, session history, and the new turn. The first two change only with a release, the history grows, and the last part differs every time. This leads to a practical rule: everything stable goes at the start, and everything variable (date, username, request ID) goes after the cacheable instructions. The cache hits only on an exact text match, so tools must be serialized identically on every request, including when they come from MCP servers.
There are two modes. In implicit mode, Bedrock decides where the reusable start of the request ends. In explicit mode, you place a prompt_cache_breakpoint marker wherever you want caching to end. AWS recommends combining both: an explicit marker after the instructions creates a shared starting point for different sessions and helper agents, while implicit mode caches the growing dialogue on its own. Keep the minimum size in mind: the cacheable prefix must be at least 1024 tokens. In the examples, the cache lifetime is 30 minutes, and the prompt_cache_key parameter tells the service which requests should share one cache. The store=False parameter disables storage of responses but does not turn off caching: the history stays in your application.
Numbers from the test
In one of the tests, the prefix made up of tools and instructions was about 4000 tokens. On the first request, 4120 tokens were written, and on subsequent requests the same 4120 tokens were read from the cache. The user's task sits after the cache boundary, so it does not create a new write.
On the six-turn workload, GPT-6 Astra showed the best result in implicit mode: input costs about 62% lower than without caching. Explicit sliding markers gave about 61% on the same model. In both cases, the cache reused more and more of the history as the dialogue grew. AWS is explicit that these are figures for one specific workload, and you should measure savings for your own agent. The API response includes cached_tokens and cache_write_tokens counters for this purpose.
Common mistakes and what to do
The first mistake applies only to explicit mode. If you keep a single static marker, the tools and instructions will be cached, but the growing history will be reprocessed on every turn. The second is moving the single marker forward. The cache may then fail to reuse, so it is better to keep the markers from earlier turns in the history. The third involves implicit mode and a changing task. A new task does not match the old prefix, and without an explicit marker after the shared instructions, the short common prefix will not be saved as a separate entry.
There is also a limitation: the figures in the article come from one model and one six-turn workload. An agent with long tool outputs or frequent prompt edits will see different savings.
What agent developers should do. First, tidy up the prompt: stable content at the start, variable content at the end, and identical tool serialization. Second, use both schemes together: an explicit marker after the instructions and implicit mode for the history. Third, log the share of tokens read from the cache right away. If it is low, something at the start of the request is changing, and you are overpaying for every turn.
- Nov 22, 2026Analysts estimate: by 2028, gas will supply 87% of Anthropic's power and coal 11%
- Nov 22, 2026AWS Quick gets a Meta Ads connector: campaigns and reports from a text prompt
- Nov 22, 2026LangChain, a Side Project Turned $3 Billion Startup in Four Years, Raises $300 Million
- Nov 21, 2026Amazon Offers AWS Quick AI Agent with Claude for Free, Plus Tier at $25 a Month
Key Questions Answered
How the cache works and what to put in it?
Before responding, the model processes the entire input and builds a working state from it: a set of keys and values (KV). The cache stores this state. If the next request starts with the same tokens, the service loads the saved state and computes only the new tail. Tokens read from the cache cost noticeably less than regular ones. An agent request consists of four parts: the tool list, developer instructions, session history, and the new turn.
What are the key details on Numbers from the test?
In one of the tests, the prefix made up of tools and instructions was about 4000 tokens. On the first request, 4120 tokens were written, and on subsequent requests the same 4120 tokens were read from the cache. The user's task sits after the cache boundary, so it does not create a new write. On the six-turn workload, GPT-6 Astra showed the best result in implicit mode: input costs about 62% lower than without caching. Explicit sliding markers gave about 61% on the same model.
What are the key details on Common mistakes and what to do?
The first mistake applies only to explicit mode. If you keep a single static marker, the tools and instructions will be cached, but the growing history will be reprocessed on every turn. The second is moving the single marker forward. The cache may then fail to reuse, so it is better to keep the markers from earlier turns in the history. The third involves implicit mode and a changing task.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.