Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarAWS Machine Learning Blog→ original

Amazon Bedrock Adds Explicit Prompt Caching for Open-Weight Models

If your application resends a long instruction or document with every request, you pay to process it again and again. Amazon Bedrock now supports explicit prompt caching for open-weight models: a shared prefix is counted once and then read from cache at a reduced price and faster. There are two modes: implicit works automatically, while explicit requires a "cache up to here" marker. That is why it's best to put everything static at the start of the prompt and everything that changes at the end.

AI-processed from AWS Machine Learning Blog; edited by Hamidun News
Amazon Bedrock Adds Explicit Prompt Caching for Open-Weight Models
Source: AWS Machine Learning Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceAWS Machine Learning Blog ↗
Editor-in-Chief
Key metric #1Explicit mode is enabled with the prompt_cache_options field and the prompt_cache_breakpoint marker
Key metric #2Cache reads are billed below the regular input price, while cache writes cost more for some models
Key metric #3OpenCode works with implicit caching without any configuration: each step processes only the new tail

Most AI applications send the model the same content with every request: a system instruction, reference documents, tool descriptions, and dialogue history. You pay to process this repeated chunk every time, and you wait for it every time too. Amazon Bedrock now supports explicit prompt caching for open-weight models. It lets you compute a shared prefix once and then read it from cache at a lower cost and with less latency.

How It Works

The principle is simple. If a new request starts with the same text as the previous one, Bedrock takes the beginning from the cache and processes only what comes after it. Cache reads are billed below the regular input token price. For some models, writing to the cache costs more than standard input, so you should check the exact rates in the pricing for the specific model. The longer the repeated context and the more often it is used, the greater the savings.

There are two modes. Implicit caching requires no changes to your requests: Bedrock selects suitable prefixes itself on a best-effort basis. It works well for dialogues and agent loops, where each subsequent request contains the previous one. Explicit caching is for when you want to decide what gets cached. To do this, you pass prompt_cache_options with the explicit mode to the Chat Completions API and place a prompt_cache_breakpoint marker on the text part where the cache should end. Everything up to and including the marker is written to the cache, and everything after it is processed anew. You can place several markers if layers change at different frequencies, for example, shared instructions followed by a specific customer's document.

A caveat: a marker without explicit mode works like implicit caching, and explicit mode without a marker caches nothing. The parameters must be set together.

Example: A Water Filter Support Assistant

AWS demonstrates this with a hypothetical assistant for the AquaPure 500 filter. The system prompt contains an instruction and a user manual of about 4,000 tokens, and each request asks a new customer question. If the model has a minimum prefix length, the manual must be padded up to it. For the test, placeholder text is fine, since the cache only cares about prefix matching.

The client is a standard OpenAI SDK pointed at the Bedrock endpoint. Authentication uses a short-lived API key (up to 12 hours) generated by the provide_token() function, so long-lived keys do not need to be stored. The response's usage.prompt_tokens_details shows how many tokens were read from the cache (cached_tokens) and how many were written (cache_write_tokens).

In implicit mode, the picture looks like this: the first call writes the prompt to the cache, and the second and third calls read the instruction and the manual from the cache but add a few tokens, because the prefix grows with the new question. In explicit mode, only the first call writes, and the repeated calls write nothing: the question after the marker simply remains an uncached remainder. This pays off when writes cost more than reads and you don't want to pay to cache questions that won't be repeated.

An important detail: in implicit mode, the manual and the question must go in separate messages (system and user). When AWS sent them as a single user message, there were no cache hits.

The OpenCode Coding Agent

Coding agents resend a large, stable prefix at every step: the system prompt, tool descriptions, and history. That is why they naturally fit implicit caching. AWS connected the open-source OpenCode agent (version 1.

18.35) through a custom OpenAI-compatible provider pointed at the same endpoint. In a three-turn session, the first request processes the entire prompt and writes it to the cache, and the following ones read everything up to the end of the previous request and process only the new tail.

No caching configuration was needed. The limitation: this version of OpenCode does not record cache_write_tokens for such a provider, so the cache.write field stays at zero.

The charts in the article are marked as illustrative, and AWS does not give specific savings percentages.

What to Do in Practice

  • Put static content at the start of the prompt: the system instruction, tools, and documents. Send timestamps and request IDs to the end.
  • Verify that the prefix is long enough: each model has a minimum token threshold.
  • Check the usage fields in each response, and for overall workload, check the CacheReadInputTokenCount and CacheWriteInputTokenCount metrics.
  • Keep the cache warm: the time to live (TTL) resets on each hit, and after a pause longer than the TTL, the prefix must be written again.
  • Cache only what is actually repeated. With cross-region inference during peak hours, additional cache writes are possible.

Support for modes, minimum prefix length, the number of markers, and the TTL depend on the model and region, so check the model card first. According to AWS, prompts and responses are not passed to the model provider and are not used for training.

Story

Key Questions Answered

How It Works?

The principle is simple. If a new request starts with the same text as the previous one, Bedrock takes the beginning from the cache and processes only what comes after it. Cache reads are billed below the regular input token price. For some models, writing to the cache costs more than standard input, so you should check the exact rates in the pricing for the specific model. The longer the repeated context and the more often it is used, the greater the savings. There are two modes.

What are the key details on Example: A Water Filter Support Assistant?

AWS demonstrates this with a hypothetical assistant for the AquaPure 500 filter. The system prompt contains an instruction and a user manual of about 4,000 tokens, and each request asks a new customer question. If the model has a minimum prefix length, the manual must be padded up to it. For the test, placeholder text is fine, since the cache only cares about prefix matching. The client is a standard OpenAI SDK pointed at the Bedrock endpoint.

What are the key details on The OpenCode Coding Agent?

Coding agents resend a large, stable prefix at every step: the system prompt, tool descriptions, and history. That is why they naturally fit implicit caching. AWS connected the open-source OpenCode agent (version 1.18.35) through a custom OpenAI-compatible provider pointed at the same endpoint. In a three-turn session, the first request processes the entire prompt and writes it to the cache, and the following ones read everything up to the end of the previous request and process only the new tail.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…