ArisGlobal cut its AI bill by 40% by caching the shared prompt instead of the PDF
ArisGlobal's AI parses drug side-effect reports, and the shared prompt for pharma companies grew from 72 to 180 thousand tokens. The team first cached the 10–200-page PDF, but that document is unique to every case. Now the shared rules come first, 65% of input tokens are served from cache, and the inference bill has dropped by about 40%. The takeaway: cache what repeats, not what is biggest.
AI-processed from AWS Machine Learning Blog; edited by Hamidun News
If you build an AI product for several enterprise customers, you know the problem: the system prompt keeps growing. Each customer brings its own rules and examples, and the model re-reads them on every request, and you pay for it every time. ArisGlobal, a software vendor for the pharmaceutical industry, described on the AWS blog how it cut its monthly inference bill (the cost of running the model) by about 40%.
It did not shrink the prompt. It changed the order of the prompt's parts. Amazon Bedrock can cache the beginning of a request: if the next request starts with the same text, that chunk is read from the cache at a reduced price.
Previously, ArisGlobal cached the PDF document, which changes with every case, and paid full price for the shared instructions. Now 65% of input tokens come from the cache, and the effective input price has fallen by about 50%.
Where the growing bill came from
ArisGlobal's AI engine NavaX runs on Amazon Bedrock with Claude models. It processes reports of adverse drug reactions: multi-page PDFs of 10–200 pages. From each one, an agent extracts about 130 fields: patient data, the suspected drug, the timeline of events, seriousness criteria, and regulatory classification. It does this through a series of requests, each responsible for its own section of the form.
Four blocks repeat in every request: system instructions, the specific pharma company's rules, the extraction schema with examples, and the document itself. While there was only one customer, the prompt stayed compact. Then new customers arrived, each with its own requirements: for example, how to record the onset of a reaction, which format the EU, the FDA, or the Japanese regulator requires, and how to assess the seriousness of a case. Over 6 months, the shared part of the prompt grew from 72 to 180 thousand tokens, an increase of 150%. The cost of processing a single case rose by more than 200%, because this shared part was reprocessed in every request.
What changed
Caching in Bedrock works on a prefix basis: it caches a continuous chunk from the start of the request to a checkpoint. Everything after the checkpoint is billed at full price. The team first cached the largest block, the document, and put it first. Within a single case, that worked: the first request wrote the PDF to the cache, and the rest read it cheaply. But the next case brought a different PDF, and the cache started over. Meanwhile, the bloated shared prompt sat after the checkpoint and was billed in full on every request.
The new scheme reorders the blocks. The shared prefix comes first: system rules, schema, examples. Then comes the checkpoint, and only after it the document and the question about a specific section. The same checkpoint now produces a cache that every request, every case, and every customer using this agent can read. The article's simple rule: cache what repeats, not what is bigger. A 200-page document is large, but it is barely reused.
By default, the cache lives for 5 minutes, and each hit extends its lifetime. In active accounts, a new case arrives roughly once every 5 minutes per customer, and there are several customers, so the shared prefix stays "warm" during working hours. For some models, you can choose a 1-hour TTL.
Numbers and caveats
According to ArisGlobal's figures, provided to AWS, at a volume of more than 19 billion tokens per month the results were as follows:
- 65% of input tokens in a typical request come from the cache;
- the effective input price is about 50% lower;
- the total monthly inference bill is about 40% lower;
- latency of cached requests dropped by up to 25%;
- about 15% more enterprise customers were onboarded to the platform without a proportional rise in costs.
Routing within the EU via cross-region inference remained unchanged. In CloudWatch, cache reads during peak load are roughly 10 times higher than cache writes.
There are caveats. These are the company's own figures, and AWS states plainly that the result depends on prompt structure, traffic, and cache hit rate. The team gave up reusing the document within a case, so the PDF is now processed at full price in every request. A second checkpoint after the document is not needed: such an entry is unique to a single case. For Claude Sonnet 4.5, the minimum cacheable prefix size is 1 024 tokens, and a request can have up to four checkpoints.
What to do in practice
Put everything stable at the start of the request: tool descriptions, system instructions, schemas, and examples. Leave customer data, the document, and the question after the checkpoint. Make sure the prefix matches byte for byte: a timestamp, a random ID, or a different tool order will cause a cache miss. Finally, watch cache reads and writes in CloudWatch. If your hit rate falls steadily below your usual level (for ArisGlobal, that is 65%), something has broken the prefix.
For teams running multi-customer agents, the conclusion is straightforward: before you trim the prompt, check what sits in front of the cache checkpoint. Reordering blocks does not require rewriting the pipeline, and at high volumes it pays off immediately.
- Nov 18, 2026AI agent left SOC 27 minutes to respond versus 53 for a junior attacker
- Nov 18, 2026Garry Tan Deletes the /investigate Skill: 50.0% vs. 47.2% Success Rate, and Fixes Cost 43% More
- Nov 18, 2026After a Ban, the Author Launched an Agent Team in Kimi, Codex and OpenCode: Runs from $0.30
- Nov 18, 2026TokenEconomizer plugin cuts AI coding agent costs by 18.4–30.9% without changing the model
Key Questions Answered
Where the growing bill came from?
ArisGlobal's AI engine NavaX runs on Amazon Bedrock with Claude models. It processes reports of adverse drug reactions: multi-page PDFs of 10–200 pages. From each one, an agent extracts about 130 fields: patient data, the suspected drug, the timeline of events, seriousness criteria, and regulatory classification. It does this through a series of requests, each responsible for its own section of the form.
What changed?
Caching in Bedrock works on a prefix basis: it caches a continuous chunk from the start of the request to a checkpoint. Everything after the checkpoint is billed at full price. The team first cached the largest block, the document, and put it first. Within a single case, that worked: the first request wrote the PDF to the cache, and the rest read it cheaply. But the next case brought a different PDF, and the cache started over.
What are the key details on Numbers and caveats?
According to ArisGlobal's figures, provided to AWS, at a volume of more than 19 billion tokens per month the results were as follows: 65% of input tokens in a typical request come from the cache; the effective input price is about 50% lower; the total monthly inference bill is about 40% lower; latency of cached requests dropped by up to 25%; about 15% more enterprise customers were onboarded to the platform without a proportional rise in costs. Routing within the EU via cross-region inference remained unchanged.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.