Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarAWS Machine Learning Blog→ original

AWS shows how to give developers Claude with a personal key and a $250 limit per 7 days

If the whole team uses Claude through Amazon Bedrock, finance teams can't see who spent how much. AWS recommends placing LiteLLM, a proxy with an OpenAI-compatible interface, between the code editor and Bedrock, so each developer gets their own key. In the example, each key has a $250 limit over a rolling 7-day window. This makes AI assistant spending visible per person, not as a single line on the bill.

AI-processed from AWS Machine Learning Blog; edited by Hamidun News
AWS shows how to give developers Claude with a personal key and a $250 limit per 7 days
Source: AWS Machine Learning Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceAWS Machine Learning Blog ↗
Editor-in-Chief
Key metric #1Developers issue their own keys in a portal with corporate SSO
Key metric #2Self-service key issuance works only in the paid LiteLLM Enterprise edition
Key metric #3The free version of LiteLLM is meant for a pilot of up to five people

Imagine a company where developers have connected Claude Code, Continue, or Cursor to Claude models through Amazon Bedrock, AWS's cloud service with ready-made language models. A month later, a single shared bill for tokens arrives, and nobody can say which team or which person ran up the usage. Platform engineers need to see latency and errors, finance needs to split costs by project, and security needs uniform access rules. AWS has published a guide on how to cover all three tasks with one gateway: an intermediate server between developer tools and Bedrock. The example with a $250 per-key limit over a rolling 7-day window is illustrative in the article; the figures in the config can be changed.

How the gateway works

The foundation is the LiteLLM proxy, which runs as a container on AWS Fargate (a serverless container service) and presents itself as the OpenAI API. Developers enter one address and a personal key in their tool. They never talk to Bedrock directly. LiteLLM converts the model name in the settings, such as claude-sonnet-4-20250514, into a reference to the required profile in Bedrock and calls it using the Fargate task's IAM role.

Around the proxy sits a familiar set of services:

  • PostgreSQL on Amazon RDS stores keys and spending records.
  • ElastiCache for Valkey caches responses and coordinates request limits across multiple gateway instances.
  • Secrets Manager stores the master key and the database password; they must not be written into config.yaml.
  • Langfuse, an open-source tool, shows prompts and latency and helps analyze failures.
  • CloudWatch collects requests-per-minute (RPM) and tokens-per-minute (TPM) metrics and signals when a quota is approaching.

Each model gets a separate Application Inference Profile, that is, a named entry point into Bedrock. It provides an isolated quota per model and fine-grained access permissions. It also separates the LiteLLM config from model versions in Bedrock. As a result, a model can be replaced, a fallback added, or the provider switched, and developers notice no change.

Operators set the spending policy in the portal, and developers issue their own keys within it. The portal can be linked to corporate SSO.

Limits and pitfalls

The budget is set in the config: max_budget: 250 and budget_duration: 7d give $250 per key in a rolling 7-day window. For long modes with extended thinking, the example raises the timeout to 900 seconds, or 15 minutes.

The authors list traps that can easily cost hours:

  • If a model is specified by a raw profile ARN, LiteLLM does not find it in its price table and counts its cost as zero. The key's spending stays at zero, and budgets never trigger. Per-token prices must be entered manually in the model_info block.
  • For a global profile, the copyFrom field needs the ARN of the system profile with the global. prefix, not the ARN of the model itself. Otherwise you get a profile for a single region without cross-region routing.
  • In the IAM policy, specifying the profile alone is not enough. You also need the ARN of the base model in the region and a wildcard pattern for other regions. Otherwise, calls fail with the error "not authorized to perform bedrock:InvokeModel".

AWS recommends global cross-Region inference if the selected models support it and company policy allows processing requests in different regions of the world. In that case, Bedrock itself routes each request to wherever capacity is available, which can be cheaper than the geographic option. If data must stay within one geography, use a geographic profile, for example one limited to US regions.

Restrictions

According to the documentation, the free version of LiteLLM AI Gateway is suitable for a pilot and is designed for five users; beyond that, a license is required. Virtual keys are available in the free version too, but the self-service key issuance described in the article requires LiteLLM Enterprise. Enterprise also includes SSO, audit logs, delegated administrator roles, and support. Bedrock Guardrails, with content filtering and grounding checks on responses, are not covered in the example, but they must be added for production.

The practical takeaway: if a company has more than five developers using AI assistants, the gateway provides cost control and observability without changes to each tool. Before launch, check three things: the token prices in model_info, the IAM permissions, and where prompts end up in Langfuse, since traces contain the text of requests and responses.

Story

Key Questions Answered

How the gateway works?

The foundation is the LiteLLM proxy, which runs as a container on AWS Fargate (a serverless container service) and presents itself as the OpenAI API. Developers enter one address and a personal key in their tool. They never talk to Bedrock directly. LiteLLM converts the model name in the settings, such as claude-sonnet-4-20250514, into a reference to the required profile in Bedrock and calls it using the Fargate task's IAM role.

What are the key details on Limits and pitfalls?

The budget is set in the config: max_budget: 250 and budget_duration: 7d give $250 per key in a rolling 7-day window. For long modes with extended thinking, the example raises the timeout to 900 seconds, or 15 minutes. The authors list traps that can easily cost hours: If a model is specified by a raw profile ARN, LiteLLM does not find it in its price table and counts its cost as zero. The key's spending stays at zero, and budgets never trigger. Per-token prices must be entered manually in the model_info block.

What are the key details on Restrictions?

According to the documentation, the free version of LiteLLM AI Gateway is suitable for a pilot and is designed for five users; beyond that, a license is required. Virtual keys are available in the free version too, but the self-service key issuance described in the article requires LiteLLM Enterprise. Enterprise also includes SSO, audit logs, delegated administrator roles, and support.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…