AWS publishes a Bedrock data lake template that shows every prompt, token and its cost

It's a familiar story: the Bedrock bill arrives at the end of the month, the total has grown, and nobody can answer "who and why." You can see the aggregate, but not what's inside the calls. You can't tell which prompt was sent, what the model returned, or how many tokens it consumed. AWS offers a ready-made solution: a central data lake for all Bedrock invocations across an organization. The CloudFormation templates, Lambda code and SQL queries are in the AWS Sample Bedrock Invocation Data Lake repository.

What you get out of the box and what's missing

Bedrock has model invocation logging. It writes a full trail of requests and responses to S3 or CloudWatch. But you have to enable it separately in each account, and the data stays in that account. With ten accounts, you end up with ten isolated islands. Comparing teams, finding a wasteful prompt or tracing a spike in spending back to its source is hard.

How the pipeline works

Each call passes through five stages:

• Bedrock in the source account writes the log to a CloudWatch log group.
• A subscription filter acts as a standing forwarding rule: every new event goes to a CloudWatch Logs destination in the central account. The source accounts get no permissions on the central pipeline; they only send events.
• A small Lambda unpacks the CloudWatch batches, which arrive compressed and encoded. It drops service noise such as the "Permissions are correctly set" message and lifts the nested field with the caller's identity to the top level.
• Amazon Data Firehose buffers the records, converts JSON to Parquet and writes it to S3 partitioned by date (year/month/day).
• A Glue table tells Athena how to read these files, and from there you work with ordinary SQL.

The result is a single table where every call has its prompt, response, input and output token counts, model, stop reason, account, timestamp and IAM role. This moves you from "we spent X tokens" to "here is what was in each call."

What to do with it in practice

Here are several scenarios from the post:

• Debugging. Pull the exact prompt and response of a specific call without digging through raw logs.
• Efficiency. Compare input and output tokens by team and model to find where tokens are wasted.
• Prompt engineering. Look for responses with stop_reason = max_tokens: they were truncated, so the prompt or config likely needs work.
• Cost. The table joins with the AWS Cost and Usage Report 2.0 on IAM ARN, which gives you spending by team and day, as well as the price of the most inefficient calls.

For visualization, there is a Quick Sight dashboard, installed with the command cid-cmd deploy. You can add organization metadata to it, such as team names.

The authors tie all this to tokenomics. According to them, it was declared a separate FinOps domain at the FinOps Foundation conference in 2026. The logic is simple: you can't optimize tokens until you measure them. The lake provides exactly that foundation.

Limitations and pitfalls

The main risk is security. You are putting the full prompts and responses of all teams into one bucket, and they may contain PII. The bucket is encrypted with KMS, but you have to restrict access to it and to Athena results with IAM policies yourself.

There are also operational caveats:

• Bedrock logging is an account-level setting, not a CloudFormation resource. You have to enable it with a separate API call in each source account, and only with delivery to CloudWatch.
• Data doesn't appear instantly: Firehose flushes its buffer after about a minute, sometimes longer. That's fine for investigations, but not for real-time alerts.
• The Athena view is created manually: you have to run the saved query once.
• The cost join runs on IAM ARN. If all teams call Bedrock under one role, the per-team breakdown will be approximate. This is our conclusion; the post doesn't discuss it directly.
• When tearing down, empty the buckets first, and delete the Athena workgroup manually.

Takeaway

If you run Bedrock across several accounts and can't say which team burns the most tokens, the template is worth deploying. The authors' advice: don't try to cover everything at once. Start with one or two accounts, get comfortable with queries against the table, and only then expand to the whole organization. While you're at it, check the IAM policies and the bucket while there is still little data in it.