AWS proposes a seven-layer architecture for agents: from prototype to production

An agent prototype almost always looks convincing. It answers questions, calls tools and looks great in a demo. Then real traffic arrives, along with the token bill and a review from the security team, and the project gets stuck. According to AWS, the model is rarely to blame. The culprit is the plumbing that every team writes from scratch, usually after launch rather than before it.

The AWS Machine Learning blog has published a breakdown of the Pluggable Agentic AI Framework. It is a reference architecture for multi-agent applications built from seven layers. Each layer has its own area of responsibility and a stable contract with its neighbors. The specific services in the diagram are only an example implementation, not a mandatory choice.

Seven layers, no magic

The first layer is data and knowledge. It accepts a query and returns relevant context for the agent. In the example, this is Bedrock Knowledge Base, OpenSearch Serverless, S3 Vectors and DynamoDB.

The second layer is models and inference. The choice of model should live in configuration, not in agent code. Simple tasks go to a small, fast model; complex reasoning goes to a large one. A proxy such as LiteLLM with a unified interface sits between the agents and the providers. After each new evaluation, you can switch models without touching the application logic.

The third layer is orchestration. For a simple task, a single agent is enough. Complex cross-domain scenarios need a supervisor and narrow specialists, each with its own set of tools. A single universal agent gradually accumulates instructions and tools, its context bloats, latency grows and accuracy drops. Specialists are registered in a single registry, and the supervisor finds and calls them via the standard MCP and A2A protocols. A registry describing each agent's tools, skills and owner is also needed for governance and to avoid writing the same thing twice.

The fourth layer is security and guardrails. All external content is treated as untrusted. Every request is authenticated, whether it comes from a user, another agent or an automated process. Permissions for each tool call are checked before it executes, and instead of permanent secrets, the agent receives temporary, scoped credentials. Personal and medical data is masked in requests and responses. Key decisions are written to an audit log, risky actions require human confirmation, and kill switches stop anomalous behavior.

The fifth layer is the application. It is recommended to keep it thin: a chat, API or widget passes the request to the orchestrator, and that's it. Business logic does not belong here. At the boundary, you can place a cheap, fast classifier model. In a single call, it assesses whether the request is on topic, whether a human is needed and how urgent it is. The model returns probabilities, not text, and ordinary code uses thresholds to decide whether to launch the full agent loop.

The sixth layer is observability. End-to-end traces, cost and tokens per request, time to first token, and dashboards for latency and errors are needed before launch, not after the first incident. The seventh layer is operations: continuous evaluation of every production session for groundedness, accuracy and safety. Plus a regression gate that keeps a release with degraded quality from reaching customers.

What it looks like in an example

The reference implementation is retail customer support: order status and refund eligibility. The supervisor receives a refund request and passes it to two specialists. The orders agent pulls delivery and purchase data from DynamoDB. The returns agent takes loyalty-tier rules from the knowledge base and calculates eligibility based on the delivery date, the current date and the customer's tier. Every call goes through AgentCore Gateway, which checks permissions. Every call is traced, and if the calculation is wrong, the trace shows at which layer and in which decision.

Takeaways

The main idea: architecture first, technology second. If layer boundaries are defined by contracts, changing the model, guardrails or monitoring system doesn't turn into a rewrite of the stack. For companies, it is also a chance to keep tools that already work and add new ones gradually.

The limitations are worth noting too. The article was written by a vendor and is oriented toward the AWS stack: AgentCore, Bedrock, CloudWatch. Component replaceability is claimed, but you will have to verify it on your own infrastructure. The authors themselves caveat that the example is not a substitute for security hardening, testing and review.

The practical takeaway is simple. Even if you don't use AWS, the list of seven layers works as a planning checklist. Do you have an agent registry? Are permissions checked before a tool call or after? Can you prove the agent works as well as it did on launch day? If the answer to even one of these is "no," you've found where your project risks getting stuck between prototype and production.