How an AI Agent Is Built: The Wrapper Layers Around an LLM and Where to Look for the Source of a Bug

A developer with an ML background published a breakdown of AI agent internals on Habr in July 2026. Every agent setup has a single LLM at its center, he writes, and an "onion" of wrapper layers grows around it. Most failures happen in those layers, not in the model itself.

What hides behind the word "agent"

An AI agent is not the neural network itself but several layers of code around it. Frameworks change, MCP servers appear, and cloud platforms promise an "agent out of the box," but the core is always the same: a large language model (LLM) that reads the context and proposes the next step. Everything outside it is plumbing that prepares the model's input and processes its output.

The author lists the typical layers used to build an agent on top of a model:

• LLM — the core, which predicts next-token probabilities
• Prompt and context layer — what exactly goes into the model as input
• Orchestration and tool calling — how the model's response turns into an action
• MCP servers — standardized access for the agent to external data and functions
• Frameworks and cloud wrappers — the "agent out of the box" on top of all this

Each layer adds its own logic: one assembles the context, another decides which tool to call, a third connects external data. The model itself performs none of these steps — it only predicts text.

Why the bug is usually not in the model

The source of a failure in an agent usually lies not in the LLM but in one of the outer layers: in how the prompt is built, how the context is passed, or how a tool call is handled. By the author's own wording, the model does exactly one thing — it predicts the next token; everything else is added by the wrappers around it.

"The model itself only predicts next-token probabilities; an onion of wrappers grows around it," — the author of the

Habr article.

That is why, the author notes, understanding the whole construction matters when debugging: if you are not building the agent from a ready-made kit, knowing the layers helps you localize a problem faster. Checking the model first is a common mistake: if an agent's answer looks wrong, it is more often because the wrong context reached the model, or because an adjacent layer parsed its response incorrectly. The model, meanwhile, worked as intended.

Why look from the model outward

The breakdown runs from the core to the periphery — from the LLM to the outer wrappers — because the author came to agents from ML, where each new layer sits on top of one that is already familiar. This order, he says, makes the picture clearer: first what the model does, then what each wrapper above it does. The reverse path — from a ready-made "agent out of the box" inward — hides where exactly the boundary of responsibility for each layer lies.

The occasion for the article is mundane: the notes came out of yet another joint bug hunt with a technical team. The author gathered them into a single overview so that next time it is easier to explain to colleagues what sits where in an agent setup and where to look when something breaks. In essence, it is a map of the layers that is handy to point to during debugging.

What this means

Behind the fashionable word "agent" stands a stack of several layers, and understanding that structure is a practical debugging skill: when an agent misbehaves, the investigation should start not with the LLM but with the wrappers around it, where most errors hide.