Habr AI explained how memory helps AI agents remember conversations without losing context

Habr AI published a detailed breakdown of how AI agent memory is structured and why it's impossible to build a useful assistant for more than one conversation without it. The material covers the basic mechanics: context window limitations, three types of external memory, and the way an agent combines all this into a single working request to the model.

Why Windows Aren't Enough

The author starts with the most important point: LLMs don't "remember" past sessions on their own. Each new request the model receives comes with the system prompt, chat history, tool results, and additional documents anew. All of this lives inside the context window—a limited amount of text that the model can process in a single call. If irrelevant content gets in, like huge HTML from page parsing, useful details get displaced and answer quality drops.

"What doesn't fit doesn't exist."

Even when the limit isn't formally exceeded, another problem arises—lost in the middle. The model keeps the beginning and end of long context in focus better, while the middle starts to "drift." That's why simply expanding the window doesn't solve the memory problem. The article highlights three basic techniques that reduce overload: summarizing old messages, a sliding window for only recent exchanges, and selective storage of truly important fragments. In practice they're more often combined than used separately.

Three Types of Memory

Beyond the context window lies external memory—files, databases, vector indexes and knowledge graphs that outlive any session. The author divides it into three layers by analogy with human memory. This framework is useful not for terminology's sake, but because each layer has its own storage logic, search, and loading into context. If you mix everything into one heap, the agent will struggle to understand what to always remember and what to fetch only on demand.

• Episodic memory—facts about the user and past interactions: preferences, complaints, habits, successful and unsuccessful agent actions. It's especially needed for personal assistants and support.
• Knowledge base—documents, product reference, domain information, and everything usually called RAG over documents. This memory answers for facts about the world or company, not about a specific person.
• Procedural memory—rules, instructions, and behavior scenarios. These can be chunks of system prompt, markdown files for different tasks, or sets of rules in coding agents.

From this follows an important practical conclusion: agent memory isn't one "magic database," but a set of different-type sources. Episodes are useful to store both in raw form and in compressed, searchable form. Domain knowledge can be kept in a vector DB or graph. Instructions often live in text files and are loaded by situation. Architecture depends less on the tool than on what kind of memory you're saving.

How Memory Is Turned On

An important thought from the article: episodic memory can't simply be "turned on with a checkbox." You have to design it in code. A typical pipeline looks like this: the system saves the dialog, then with a separate LLM call makes a summary of the conversation and extracts long-term facts from it in structured form—for example, JSON with record type, importance, user ID, and date.

After that, each record becomes an embedding and is sent to an appropriate storage. So the agent doesn't drag the entire correspondence into the next session, but returns only relevant conclusions. During a new request, the orchestrator in parallel pulls instructions, domain knowledge, and user memories, then glues them into a single prompt for the model.

At the same time, different types of memory are better kept in different collections or access channels: procedures and user facts can load almost always, while the knowledge base—only after semantic search by meaning. The article separately mentions Mem0, Letta, and Graphiti as off-the-shelf solutions that automate part of this process and hide complexity under the hood.

What This Means

For agent system developers, this material is useful as a minimum terrain map. It reminds us that a working agent is built not around one powerful LLM, but around memory, orchestration, and careful context loading. The sooner these layers are laid into architecture, the fewer hallucinations, detail losses, and repeated errors in real scenarios.