Agent Memory
Agent memory refers to the mechanisms an AI agent uses to store, retrieve, and update information across steps, sessions, or tasks — spanning in-context working memory, external vector databases, key-value stores, and structured knowledge representations.
Agent memory encompasses all the ways an AI agent retains and accesses information beyond what is encoded in its static model weights. Because transformer-based models have a fixed context window and no persistent state between independent inference calls, building useful multi-step or multi-session agents requires explicit memory systems that the model can read from and write to at runtime.
Memory systems are typically organized into four categories. In-context (working) memory is the information present in the active prompt — the most immediate and reliable form, but bounded by the context window size, which ranged from 128K to several million tokens across leading models by 2026. Episodic (external) memory stores past interactions or retrieved documents in a vector database such as Pinecone, Chroma, or pgvector, enabling semantic similarity search over histories too long to fit in-context. Semantic memory holds structured factual knowledge — often in key-value stores or knowledge graphs — that the agent queries by key or relation. Procedural memory captures learned or hard-coded behaviors and is typically encoded in system prompts or fine-tuned into model weights rather than maintained as a separate runtime store.
Effective memory management is essential for agents operating over long horizons: a software engineering agent maintaining state across multiple coding sessions, a personal assistant recalling past preferences and prior commitments, or a research agent avoiding duplicate retrieval work. Poor memory handling is a frequent cause of agent failures — forgetting earlier instructions, repeating completed steps, or retrieving irrelevant context that crowds out useful information within a limited context window.
Research as of 2026 focuses on memory compression (summarizing old episodes before context overflow), retrieval precision (improving embedding quality and re-ranking for external stores), and update policies (deciding what to store, overwrite, or discard). Frameworks such as Letta — formerly MemGPT — have demonstrated architectures where the agent itself manages paging between fast in-context storage and slower external storage, drawing a direct analogy to operating system virtual memory management.