Habr AI shows how to add memory and context to a Python LLM chat with Ollama and LiteLLM

A simple console chat with an LLM stops being a real chat if the model sees only the current question. In the new part of the practical guide to Python, Ollama and LiteLLM, the author shows the most important step after the basic integration: how to add message history and turn a chain of separate requests into a coherent dialogue with context.

The problem with the previous version was that on every turn the program sent the model only the system instruction and the user's latest message. For a person this seems an almost invisible limitation, until dependent questions appear, such as "make it shorter" or "add practice to it." Without the earlier messages, the model does not know what the pronouns and clarifications refer to, so it may answer at random or lose the thread of the conversation. This is exactly what separates a one-off model call from a conversational interface.

The walkthrough stresses a point that discussions of LLMs often oversimplify: the model has no hidden long-term memory between calls. In such a chat, memory is created not by Ollama or LiteLLM themselves but by the application, which stores the conversation and sends it to the model again with every new request. The tutorial uses an ordinary Python list, `conversation_history`, into which messages with the roles user and assistant are appended in turn. The system prompt is not stored in the history; it is added separately each time a request is built.

Architecturally the change looks small, but it alters the program's behavior radically. The request function now accepts not only the current `user_message` but also the history. It then builds the `messages` list in a strict order: system, then all past turns, then the user's new question. After the model replies, the application saves both sides of the exchange, the question and the answer. This is not a cosmetic detail: if only the user's messages are recorded, the next request will be incomplete, because the model will see that a question was asked but not how it answered.

History limits are covered separately. The example introduces the constant `MAX_HISTORY_MESSAGES = 6`, and the helper function `trim_history` keeps only the last six messages, that is, the last three exchanges. For a local prototype this is a practical compromise: the history does not grow endlessly, requests do not bloat, and the model still gets the most recent context. It is a good illustration that in LLM applications memory always has to balance answer quality, latency, and cost or load, even with a local model.

The article uses a local setup based on Ollama, LiteLLM and the `qwen2.5:3b` model. Such a stack suits learning because it lets you assemble a working chat without an external API and without complex infrastructure. The examples show clearly how the system's response changes once history is added: the request "tell me more about the second one" now correctly refers to the previous list, and the phrase "make it shorter" is no longer treated as a separate, abstract command. In other words, the model does not get smarter on its own, but the application gives it the context that was previously simply lost.

The limitations of this version are also stated plainly. The entire history lives only in the memory of the current run: if you close the script, the dialogue disappears. That is enough for a teaching CLI, but a Telegram bot, a web service or a customer-facing assistant would need persistent storage, at least in a file, SQLite or a full database. Next steps usually also include summarizing long dialogues, filtering out irrelevant messages and more flexible management of the context window.

The main takeaway from this part is very down-to-earth, and therefore useful: memory in an LLM chat is not a separate magic feature of the model but ordinary responsibility of the code around it. A few lines with `conversation_history`, the right order of `messages` and a simple cap on history size already turn a demo script into the basis for a more realistic assistant. For developers, it is a good example of how the value of a chat interface often comes not from the choice of the model itself but from how carefully the application assembles and passes along the conversation's context.