Habr AI explains why LLMs don't calculate, don't learn in conversation and depend on tools

An article on Habr AI tackles the popular myth that a modern chatbot is already a universal intelligence on its own. The author's main thesis is simple: a base LLM, by its nature, can only work with text. It takes a text prompt and generates a text response. Everything else that users perceive as the model's "magic abilities" is usually provided by external tools, integrations and orchestration. That is why the same interfaces can draw pictures, search the internet and calculate numbers, even though the language model itself does not thereby become an artist, a search engine or a calculator.

The first misconception is the sense that an LLM "can do everything." If you ask it to create an image, it composes a request for a separate generation model. If you talk to it by voice, speech recognition and voice synthesis join the chain.

If you need an exact calculation, a reliable result most often appears only after a call to a code interpreter or another computing tool. Without such add-ons, an LLM relies on probabilistic reproduction of patterns from training: it may solve a simple example correctly, but easily makes mistakes with long numbers, formulas and tasks requiring high precision. This points to an important practical boundary: the model's strength is not math as such, but describing a task in text and choosing the right tool.

The second myth is that a model learns during a conversation. The author reminds readers that inference and training are two different processes. When a user writes a prompt, the model generates tokens one after another based on weights that are already fixed, and those weights do not change at that moment.

So a given LLM in a given session does not "memorize the lesson" and does not get smarter from a user's correction. Providers may later use anonymized conversations to train future versions, but that is a separate fine-tuning cycle, not magical self-updating in the chat. Another conclusion follows: memory about a user between conversations is usually not model training but saved context that is mixed back into the prompt.

The article then briefly explains what a public LLM consists of. At its core is a transformer, which sees the entire available context at once and builds a response as a sequence of probable tokens, keeping the text coherent through learned patterns. On top of that comes RLHF, which tunes the model for the assistant format, politeness, instruction following and safety restrictions.

But RLHF does not turn the model into a logic machine and does not fix its fundamental weaknesses. That is why language models are good at text analysis, summarization, style changes, step-by-step instructions, working with formats such as JSON, and choosing tools. They are weak at exact calculation, processing large tables, holding huge volumes of data in context, and knowing the current state of the world after the training cutoff date.

Add to this the probabilistic nature of responses, sensitivity to prompt wording and the risk of hallucinations.

To make an LLM useful in production, teams build an additional layer around it. For static knowledge they use RAG: documents are split into fragments, semantically similar chunks are retrieved for a query, and the model receives only the relevant context. For dynamic data and actions they use function calling: the LLM decides when to call an API, a database, a calculator or a simulation, while the orchestrator validates the calls, adds the tools' responses to the history and manages the whole loop. The same orchestrator handles dialogue memory, system prompts, output format checks and the launch of subagents.

Louder concepts emerge on this foundation: AI agents, digital employees, copilots and digital twins. In essence, this is not separate magic but combinations of LLMs, knowledge bases, APIs, automation and classical computing engines. It follows that discussing "artificial intelligence" without distinguishing between technologies is no longer enough. If a business needs exact calculation, strict automation or a forecast from structured data, an LLM alone is not enough. If it needs work with emails, documents, instructions, knowledge search and a conversational interface to a complex system, an LLM really does provide a strong boost.

The sober view in the article is useful precisely because it removes excess expectations: you shouldn't attribute superpowers to a language model, but you shouldn't underestimate it as an interface and coordinator of other tools either.