Habr explains why language models and classic RAG lose track of connections

Habr has published an analysis of why the euphoria around large language models and RAG is running into architectural limits. The main idea is simple: a model can work confidently with documents, but that does not mean it truly understands knowledge and the links between facts.

Why RAG took off

RAG quickly became the standard way to "connect" a large language model to corporate data. The logic is clear: the model itself is good at phrasing answers, summarizing complex texts and keeping a consistent style, but without external memory it is limited to what it was trained on in advance.

Add document search, and the system starts to look like a universal analyst: it answers questions about internal policies, summarizes contracts, assembles reports and helps find the right fragments without retraining the model.

On small data sets this approach is genuinely impressive. If the knowledge base consists of dozens of files and the questions are fairly direct, classic RAG pulls relevant chunks of text almost flawlessly, passes them into the model's context and gets a coherent answer. That is why the approach caught on quickly in support, internal assistants, legal services, educational products and analytics: it is relatively simple to deploy, and the practical payoff comes fast.

Where it breaks down

The problem is that RAG, by its nature, remains a search layer on top of a model rather than a full knowledge system. It can find similar fragments of text, but it does not guarantee an understanding of causality, hierarchies and hidden links between entities.

When information is scattered across different documents, the answer often cannot be lifted from a single paragraph: it has to be assembled from several facts and intermediate reasoning steps. That comes naturally to a person, but for classic RAG it is already an edge case.

• the system returns the most similar fragments, not necessarily the most important ones;
• as the knowledge base grows, a critical piece of text can easily miss the context;
• a large context window does not solve the problem of selecting and ordering data;
• the model can still mix up sources and generalize too boldly.

As a result, a model may honestly say there is not enough data, even though the necessary chain of facts is already in the documents.

The article gives a simple example: if one text links Alice to Bob, and another says Bob studied paintings by Leonardo da Vinci, a person can build the intermediate connection. A search system based on text similarity often looks for direct confirmation and does not take that step on its own. This is the gap between "searching for similar content" and actually working with knowledge.

Why ontologies are needed

The author leads to the idea that the next stage for enterprise AI systems is a more explicit representation of knowledge. When you count documents in the hundreds of thousands or millions, storing meaning as a set of chunks and vector embeddings becomes unwieldy. What is needed is a structure in which entities, their properties and their relationships are defined explicitly, rather than reconstructed on the fly from scraps of text every time. Otherwise the system depends on search luck and on how well the query is phrased.

This is where ontologies become relevant again, a topic that long seemed too academic for applied AI. In the article's logic, this is not an attempt to abandon language models or RAG, but a way to build the next layer on top of them. The model is still needed for natural-language communication, but the knowledge base itself should describe the world not only through text, but through relationships. This approach is harder to implement, but it is better suited to tasks where dependencies, causality, overlaps between objects and long chains of inference matter.

What this means

The RAG boom is not going away, but the market is gradually hitting its ceiling. If an AI system must do more than search for paragraphs, and has to explain links between facts and draw stable conclusions over large data sets, a vector database alone is no longer enough. The next wave will belong to solutions that combine language models with stricter knowledge structures.