Habr explained why language models and classic RAG lose their understanding of relationships

Habr published an analysis of why the euphoria around large language models and RAG is beginning to hit architectural limitations. The main idea is simple: a model can confidently work with documents, but that doesn't mean it actually understands knowledge and the connections between facts.

Why RAG took off

RAG quickly became the standard way to "connect" a large language model to corporate data. The logic is clear: the model itself formulates answers well, paraphrases complex texts and maintains style, but without external memory it is limited to what it was trained on beforehand. Add document search, and the system starts to look like a universal analyst: answers by regulations, retells contracts, assembles reports and helps find the necessary fragments without retraining the model.

On small datasets, this approach really makes a strong impression. If the knowledge base consists of dozens of files and questions are fairly straightforward, classical RAG almost flawlessly retrieves relevant text chunks, passes them into the model's context and gets a clear answer. This is why the approach quickly took root in support, internal assistants, legal services, educational products and analytics: it's relatively simple to implement, and practical results appear quickly.

Where it starts to fail

The problem is that RAG by its nature remains a search overlay, not a full-fledged knowledge system. It can find similar text fragments, but does not guarantee understanding of causality, hierarchies and hidden connections between entities. When information is scattered across different documents, the answer often cannot be taken from a single paragraph: it needs to be assembled from several facts and intermediate reasoning steps. For a human this is natural, but for classical RAG it's already a boundary scenario.

• the system returns the most similar fragments, not necessarily the most important ones;
• as the knowledge base grows, critical text chunks easily fail to make it into the context;
• a large context window does not solve the problem of selection and data ordering;
• the model can still mix sources and overgeneralize too boldly.

Because of this, the model can honestly say that there is insufficient data, even though the needed chain of facts is already in the documents. The article provides a simple example: if in one text Alice is connected to Bob, and in another Bob studied Leonardo da Vinci's paintings, a human is able to build an intermediate connection. A search system based on text similarity often looks for direct confirmation and doesn't make this step itself. This shows the gap between "finding similar" and actual work with knowledge.

Why ontologies are needed

The author leads to the idea that the next stage in the evolution of corporate AI systems is more explicit representation of knowledge. When we're talking about hundreds of thousands or millions of documents, storing meaning as a set of chunks and vector representations becomes inconvenient. We need a structure where entities, their properties and connections are defined explicitly, rather than reconstructed each time on the fly from text fragments.

Otherwise the system remains dependent on search luck and query formulation quality. This is where ontologies become relevant again — a topic that for a long time seemed too academic for applied AI. In the logic of the article, this is not an attempt to abandon language models or RAG, but a way to create the next layer on top of them.

The model is still needed for communication in natural language, but the knowledge base itself should describe the world not just through text, but through connections. This approach is more complex to implement, but it is better suited for tasks where dependencies, causality, intersections between objects and long chains of inference matter.

What this means

The RAG boom is not going anywhere, but the market is gradually hitting its ceiling. If an AI system is supposed to not just search for paragraphs, but explain connections between facts and make stable conclusions on large datasets, a vector database alone is no longer enough. The next iteration will belong to those solutions that combine language models with more rigorous knowledge structures.