Hallucinations by the Book: Why You Shouldn't Trust ChatGPT With Construction

Imagine building a house on the advice of the best-read person in the world, who nonetheless has a mild case of amnesia and a tendency to make things up. That is the situation Aleksey Krivonosov, owner of a construction business, found himself in after deciding to hand routine work to ChatGPT. At first everything went perfectly. The model cheerfully wrote scripts for the company's YouTube channel, drew up content plans and polished technical reports. It is a classic trap: when AI excels at creative tasks, it creates the illusion that it is just as good at the exact sciences. But the devil, as usual, was in the building codes and regulations.

The problem surfaced when ChatGPT was trusted with regulatory documentation: SP codes and GOST standards. For those far from construction, these are not boring books but a rigid set of rules in which every figure is paid for with someone's safety. The model began behaving like a careless student at an exam: if it did not know the exact answer, it invented one. It did so with such confidence that the catch went unnoticed at first. The algorithm generated nonexistent clauses and produced figures that had never appeared in the official documents. In an industry where an error in calculating the load on a beam can lead to disaster, such "creativity" is unacceptable.

Why does this happen? ChatGPT is a language model trained to predict the next word, not to check facts against a database. It works with probabilities, not truth. When you ask it to find a specific clause in a GOST, it does not "go to the library"; it constructs an answer that sounds as plausible as possible. That is the insidious thing about hallucinations: they look like truth. For marketing this is not critical, but for engineering it is a death sentence. Aleksey realized that using a "bare" LLM in professional work is like playing roulette with a loaded revolver.

Instead of giving up on the technology, Aleksey's team set out to build a specialized tool. Within six months they developed "Digital Standard". The main difference from an ordinary chatbot is the use of RAG (Retrieval-Augmented Generation). The idea is simple: the model is not allowed to "recall" information from its own memory. Instead, the system is forced to look for the answer in a strictly limited, vectorized database of real construction regulations. If the information is not in the database, the system says so rather than drifting into fantasy. That turns the AI from a storyteller into a high-speed librarian.

Krivonosov's case highlights an important tectonic shift in the industry. The era of fascination with universal models is passing. Businesses are starting to understand that real tasks call for vertical solutions. It is not enough to simply "plug in the OpenAI API". You have to process data by hand, clean the junk out of it and set up strict output filters. Only then does a neural network turn from a toy into a working tool. Today we see similar systems appearing in law, medicine and now construction. It is a natural stage in the technology's maturing.

What does this mean for the market? First, demand for "prompt engineers" is falling, giving way to demand for data architects who can make LLMs work with corporate knowledge. Second, trust in open models in critical industries will only decline. We are entering the era of "trusted AI", where accuracy is valued above eloquence. Aleksey's experience shows that for AI to be useful, you first have to take away its right to be creative wherever numbers and laws rule.

The takeaway: general-purpose neural networks like ChatGPT have hit a ceiling in professional tasks. The future belongs to RAG systems and narrowly specialized knowledge bases. Are you ready to entrust the foundation calculations for your house to an algorithm that cannot tell truth from probability?