VTB explains why AI pilots stall before production — and how architecture can fix it

At the Data Fusion conference on April 8–9, VTB publicly acknowledged a problem nearly every enterprise AI customer recognizes: pilots look convincing, but few make it into real production. The analysis focuses not on the quality of any single model but on the architecture of the deployment itself.

Why pilots break

The core idea is simple. A pilot usually tests one step under controlled conditions, while production introduces an entire chain of actions in which errors accumulate. If eight links each run at 85% accuracy, the reliability of the whole chain falls to 27%. In a presentation, such a system still looks "almost good," but in a real process three out of four results are wrong — and, most dangerously, there is no way to know in advance which ones. That is why the problem shows up not as a one-off bug but as a systemic collapse in quality at scale.

This leads to a more uncomfortable conclusion: the market often optimizes AI for autonomy rather than accuracy. A metric like "percentage of tasks completed without a human" is convenient for marketing and reporting, but it says little about how well the system stays grounded in reality over the long run. The article links this to automation bias and deskilling: people increasingly trust wrong suggestions while losing the ability to make decisions without the machine. The company ends up with not only a fragile pipeline but also a gradual erosion of its own expertise.

A human-centered architecture

Instead of full autonomy, the author proposes a low-entropy design in which the human is a mandatory part of the system, not an emergency stop button. It splits the work into four levels, from a field operator next to the physical asset to a domain expert who reviews the model's recommendations and feeds corrections back into training. The idea is to "reset" uncertainty at every level rather than let it creep up the chain unchecked.

• Level 0 — an on-site operator or specialist who sees the real object and validates the input data.
• Level 1 — narrow models for specific signals: temperature, humidity, defects, images, or other physical parameters.
• Level 2 — a coordinator that aggregates the models' outputs, reasons over them, and produces a recommendation for a human.
• Level 3 — a domain expert who confirms or corrects the conclusion, thereby giving the system a training signal.

In this setup, AI's job is not to replace the specialist but to extend their reach and productivity. The author cites a digital twin of a forest ecosystem covering more than 180 thousand hectares: as coverage grew from 2 to 50 thousand hectares, capital expenditures rose 2.1x and operating expenses 2.2x, while the team grew only from four to eight people. A traditional approach, by the author's estimate, would have required many times more field staff.

Why an API isn't enough

A separate argument concerns the stack. The article claims such a design is hard to build on top of public APIs from large models alone, because domain expertise has to live not in prompts, and not only in RAG, but in the weights of a locally controlled model. The proposed solution is LoRA or QLoRA adapters fine-tuned on verified pairs of expert answers and preferences. After the workday, humans validate the logs, fine-tuning runs overnight, and by morning the system comes online with updated domain knowledge.

"A prompt is forgotten at the end of the context window.

An adapter never is."

This approach bets on in-house infrastructure. The article's hardware estimate is roughly 900 thousand to 1.2 million rubles: a server with an RTX 4090 or 5090 for the coordinator and overnight training, several Raspberry Pi boards for the narrow on-site models, and separate log storage. The main argument is not that cloud models are useless, but that they are better used as an external research tool than as the decision-making layer in a critical production loop.

What it means

For the market, this marks an important shift: the question is no longer how many people can be removed from a process, but how to maintain quality as AI scales. If this thinking takes hold, enterprise deployments will increasingly be built around local models, continuous verification, and human-machine loops rather than promises of full autonomy.