AI in Finam: How to Survive Migrating from GPT-3.5 to a Corporate Pipeline

AI at Finam: how to survive moving from GPT-3.5 to a corporate pipeline

Many still sincerely believe that implementing artificial intelligence in a large company looks like buying a ChatGPT Plus subscription for each department. In reality, Finam faced the fact that individual enthusiasm quickly crashes against harsh corporate reality. When your Flutter prototype works for one developer — that's magic and a technological breakthrough. When you need to distribute the same technology to a thousand employees of a financial organization, the magic turns into an endless series of security problems, request costs, and answer quality. Finam's story is particularly instructive precisely because they weren't afraid to go the path from "toy" to industrial solution, stepping on every possible rake.

It all started with a classic GPT-3.5 experiment. At that moment, it seemed that all you needed to do was give people access to the API, and productivity would skyrocket.

However, the financial sector is not a sandbox. The first serious check showed that uncontrolled use of external neural networks carries risks of confidential data leaks and unpredictable expenses. You can't just send customer data to the OpenAI cloud and hope for the best.

The team had to urgently develop an internal platform that would become a "layer" between the employee and the neural network. This layer took on the functions of access control, logging every request, and strict token limiting, so the AI budget wouldn't run out in the first week of the month.

The next stage was the fight against "hallucinations". Neural networks are excellent storytellers, but in finance, lies cost too much. To make AI tell the truth, Finam implemented RAG (Retrieval-Augmented Generation) technology.

Now the model doesn't just make up an answer from scratch, but first searches for information in the database of internal regulations and company documents, and only then forms an answer based on the facts found. This turned an abstract chatbot into an expert who understands the nuances of the company's internal processes. In parallel with this, experiments began with local models.

Using their own computing power to run Llama or other open-source solutions became the answer to security requirements: the most sensitive data should not leave the internal network at all.

Scaling to 1000+ people forced a complete rethinking of architecture. It turned out that maintaining stable operation of an AI service under high load is a separate engineering discipline. Complex monitoring systems had to be built that track not only server availability but also answer quality in real time. Finam understood: technology is only 20% of success. The other 80% is painstaking work on infrastructure, training employees to formulate requests correctly, and constant adaptation to changing regulatory requirements. The irony is that the smarter AI becomes, the more qualified engineers it requires for support.

Currently, Finam's platform is not just a chat, but a full-fledged hub where both external APIs and local models live, along with specialized agents for different departments. This is a path of compromises between response speed, the cost of one token, and data security. The company clearly demonstrated that the era of "just chatbots" has ended, giving way to the era of corporate AI ecosystems. Those who today try to simply "bolt on an API" will tomorrow face the same scaling challenges that Finam has already overcome. The question is only whether others will have the patience to turn a trendy trend into a working tool.

Key takeaway: Implementing AI in enterprise is 80% building infrastructure and only 20% choosing the model itself. Are you ready to pay for your data security at double the price in the form of engineering hours?