Fintech team launched an AI chatbot: concepts, pitfalls, and lessons from implementing GenAI

Fintech team openly described the journey of implementing a generative AI chatbot — from the first experiments with prompts to a live production launch. It turned out that the gap between how GenAI works in a demo and what it requires in a real financial product is enormous.

Financial AI is a special case

Generative models handle broad, open-ended questions well. But the fintech domain imposes fundamentally different requirements: data accuracy, up-to-date information, and strict regulatory caution.

If a language model makes a mistake in a recipe, that is unpleasant. If it makes a mistake in describing a banking product or loan terms, that is legal risk and a loss of customer trust.

From the very first steps, the team ran into a key question: how do you prevent “hallucinations” where every word carries financial responsibility? The answer turned out to be non-trivial and required several architecture iterations before a working prototype emerged.

Concepts that changed the approach

Before writing the first line of code, the team had to make sense of several fundamental concepts that are poorly described in most tutorials:

• RAG (Retrieval-Augmented Generation) — instead of answering “from memory,” the model retrieves facts from an external knowledge base. Without RAG, a financial bot either makes up data or replies “I don’t know” to most specific questions.
• Prompt engineering — system instructions that define the tone, constraints, and response format. In finance, this is not optional but a mandatory layer of control over system behavior.
• Guardrails — protective mechanisms: a ban on specific investment advice, a requirement to cite sources, and a refusal to provide exact figures without verification.
• Quality evaluation — how do you measure whether the bot is answering correctly? Automatic metrics are often misleading, while manual review of every answer does not scale.
• Latency vs. quality — a smarter model responds more slowly. In a mobile app, 7–8 seconds of waiting is already a serious UX and conversion problem.

Where unexpected difficulties emerged

The most painful challenges turned out to be not technical, but organizational and conceptual.

Boundaries of responsibility. From the outset, the legal department took a clear position: the bot should inform, not advise. This significantly narrowed the range of acceptable answers and required the team to rethink key user scenarios. Some planned features had to be removed.

Knowledge base. It would seem that a fintech company has plenty of structured documents — pricing, terms, FAQ. But when assembling the RAG system, it turned out that the formats were incompatible, some information was outdated, and the data was scattered across several systems with no single registry. Building a high-quality knowledge base took significantly more time than developing the bot itself.

User trust. People are happy to ask the bot neutral questions, but become cautious when the topic turns to finance. Product adoption moved more slowly than the team expected, and additional work on UX and wording was needed.

“This article was written to explain the concepts and difficulties

that the team and I encountered,” the authors write, deliberately avoiding technical details.

What this means

This team’s story is an accurate snapshot of what most companies face during their first serious GenAI implementation in a product. The financial domain amplifies every risk several times over: legal, technical, and user-related. That makes an honest write-up from practitioners all the more valuable — one that captures not only the successes, but also the surprises, the ones they did not expect.