How Just AI Helped a Bank Break Through Automation Ceiling with LLM-Agents

The Just AI team published an analysis of a real case study: how to transition a bank's customer support from NLU-bots to LLM-agents without ending up with a wave of hallucinations instead of automation growth.

The Ceiling You Can't Get Around

When NLU scenarios grow to hundreds of branches, adding new dialogues stops helping. The bot starts making mistakes on edge cases, requires constant markup support, and can't handle non-standard phrasings. The automation percentage stalls.

The bank faced exactly this: a mature NLU system for cashback had hit its limit. The problem wasn't in the quality of configuration — the problem was in the architecture.

Classic NLU-bots operate on rigid rules: they can recognize intents they've been trained on, but struggle with semantic variations and context within long dialogues.

What Changed with LLM-Agents

Just AI proposed a transition to an LLM-agent architecture. Instead of a rigid scenario tree — a language model that understands the meaning of requests, maintains context, and generates answers based on current knowledge bases.

Key changes to the system:

• NLU classifier replaced with LLM understanding — including paraphrases and non-standard phrasings
• Support for multi-turn dialogues with context preservation
• Answers built from the bank's knowledge base, not hardcoded scripts
• A judge-agent introduced that validates each answer before sending
• If an answer isn't confirmed by the source — it's blocked, and the customer is routed to a live agent

Judge-Agent Against Hallucinations

The main risk when implementing LLM in banking support is hallucinations: the model can confidently report incorrect cashback conditions or non-existent rules. For a bank, this isn't just poor UX — it's regulatory and reputational risk.

Just AI solved this through two-level verification. The first agent generates an answer. The second — the judge-agent — verifies it against the original knowledge base. The customer receives either a correct answer or is transferred to a live operator.

"The automation ceiling is not a bug, it's an architectural limit of NLU.

We helped the bank break through it by changing the technology, not by tweaking settings," — Just AI.

What This Means

The transition from NLU-bots to LLM-agents is not an upgrade, it's a paradigm shift. For banks and other regulated companies, this is only possible with built-in quality control: the judge-agent becomes a mandatory architectural component, not an option.