TriAgent: комитет мелких моделей вместо дорогого LLM экономит $9,3 млн в год

How the TriAgent committee is built

TriAgent distributes queries across three tiers of "vision" into text, each more expensive than the last. Cheap tiers handle most simple cases, while the expensive LLM is engaged only for disputed ones. The idea rests on a structural cost trap: most queries are trivially classified, but the cloud reasoner processes all of them, and the bill grows linearly with the number of users.

• VADER lexicon — word-level analysis
• FinBERT transformer — sentence-level analysis
• Qwen2.5 (from 0.5B to 14B, 4-bit) as the reasoner over multiple sentences, with cross-checks from Mistral-7B and Phi-3.5-mini
• Semantic Divergence Index (SDI) measures how much the tiers diverge and decides who gets the query
• Savings — $9.3 million a year at 10 million users versus GPT-4o-mini

"Most queries are trivially classified, yet expensive cloud reasoners

process all of them" — from the paper's abstract on arXiv.

What the critic plateau is

The paper's main finding is the "critic plateau": when LLMs are reassigned the role of critic over the outputs of smaller agents, the F1 metric plateaus at around 0.87 for Qwen models from 1.5B to 7B (confidence intervals overlap). By comparison, voting among three "personas" of the same size yields only F1=0.66.

According to the authors, the difference is explained by the diversity created by different granularities of analysis: the three tiers "see" the text differently, and that turns out to be more useful than three identical models with different roles.

Three side findings from SDI

From the same SDI signal, the authors derive three further findings. A shared consensus vocabulary on a multilingual sentence-BERT answers 95% of Chinese-language queries from an English-language cache at F1=0.99 — meaning it transfers labeling between languages at almost no extra cost.

SDI also works as a post-hoc LLM hallucination detector with AUC=0.90. And in a backtest on 20 tickers, an SDI-based strategy delivered better risk-adjusted returns: Sharpe 3.50 versus 1.36 for the "always FinBERT" strategy and 0.11 for "always LLM."

What this means

TriAgent shows a practical recipe: don't run every query through an expensive model — route by difficulty instead. For production systems with millions of users, that's the difference between a linearly growing bill and savings in the millions of dollars — without losing quality. The authors have released the code, the lexicons, and the consensus vocabulary in open access.

Frequently asked questions

How much does TriAgent save?

According to the authors' calculations, at 10 million users the system saves $9.3 million a year compared to a GPT-4o-mini baseline — because the expensive LLM is engaged only for disputed queries.

What is the SDI index?

The Semantic Divergence Index (SDI) is a metric of pairwise divergence between the three analysis tiers (word, sentence, multiple sentences). It routes queries and also serves as an LLM hallucination detector with AUC=0.90 accuracy.

Which models does TriAgent use?

The VADER lexicon, the FinBERT transformer, and the Qwen2.5 reasoner (0.5B–14B in 4-bit), with cross-checks from Mistral-7B and Phi-3.5-mini.