Alice AI Picks the Model for Each Request: Alice AI LLM Handles 80% of Traffic

Anyone who has built a product on LLMs knows the pain: one big model for all requests is expensive and slow, one small model is fast but gets dumb on hard tasks. In Alice AI, Yandex chose a third path: the assistant looks at the request itself and decides who should handle it.

How it works

The logic is simple. For everyday tasks ("what's the weather," "rewrite this message," "what should I cook") Alice AI uses models that produce answers quickly. If a request has many conditions, requires studying several documents, doing calculations or drawing up a step-by-step plan, "Expert" mode kicks in. You can turn it on by hand, but by default the system makes the decision itself.

Imagine this request: "Compare two rental agreements, take my budget and deadlines into account, and draw up a moving plan." This is no longer small talk: the model has to hold several sources, constraints and the order of steps in mind. Such a request gets more resources, and the user is ready to wait a bit longer. A question about pancake recipes shouldn't wait just as long.

A figure from the announcement: about 80% of requests today are handled by Alice AI LLM, Yandex's own model. The main flow goes down this path, and "Expert" is brought in where a simple answer isn't enough.

Which models are inside

The makeup of "Expert" is not fixed. Yandex says it tests different models every day and plans to update the set regularly. It may include both models from the Alice AI family and publicly available open-weight models running on Yandex servers. For the user this means there is no lock-in to a single model: if an open model appears that handles calculations or long documents better, it can be plugged in without changing the product itself.

For developers this is a telling shift. The product is no longer "our model" but a router plus a pool of models plus quality control. The model becomes a replaceable part.

The model is not the whole product

The authors of the announcement separately stress that answer quality is determined not only by the model but also by the harness. This word usually refers to everything surrounding the model: system instructions, access to tools and search, document handling, checks and retries. The same model in a different harness can respond noticeably differently. Yandex says it measures the harness's influence with benchmarks. Details of the methodology and results are in the full article on Habr. They are not in the announcement, so we will not cite specific figures for the gain.

Limitations and what to do

There are things the announcement does not reveal. It is unclear exactly how a request is classified: by a separate model, by heuristics or in some other way. There is no data on how often the router errs: sending a hard task to a fast model or spending "Expert" resources on a simple question. There are also no quality metrics compared with a single strong model. Any routing adds a step to the chain, which means latency and one more point of failure.

Practical takeaways for teams building their own LLM products:

• Don't push all traffic through the most expensive model. Label your requests by complexity and see what share really needs a strong model. At Yandex, Alice AI LLM handles about 80% of requests; your numbers may differ.
• Give users a manual switch, like "Expert" in Alice. It lowers the cost of a router error.
• Measure the harness separately from the model. Before changing the model, run a benchmark across different prompts, tool sets and checks.
• Keep the architecture modular so you can swap models without rewriting the product.

The race is no longer about a single biggest model but about who assembles a better system around several. Alice AI is a fresh mass-market example of this approach.