LLM Development in 2026: RAG, Agents, and Choosing Models

The market is seizing the moment. In 2024, the world started figuring out how to apply LLMs; in 2025-2026, Russian business woke up — AI needs to be deployed everywhere: in development, CI/CD, evaluation, code review, task setting. But demand is outpacing talent. There are few people professionally working in LLM development. Start with LLM integration — these are specialists who understand business processes and solve practical problems. They're snapped up like hotcakes, the way CRM integrators were in the 2000s.

RAG and agents: the core toolkit

RAG (Retrieval-Augmented Generation) gives an LLM access to your data. Documents, contracts, and customer records are chopped into semantic chunks, converted into vectors, and stored in a vector database (Qdrant). When a user submits a query, the system finds the chunks closest in meaning and feeds them to the model to generate an answer. The result is search based on relevant information, not keywords.

Agents take action: they create tasks in a CRM, send emails, call services. They work through Tool Calling — a developer defines the available tools for the model in advance (database search, sending emails, command line), the model picks the right one, the application executes the action, and returns the result.

MCP (Model Context Protocol) is the standard for Tool Calling. Instead of writing custom code every time, use ready-made MCP servers. This becomes critical when orchestrating multiple agents. Distribute tools across departments and delegate work. For a prototype, use AI studios (Google, Yandex).

A real-world example: a contract analysis system. The first version was full automation with uploaded statutes, laws, and numerous checks — it achieved only 20% efficiency, since lawyers still manually double-checked the results. The second version embedded an agent directly into Word. Lawyers did the work themselves, discussing potential issues with the assistant. Speed doubled. The takeaway: technology matters, but user perception is critical. People want a controllable tool, not a replacement.

Models: big and small

Large providers cost money: • OpenAI • Anthropic • Google • Minimax • Llama • Qwen • DeepSeek • GLM • Kimi

There are open small models (up to 100 billion parameters): • GLM • Qwen • Gemma • Llama • open-source GPT

You can deploy them locally without worrying about licensing. Use large models in the early stages, when it's not yet clear exactly what you need. But in 90% of cases, small models get the job done. For example, when you just need to return pre-prepared text, small models save not just a lot of money, but a huge amount. The selection criteria are simple: how you work with the client and where you can cut costs. Most modern models support Tool Calling and handle Russian well.

Production: security and metrics

The workflow matters: who the users are, what questions they ask, what concerns them. Logging questions is the most valuable practice — queries reveal user pain points and system effectiveness.

Russian legislation requires storing citizens' data locally and restricts the transfer of personal data to third parties. Use an embedder on your own side (small, fast ones are available now). A local LLM solves the data leak problem. Filters, request limits, and authorization protect against prompt injection and token theft. Collect quality metrics — they show customer satisfaction and the system's real-world effectiveness.