ICLR 2026 в Рио: что думают исследователи о масштабировании больших моделей

ICLR 2026 took place in Rio de Janeiro in late April and confirmed its status as the premier annual gathering of AI researchers. For the scientific community, it is more than a prestigious publication venue: it is a place where ideas are born and tested that will become industry standards within a year.

Scaling with Efficiency in Mind

The era of 'more parameters = better model' is slowly but surely coming to an end. At ICLR, the loudest talk was not about the size of new models, but about Return on Investment: invest 100 million, get a 3% improvement — economically, this no longer makes sense. Researchers are tired of pretending that scaling guarantees progress. Why has this happened? Chips are getting more expensive, not cheaper. Doubling parameters no longer yields a doubling of performance. Companies have started honestly calculating the cost of training a base model plus fine-tuning for each specific task.

• Long contexts (200K+ tokens) are becoming a standard, not an exotic feature
• Engineers are optimizing inference speed and energy efficiency, not just model weight size
• Methods for dynamic computation scaling based on input data are emerging
• Mixture of Experts models (MoE) are displacing classical dense architectures

A shift in mindset was evident at the conference: researchers are talking about long-lived foundation models — models that persist and improve iteratively. Instead of constantly retraining from scratch, they add new data, specialized adapters, new layers. This is cheaper, simpler in production, and is becoming the new norm.

Safety as Priority

There were apparently more topics about safety and guarantees at the conference than a year earlier. At ICLR 2025, this was exotic. In 2026 — mainstream. Researchers are talking about the fact that large models need to be trained not only for performance but also for robustness: resistance to adversarial examples, to distribution shifts, to manipulation attempts.

"We can no longer release a model and hope that nobody breaks it in

combat conditions" — a typical position at the conference's panel discussions.

This is not just about regulation (although states are creating new requirements). It is about the fact that the scientific community itself has realized: the race for SOTA results is dangerous if you do not account for the real costs of harm.

Open Weights in the Shadow of Corporate Systems

The most contentious question at the conference: will open-source AI have a chance if Anthropic, OpenAI, and Meta keep the most advanced models closed? At ICLR, there was a divided opinion. One part of the scientific community insists: open models are critical infrastructure that must be developed despite the risks. The other is already realistic: corporations will have the best models, they have resources for training and responsibility to investors. Open science may remain in a supporting role — developing methodology but not holding the front line.

What This Means

ICLR 2026 showed: the era of exponential scaling is ending, but AI is not slowing down — the direction is changing. Ahead lies the era of systemic optimization: how to use resources more efficiently, how to make models more reliable, how open science can remain relevant. For startups and engineers, this means: competitive advantage now lies not in training yet another large model, but in deploying existing models cheaper, faster, and safer.

*Meta is recognized as an extremist organization and banned in Russia.