ICLR 2026 in Rio: What Researchers Think About Scaling Large Models
ICLR 2026 brought leading researchers to Rio de Janeiro at the end of April. At the year's most prestigious conference, the debate centered on how to scale models more efficiently (chips are getting more expensive), where AI is dangerous and how to defend against it, and whether open models will have a place if the giants keep their most powerful systems closed. AIRI attended the conference and shares its first impressions.
AI-processed from Habr AI; edited by Hamidun News
ICLR 2026 took place in Rio de Janeiro at the end of April and confirmed its status as the main annual gathering of AI researchers. For the scientific community, it is more than a prestigious journal publication: it is where ideas are born and tested that will become industry standards a year later.
Scaling with an eye on efficiency
The era of "more parameters = better model" is slowly but surely coming to an end. At ICLR, people talked less about the size of new models and more about return on investment: spend 100 million, get a 3% improvement, and it no longer makes economic sense. Researchers are tired of pretending that scaling guarantees progress.
Why has this happened? Chips are getting more expensive, not cheaper. Doubling the parameters no longer doubles performance. Companies have started to honestly count what it costs to train a base model, plus fine-tune it for each specific task.
- Long contexts (200K+ tokens) are becoming standard rather than exotic
- Engineers are optimizing inference speed and energy efficiency, not just the size of the weights
- Methods are emerging for dynamically scaling compute depending on the input data
- Mixture-of-experts (MoE) models are displacing classic dense architectures
A shift in mindset was noticeable at the conference: researchers talk about long-lived foundation models, which live a long time and improve iteratively. Instead of constantly retraining from scratch, teams add new data, specialized adapters, and new layers. This is cheaper, simpler in production, and is becoming the new norm.
Safety as a priority
There were noticeably more talks on safety and guarantees at the conference than a year earlier. At ICLR 2025 the topic was exotic. In 2026 it is mainstream. Researchers argue that large models must be trained not only for performance but also for robustness: resistance to adversarial examples, to distribution shifts, and to manipulation attempts.
"We can no longer release a model and hope that nobody breaks it in the field" — a typical position in the conference's panel discussions.
This is not just about regulation (although governments are creating new requirements). It is about the scientific community itself realizing that the race for SOTA results is dangerous if the real costs of harm are not counted.
Open weights in the shadow of corporate systems
The most tense question at the conference: will open source in AI stand a chance if Anthropic, OpenAI and Meta* keep their most advanced models closed?
Opinion at ICLR was split. One part of the scientific community insists that open models are critical infrastructure and must be developed despite the risks. The other is already realistic: corporations will take the lead on the best models, since they have the resources for training and answer to investors. Open science may remain in a supporting role, developing methodology but not holding the front line.
What this means
ICLR 2026 showed that the era of exponential scaling is ending, but AI is not slowing down; its direction is changing. Ahead lies an era of systemic optimization: how to use resources more efficiently, how to make models more reliable, and how open science can stay relevant.
For startups and engineers, this means the competitive advantage is no longer in training an even bigger model, but in deploying existing models more cheaply, faster, and more safely.
*Meta has been designated an extremist organization and is banned in Russia.
- Nov 20, 2026GitLab makes Duo Agent Platform credits last up to 2x longer
- Nov 20, 2026Superhuman opens Go to 20 third-party agents as the assistant passes 1 million users
- Nov 20, 2026AWS launches managed NVIDIA Dynamo for models that don't fit on a single server
- Nov 20, 2026Kimi Code CLI now works with Amazon Bedrock without a separate API key
Key Questions Answered
What are the key details on Scaling with an eye on efficiency?
The era of "more parameters = better model" is slowly but surely coming to an end. At ICLR, people talked less about the size of new models and more about return on investment: spend 100 million, get a 3% improvement, and it no longer makes economic sense. Researchers are tired of pretending that scaling guarantees progress. Why has this happened? Chips are getting more expensive, not cheaper. Doubling the parameters no longer doubles performance.
What are the key details on Safety as a priority?
There were noticeably more talks on safety and guarantees at the conference than a year earlier. At ICLR 2025 the topic was exotic. In 2026 it is mainstream. Researchers argue that large models must be trained not only for performance but also for robustness: resistance to adversarial examples, to distribution shifts, and to manipulation attempts. "We can no longer release a model and hope that nobody breaks it in the field" — a typical position in the conference's panel discussions.
What are the key details on Open weights in the shadow of corporate systems?
The most tense question at the conference: will open source in AI stand a chance if Anthropic, OpenAI and Meta keep their most advanced models closed? Opinion at ICLR was split. One part of the scientific community insists that open models are critical infrastructure and must be developed despite the risks. The other is already realistic: corporations will take the lead on the best models, since they have the resources for training and answer to investors.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.