Author of the 'Bitter Lesson' Sutton: Scaling LLMs Alone Won't Get Us to Intelligence

If you build a product on large language models, you almost certainly hold an implicit bet: models will keep improving through data and compute, and your product will grow along with them. This week, the person whose essay is usually cited as the main argument for that bet said it's not so simple.

What Sutton Said

Richard Sutton, a Turing Award winner, wrote 'The Bitter Lesson' in 2019. His idea: general methods that scale with compute ultimately beat clever solutions devised by humans. Since then the essay has become almost a sacred text for proponents of scaling.

Now Sutton writes that compute alone is not enough, and that the science of AI is 'far from complete'. For him this is not a U-turn but a continuation of his earlier line. He has previously criticized the emphasis on scaling LLMs and called synthetic data 'a big mistake': the real world is infinitely complex, and any simulation remains 'microscopic'. Another bottleneck, in his view, is human expertise, which keeps a system from truly scaling.

His alternative is agents with world models that constantly learn from their own experience. Modern language models learn nothing new after training, and internet text describes only a small part of reality. What's needed is continual learning that does not erase old knowledge.

How Colleagues Responded

Jerry Tworek, who left OpenAI to explore new approaches, believes the 'bitter lesson' still holds, with a caveat: the more compute is put into training, the more the algorithm matters. Some algorithms scale, others hit a dead end, and many scaling directions have not been explored at all. In his words, the temporary success of transformers, together with a misreading of Sutton, 'hacked the reward' in the minds of an entire generation of scientists.

Jayden Teo adds an academic angle. He has repeatedly seen the 'bitter lesson' hold back research: promising ideas get discarded because they supposedly don't fit its spirit. But that is a misreading, not Sutton's fault. An idea doesn't have to survive scaling forever to contribute to progress.

The Other Side

Sam Altman believes the potential of LLM scaling is far from exhausted, and accuses critics of holding back a whole generation of researchers with excessive certainty about the limits of the method. As an argument he cites an OpenAI model that recently disproved a mathematical conjecture that had resisted mathematicians. He considers world models useful, for example in robotics, but says the data so far favor continued LLM scaling. Dario Amodei speaks similarly: for him, scaling progress is a rainbow without an end.

Still, it's worth remembering that both companies have built their entire technology on this bet, so it is hard to call them objective.

What This Means in Practice

The debate is already turning into money. According to The Decoder, the latest Frontier Radar review tracks 45 new AI labs betting on alternative approaches. Tworek launched Core Automation, which searches for learning algorithms beyond pretraining and RL. Sutton works with Oak Lab. What unites these founders is a belief in far more capable AI and doubt that the current recipe alone will get there.

What developers and product managers should do:

• Don't build into your roadmap the assumption that the next model will solve your problems on its own. Design the product to benefit from improvements without depending on a single scenario.
• Watch continual learning. If it works, approaches to memory, fine-tuning, and agent personalization will have to be rethought.
• Separate position from interest. Statements by the CEOs of Anthropic and OpenAI about scaling are also a defense of their business models.

This is not proof but a clash of opinions. No one has yet shown that scaling has hit a ceiling, and no one has shown a working replacement. But the fact that the criticism comes from the author of the main argument for scaling is a serious signal: betting everything on a single recipe is risky.