Bengio: Superintelligent AI Could Threaten Humanity Within a Decade

Yoshua Bengio, a Turing Award winner and one of the creators of deep learning, has once again warned about existential risk. Superintelligent machines could pose a threat to humanity within the next decade.

Who Bengio is and why his voice matters

Yoshua Bengio won the prestigious Turing Award, often called the "Nobel Prize of computer science." Together with Yann LeCun and Geoffrey Hinton, he was at the origin of deep learning, the approach that led to the AI revolution and made modern large language models possible.

For most of his career, Bengio was an AI optimist. In 2023, however, he radically reconsidered his position, saying he no longer believed AI risks were hyperbole. Since then he has taken an active part in debates about existential dangers and the safety of AI development.

The core of the warning about superintelligent systems

Bengio is worried about so-called superintelligent systems: AI that surpasses humans not in one narrow area but in all intellectual aspects at once. By his estimate, such systems could be developed within the current decade, and under certain conditions they would pose an existential threat to humanity.

The key problem, Bengio stresses, is not malicious intent on the part of their creators but the fundamental difficulty of controlling a powerful system that does not fully understand (or does not fully follow) humanity's long-term intentions. A system may optimize itself in a way that seems logical to it but is completely at odds with our values.

Warning signs we see today

Development is already moving fast:

• Exponential progress in scaling neural networks: models get more powerful every year
• Improvement in the planning and autonomous-action capabilities of current models
• A sharp drop in the energy cost of training large models
• Concentration of computing power in a handful of leading companies
• The emergence of agents that can carry out multistep tasks with minimal human intervention

Bengio believes these trends indicate that we are closer to a critical point than many realize.

What needs to be done right now

According to Bengio, research focus must shift from achieving maximum performance to achieving maximum safety. This means:

• Investing in fundamental research on AI interpretability
• Developing methods for reliable control over complex systems
• Creating mechanisms that guarantee AI aligns with human values
• International cooperation on a voluntary basis, and then possibly at the level of regulation

What this means for us

Bengio's is one of the most authoritative voices in the debate on existential AI risks. His position reminds us that unlocking AI's potential without adequate preparation for its risks may turn out to be an irreversible mistake.

The question is not whether we can create superintelligent AI. The question is whether we will be ready for it.