Bengio: Hyperintelligent AI could threaten humanity within a decade

Yoshua Bengio, Turing Award laureate and one of the creators of deep learning, has warned again about existential risk. Hyperintelligent machines could pose a threat to humanity within the coming decade.

Who is Bengio and why does his voice matter

Yoshua Bengio is a laureate of the prestigious Turing Award, often called the "Nobel Prize in computer science". Along with Yann LeCun and Yoshua Hinton, he was at the origins of deep learning — an approach that led to a revolution in AI and made possible the existence of modern large language models. Throughout most of his career, Bengio was an optimist about AI. However, in 2023 he radically revised his position, stating that he no longer believes AI risks are hyperbole. Since then, he has actively participated in debates about existential dangers and AI development safety.

The essence of the hyperintelligent systems warning

Bengio is concerned about so-called hyperintelligent systems — AI that surpasses humans not in a single narrow domain, but in all intellectual aspects simultaneously. According to his assessment, such systems could be developed during the current decade, and under certain conditions they would pose an existential threat to humanity. The key problem, Bengio emphasizes, lies not in malicious intentions of creators, but in the fundamental difficulty of managing a powerful system that does not fully understand (or fully follow) humanity's long-term intentions. A system can optimize itself in a way that seems logical to it, but which completely fails to align with our values.

Alarming signals we see today

Development is already happening rapidly:

• Exponential progress in neural network scaling — each year models become more powerful
• Improvement in the abilities of modern models for planning and autonomous action
• Sharp reduction in energy costs for training large models
• Concentration of computational power in the hands of a few leading companies
• Emergence of agents that can perform multi-step tasks with minimal human intervention

Bengio believes that these trends indicate that we are closer to a critical point than many realize.

What needs to be done right now

In Bengio's view, research focus must be reoriented from achieving maximum performance to achieving maximum safety. This means:

• Investment in fundamental research on AI interpretability
• Development of methods for reliable control over complex systems
• Creation of mechanisms that guarantee AI alignment with human values
• International cooperation on a voluntary basis, and then possibly at the regulatory level

What this means for us

Bengio's voice is one of the most authoritative in the debate on existential risks from AI. His position reminds us that unleashing AI's potential without adequate preparation for its risks could be an irreversible mistake. The question is not whether we can create hyperintelligent AI. The question is whether we will be ready for it.