Nate Soares explains on a Bloomberg podcast why AI can't be "aligned"

Bloomberg has published a conversation with Nate Soares, president of the Machine Intelligence Research Institute (MIRI). In 2025, he and Eliezer Yudkowsky released the book "If Anyone Builds It, Everyone Dies." Its main idea fits in the title: if anyone builds a superintelligent AI, everyone dies. According to the podcast announcement, after the Hugging Face incident, talk of doomsday scenarios stopped being marginal and moved into the mainstream. The piece does not specify what exactly happened on the platform, so the scale of the incident cannot be assessed here.

An old worry in a new context

Soares and MIRI were working on AI safety long before ChatGPT and large language models. Their position was usually treated as a curiosity then; now the news forces people to revisit it. The podcast is an attempt to explain why someone who has worked on this for decades sees the risk not as hypothetical but as the expected outcome.

The conversation centers on two questions: why, in Soares's view, superintelligent AI cannot be "aligned" (that is, reliably made to have goals that match human ones), and how exactly the technology will slip out of control as its capabilities grow.

Key points of the conversation

From the announcement and the book it draws on, several positions stand out:

• The authors see human extinction as the inevitable consequence of building superintelligence, not as one possible outcome among others.
• By their assessment, aligning a superintelligent system's goals with human ones is a practically unsolvable task.
• Danger grows with capability: the more capable a system is, the smaller the chance of stopping it when something goes wrong.
• Soares heads MIRI, an organization that has worked on this question since before large models were commercial.
• Format: a single episode of Bloomberg Tech with an extended explanation, not a brief comment.

Where the argument is vulnerable

Skeptics in the industry object to such scenarios along several lines. They point out that "inevitable doom" has no empirical basis: no existing system has demonstrated the ability to seize control of the world. The timeline for superintelligence is also disputed, as is whether it can be built on current architectures at all. Soares's supporters reply that waiting for proof is pointless, because once the first real example appears, it will be too late to fix anything.

The announcement contains no specific figures, timeline forecasts, or details of the "escape" mechanism, so the full argument can only be heard in the episode itself. It is also unclear from the text how Soares connects the Hugging Face case to his conclusions.

What to do about it in practice

For teams building products on top of models, the most useful takeaway is more modest than the apocalyptic one. The discussion of risk has left its academic niche, and regulators, customers, and investors will increasingly ask how access control, dangerous-capability evaluation, and system shutdown procedures are set up. Those who release open models or host them on public platforms should work out an incident response policy in advance rather than write it after the fact.

For readers trying to make sense of the debate, the main advice is to listen to both sides. The position of Soares and Yudkowsky is worth knowing even if you disagree with it, because it sets the frame in which the toughest proposals for AI regulation are now being discussed. The full episode can be found on Bloomberg.