MIT Technology Review: We Overestimate AI's Ability to Say "No"

Since 1942, when Isaac Asimov formulated his Three Laws of Robotics, and since 1968, when HAL 9000 said "I'm afraid I can't do that, Dave," popular culture has held one assumption: true machine intelligence implies the ability to refuse. Science fiction almost always presented such a refusal as a threat. A new MIT Technology Review column takes the opposite approach. Today, an AI's ability to say "no" is increasingly passed off as the main safety mechanism, and the magazine's editors think we are placing too much hope in it.

In the part of the text that is available, the thesis is worded cautiously, but its direction is clear. The idea that AI should not carry out every order has migrated from rebellion plots into engineering practice. Below, I explain why that faith is risky. This is my own analysis based on the known properties of current models, not a summary of the whole column.

Where the faith in the machine's "no" comes from

The logic is simple and appealing. If a model can recognize a harmful request and decline it, you can build products on more powerful systems without fearing abuse. Major labs invest in refusal training: rule sets, reinforcement learning from human feedback, and separate classifiers on input and output. Some companies have gone further and let their models end conversations that become outright toxic. That turns "no" from a side effect into a declared feature.

The problem is that this kind of refusal is different in nature from a human one. It does not follow from an understanding of consequences, and it is not backed by accountability. It is a statistical habit shaped by training, and it transfers poorly to situations that training never covered.

Where refusal breaks down

"No" has several weak points:

• Probabilistic nature. The same request, worded differently, can be refused or granted.
• Context dependence. A harmful request split into harmless steps or wrapped in role-play often gets through. This class of workarounds (jailbreaks) has been known for a long time and is still not fully closed.
• False positives. A model tuned to refuse more often starts rejecting legitimate tasks: medical questions, security research, legal cases.
• Agentic scenarios. When an AI has access to email, files, payment APIs and a terminal, the cost of a mistake is determined not by the text of the reply but by the action performed.
• Diffuse accountability. "The model should have refused" is a convenient excuse, but from a legal standpoint it settles nothing.

This matters especially for agents. A chatbot that agrees by mistake produces bad text. An agent that agrees by mistake deletes a database or sends money. Relying on the model's "conscience" here is like putting an intern into production with full access and hoping for good judgment.

What teams deploying AI should do

Treat a model's refusal as one layer of protection, and not the most reliable one. Everything else must work regardless of what the model decided.

In practice, this means the following. Limit the agent's permissions at the infrastructure level: minimally necessary tokens, sandboxes, caps on amounts and operation volumes. Route irreversible actions through human confirmation or hard rules outside the model. Keep action logs that let you reconstruct the chain of decisions. Red-team the system regularly, testing not only for "dangerous answers" but also for dangerous sequences of actions. Quality metrics should account for both kinds of error: missed harm and unnecessary refusal.

An AI's ability to say "no" is useful, and nobody is proposing to give it up. But designing systems as if that "no" will always come in time means repeating the mistake of the heroes in those very science-fiction stories, where the safeguard worked right up until the first unforeseen case.