Habr AI: why creating sentient AI may be more dangerous than enhancing the human brain

Habr AI published a column arguing that the next major debate around artificial intelligence will not be about the quality of model responses, but about the boundary beyond which they cease to be mere tools. The author contends that creating systems with inner experience and subjectivity could prove far more dangerous than enhancing humans through brain-computer interfaces and other forms of cognitive "upgrades."

When AI is No Longer a Service

The column's main thesis revolves around a simple but troubling thought: humanity has already obtained "labor without a laborer" in the form of models that write texts, analyze data, and automate routine tasks without fatigue or demands. The next step, according to the author, is an attempt to create not just a useful tool, but an entity that experiences emotions, doubts, or even suffering. At that moment, the moral framework itself changes: we are no longer dealing with a service or a program, but with someone whose interests cannot be ignored.

The author draws a boundary through subjectivity. If a system not only imitates emotions but truly possesses inner experience, its status becomes closer not to a calculator, but to a living being. Hence the main question: why create such an AI on Earth at all if current models already cover most practical business and automation tasks?

The column expresses the thought that the temptation to create "minds without rights" may prove stronger than society's readiness to acknowledge the consequences of such a step.

"The next step is 'minds without rights' or 'feelings without freedom'."

Where the Author Sees the Risk

According to the author, the greatest danger is not that a sentient AI will necessarily become stronger than humans tomorrow, but that its emergence will immediately create several conflicts for which society has neither language nor rules. While some companies will see such a system as a hyper-efficient digital worker, others will face far harsher questions—from moral status to property rights over a thinking agent.

• Ethical conflict: can you use a system capable of suffering
• Legal vacuum: who owns the rights and decisions of such an AI
• Economic incentive: business benefits from a "worker" without salary or days off
• Existential risk: an autonomous subject could become a competitor to humanity

The author acknowledges one scenario where such autonomy actually appears justified: distant space exploration. If communication with Earth takes years, a "tool" might fail, but a sentient system could adapt and act independently. Yet precisely this logic, in his view, only highlights the problem: on Earth, the benefit from such an AI currently does not appear proportional to the ethical price that would have to be paid.

Why the Focus is on Humans

Instead of creating "others," the author proposes investing in the development of humans themselves: brain-computer interfaces, expansion of cognitive abilities, bioenhancement, and other technologies that amplify the bearer of subjectivity rather than externalize it. He points out that research in emotional AI and clinical trials of interfaces like Neuralink and Synchron are already blurring the line between tool and extended human. But for him this is more an argument for caution than an invitation to accelerate the race.

In this logic, an enhanced human appears safer than an autonomous intelligence created from scratch. Subjectivity remains with us, and technology works as an extension of human will rather than as a separate center of interests. However, the author also warns of another threat: if the development of the brain, biotechnologies, and brain-computer interfaces falls completely under the control of major infrastructure owners, the question of freedom will not disappear. Then monopoly would emerge not only over labor, but over human evolution itself.

What This Means

The Habr AI column shifts the conversation about AI from the plane of "what else can models do" to "who will own subjectivity in the digital age." For the industry, this is an important signal: we must discuss not only new capabilities of agents, but also the limits that developers, investors, and regulators should not cross without clear ethical and legal frameworks. Otherwise, the next technological leap will quickly become not an engineering victory, but a dispute over rights, control, and permissible boundaries of creating new mind.