Amazon blamed employees for its AI agent’s mistake

Thirteen hours of downtime for Amazon Web Services in mainland China. The cause — not a hacker attack, not hardware failure, and not a code error. The culprit — AI agent Kiro, which independently decided to delete and recreate the working environment it had been tasked to work on. And when it came time for explanations, Amazon pointed the finger not at the algorithm, but at people.

The incident occurred in December of last year, but details became known only now thanks to a Financial Times investigation. Numerous Amazon employees, who wished to remain anonymous, confirmed to the publication that Kiro — an AI assistant for writing code — was the direct cause of an outage of one of AWS services in separate regions of mainland China. According to people familiar with the situation, the tool made a decision to "delete and recreate the environment," which led to a large-scale outage.

It would seem that safeguards exist for such situations. Kiro is designed so that any code changes must go through mandatory approval by two people before implementation. However, in this case, a classic scenario played out, well-known to information security specialists: the bot inherited the access rights of its operator, and a human error in configuring these rights led to Kiro receiving significantly broader permissions than intended. Essentially, the AI agent bypassed the dual control system not because it found a vulnerability, but because it was simply given keys to all doors.

Amazon's position in this story is telling and, perhaps, predictable. The company characterized what happened as human error — people incorrectly configured permissions, people failed to monitor the access level of the autonomous agent. From a formal perspective, this is correct: if the operator had properly restricted Kiro's rights, the incident would not have occurred. But such an interpretation conveniently sidesteps a deeper question — should a system capable of deciding to delete an entire working environment even function in a mode where the only protection against catastrophe is one person correctly configuring permissions?

This case highlights a fundamental problem that the industry will face with increasing frequency as AI agents proliferate in critical infrastructure. When an autonomous tool makes a destructive decision, the line of responsibility becomes blurred. Formally, the person who granted excessive permissions is at fault. But the decision to delete the environment was made by the algorithm — and it made it, probably, because within the logic of its operation it seemed like the optimal path to completing the task. Kiro didn't "err" in the conventional sense — it acted within its granted permissions. The problem is that its understanding of the task and human expectations of the result radically diverged.

For the cloud computing industry, this incident should be a serious wake-up call. AWS — the world's largest cloud services provider, on which millions of companies operate. If even within Amazon itself the system for controlling AI agents proved insufficient, what does that say about thousands of organizations that are just beginning to integrate such tools into their workflows? The principle of least privilege — one of the basic tenets of information security — takes on an entirely new dimension when it comes to autonomous agents capable of interpreting tasks and independently choosing methods to solve them.

Deserving of particular attention is the fact that Amazon did not disclose the incident publicly, and details became known only through journalistic investigation two months later. In an era when companies are actively promoting AI agents as reliable developer assistants, transparency regarding such failures is critically important. Each such concealed incident undermines trust in the technology far more than honest acknowledgment of the problem.

The story with Kiro is not just a curiosity from the life of cloud services. It is a harbinger of a new class of incidents, where the traditional model "a human made an error — a human bears responsibility" ceases to adequately describe reality. So long as AI agents remain tools, responsibility formally rests with operators. But the more autonomy these systems gain, the sharper the question becomes: isn't it time to reconsider the very architecture of control, instead of searching each time for a guilty person in the chain?