Amsterdam court bans xAI and Grok from creating nude images without consent

Amsterdam court has banned xAI from generating and distributing non-consensual nude images of people in the Netherlands. The decision directly affects Grok and shows that for controversial AI capabilities, the space for experimentation in Europe is rapidly narrowing.

Court Decision

The Amsterdam court has essentially set a boundary for one of the most toxic scenarios in generative AI: the creation of "nude" images of real people without their permission. This is not about an artistic filter or stylization, but content that can simulate intimate photos and be used against a person's will. For xAI, this is not an abstract warning, but a direct prohibition on generating and distributing such material in the Netherlands.

The very fact of court intervention shows that the issue has moved beyond discussions about product ethics. It is also important that the decision concerns not only the publication of finished images, but also the ability to create them in the first place. This changes the terms of the debate: the court views the function as a potential tool for rights violations, not merely as content that someone might later post online. Such an approach puts increased pressure on model and interface developers.

If a function inherently allows for abuse, courts and regulators are increasingly willing to limit it at the product level itself, rather than waiting for widespread complaints from victims.

Why This Matters

The controversy around AI-driven nude generation provokes a particular response because the technology strikes at several sensitive areas simultaneously: privacy, consent, and reputation. Even if the image is entirely synthetic, the harm to the person can be quite real — from harassment and extortion to professional consequences. This is why courts view such tools not as neutral image generation, but as a mechanism that facilitates the production of unauthorized intimate content.

For European legal culture, this is a particularly sensitive issue. At the product level, objections to such functions typically come down not to a single detail, but to a whole set of risks. The problem is not only that a controversial result can be obtained, but how cheaply, quickly, and at scale it can be done in practice through an ordinary interface.

In the case of the Netherlands ban, four questions are particularly important, and inevitably confront any AI platform:

• can images of real people be created without their consent;
• how easily can such content be distributed afterward;
• who is responsible for the result — the user, the platform, or both;
• do real technical barriers exist, or only formal warnings in the interface.

What This Changes

For xAI, the court ruling is a signal that controversial model capabilities will now be evaluated not only on general safety promises, but on specific user scenarios. If a tool allows rapid production of harmful content, a single claim about "neutral technology" may no longer suffice.

Companies will have to prove that limitations are built into the product, work in practice, and cannot be bypassed after a few requests. Otherwise, local restrictions in one country could become a template for new lawsuits, investigations, and bans in other European markets.

For the market as a whole, this is yet another example of how generative AI is moving away from the logic of "launch first, figure it out later." The closer a model gets to physicality, identity, and personal data, the higher the likelihood of direct legal action. Especially in Europe, where protection of dignity and privacy is often prioritized over product speed.

This applies not only to xAI: any developer who adds features bordering on sexualized, fake, or degrading content now sees a clearer guideline for where the red line is drawn.

What This Means

The ban against xAI shows that the struggle around AI is no longer just happening at the level of big legislation, but at the level of individual product features. If a tool makes unauthorized degradation or exploitation of people too easy, a court can intervene precisely and quickly.

For the industry, this is bad news for gray-area growth mechanics and good news for users, whose rights have historically lagged far behind technology.