Почему ИИ-модерация Meta не защищает пользователей: скандал с Muse Image

Rest of World published a breakdown in July 2026: Meta and other major tech companies are increasingly handing content moderation over to neural networks, but user reaction to the Muse Image service revealed a key flaw — AI does not account for a person's consent and therefore fails to protect users.

What the Muse Image scandal revealed

The scandal around Muse Image exposed a blind spot in automated moderation: the algorithm checks content against formal rules but doesn't ask whether the person consented to the use of their likeness. According to Rest of World's 2026 report, consent is exactly what AI filters are structurally unable to evaluate.

The model sees pixels and categories — "nudity," "violence," "spam" — but doesn't see the relationships between people or the context in which the image appeared. The same picture can be harmless or devastating depending on whether the person depicted allowed its creation and publication. The classifier doesn't register that difference.

Why AI doesn't understand consent

AI moderation doesn't recognize consent because it's trained on content features, not on the will of the people behind the scenes. A system can distinguish a clothed body from a nude one with high accuracy, but the question "did the person depicted give permission" lies outside the data it was trained on. Consent is a fact from the real world, not a property of the file itself: it can't be computed just by looking at the picture more closely.

This creates a typical fork: content formally violates none of a platform's rules but still causes harm. The most obvious class is generated and swapped images of real people: technically it's "just a picture," but in substance it's the use of someone else's face and body without permission. The filter lets such material through, and the victim is left unprotected.

• Meta replaced third-party fact-checkers in the US with user-generated Community Notes in January 2025 and stepped up automation
• Muse Image is the AI service that prompted Rest of World's 2026 report
• User consent is a dimension that, according to Rest of World, AI filters are structurally unable to assess

Where big tech's auto-moderation is headed

Major platforms are betting on AI moderation for scale and speed: there aren't enough human moderators for billions of uploads. Back in January 2025, Meta announced it was replacing its US third-party fact-checking program with Community Notes and leaning more heavily on automated systems — fewer humans in the loop, more algorithm.

The problem is that growing automation doesn't close the consent gap — it makes it more visible. The fewer live moderators there are, the less often anyone notices harm that isn't described in the rules — and that's exactly where stories like Muse Image fall through.

Platforms' response is usually delayed: a rule against a specific harm appears only after a public scandal, once victims have spoken up themselves. Muse Image is another case of this pattern: first a wave of outrage, then promises to "fine-tune" the filters. But fine-tuning a classifier doesn't give it an understanding of consent — it only expands the list of prohibited patterns.

AI moderation cannot protect users because it doesn't account for consent. — the thesis of

Rest of World's report

What this means

This is a technological limit, not a settings bug. As long as moderation relies on content classification, it will let through harm that hinges on consent and context. Rest of World raises a question big tech prefers to sidestep: some moderation problems can't be solved with a smarter model — only with a different arrangement of rules and human involvement. For users, this boils down to a simple rule: an automatic filter is not the same as protection, and where your own likeness is at stake, you'll have to rely on complaints, lawyers, and public pressure — not an algorithm.

• Meta has been designated an extremist organization and is banned in Russia.