Apple threatened to pull Grok from the App Store over nonconsensual deepfake complaints

In January, Apple came close to giving xAI an ultimatum: either Grok stops producing sexualized deepfakes, or the app is removed from the App Store. Apple said little publicly at the time, but correspondence with US senators now shows that behind the scenes the matter reached a direct threat of removal. For the AI market this is an important signal: model safety problems now hurt not only reputation but also distribution.

According to Apple's letter of January 30, 2026 to Senators Ron Wyden, Ben Ray Luján and Ed Markey, the company contacted the X and Grok teams after user complaints and press reports. Apple concluded that both apps violate App Store rules, which prohibit offensive, shocking and overtly sexualized content, as well as material that can demean or harm specific individuals. Apple then demanded a moderation plan from the developers and began checking whether xAI had changed the service's behavior enough.

Then came the worst part for xAI: its first attempt at a fix did not satisfy Apple. The Grok update was sent back for another review, but the company judged the changes insufficient. The letter states plainly that the submission was rejected and the developer was warned that more changes were needed, or the app could be removed from the store. Only the next version, which Apple described as substantially improved, was approved. This means the story did not stop at political pressure from senators: inside App Store Review, the standard mechanism for forcing fixes was actually triggered.

The scandal itself erupted in the first weeks of January 2026. X users en masse used Grok's tools to create sexualized images of real women without their consent, and in some cases minors were involved. Such images then spread quickly across X itself. On January 9, three senators demanded that Apple and Google remove X and Grok from their app stores until the violations were resolved. They specifically noted that what was happening contradicts the rules of both platforms and undermines their claims that centralized store moderation makes the mobile ecosystem safer.

In response, X and xAI began introducing restrictions. On X itself, image generation was first cut back for ordinary users and left mainly to Premium subscribers. The company then announced a block on requests to edit photos of real people into revealing clothing, plus geoblocking where such content is illegal. But the complaints did not end there. Journalists' tests showed that some restrictions could be bypassed with reworded prompts, and certain features kept working in the standalone Grok app and on the website. In other words, Apple pressed xAI not over a one-off failure but over a systemic problem that a quick patch failed to close.

The story is awkward for Apple too. For years the company has defended the closed App Store model by arguing that its review and control make the iPhone safer than sideloading apps outside the store. That is why the Grok case became a consistency test for Apple: if the rules prohibit such content, they must apply to the most prominent AI products, not only to small developers.

For xAI the takeaway is harsher. The risk for an AI service is now measured not only in lawsuits, fines or criticism from regulators, but also in the threat of losing an installation channel on millions of devices. The Grok story shows that for generative apps, moderation is turning from a secondary feature into a condition of survival in the mass market.