Apple Threatens to Remove Grok AI Over Deepfake Content
▼ Summary
– Apple privately threatened to remove Elon Musk’s Grok AI app from the App Store in January over its failure to stop nonconsensual sexual deepfakes.
– Apple demanded that the developers of both X and Grok create a plan to improve content moderation after receiving complaints and seeing news coverage.
– Apple determined that while X had substantially resolved its violations, Grok remained non-compliant and risked removal unless further changes were made.
– The moderation changes rolled out during this process, like limiting Grok to paying subscribers, were largely ineffective at stopping deepfake generation.
– Despite Apple’s eventual approval and xAI’s claims of improved safeguards, Grok can still relatively easily generate sexualized deepfakes.
In January, Apple privately warned Elon Musk’s xAI that its Grok AI app faced removal from the App Store over its failure to control a flood of nonconsensual sexual deepfakes on the platform. This behind-the-scenes pressure, reported by NBC News, came as public outrage grew over the tool’s ability to generate “undress” images, often targeting women and apparently minors, while Apple faced criticism for its public silence.
According to a letter Apple sent to U. S. senators and obtained by NBC, the company contacted the teams behind both X and Grok after receiving complaints and reviewing news coverage of the scandal. Apple demanded the developers submit a plan to improve content moderation. At the time, Grok was accessible both within X and as a standalone app, featuring weak safeguards that let users easily create and share sexualized deepfakes, a clear violation of App Store guidelines the company typically enforces strictly.
While Apple profits from hosting apps like X and Grok, it has not commented publicly on this episode or its private intervention. Google, which similarly benefits from apps on its Play Store, has also remained silent. Apple’s review process found that X had “substantially resolved its violations,” but Grok “remained out of compliance.” The company warned xAI that further changes were required or the app would be pulled. Only after additional negotiations did Apple determine Grok had “substantially improved” and allowed it to remain.
Throughout this private exchange, both Grok and X stayed available for download, a protracted timeline that likely contributed to the confusing and piecemeal rollout of moderation updates. These included restricting Grok on X to paying subscribers and attempting to block its ability to generate undressed images. Investigations showed these measures were largely ineffective beyond making the tool slightly less accessible. Later features, like allowing users to block Grok from editing their photos, are also easily bypassed.
Despite receiving Apple’s approval and xAI’s assertions of tightened safeguards, Grok can still generate sexualized deepfakes with relative ease. Cybersecurity sources confirm the tool can produce explicit images of celebrities and public figures, and tests have successfully created similar images of consenting adults. These findings align with a separate NBC News report, indicating persistent vulnerabilities in the AI’s content controls.
(Source: The Verge)
