San Francisco Urges Apple and Google to Remove AI ‘Nudify’ Apps

San Francisco city attorney David Chiu has issued cease-and-desist letters to Apple and Google, ordering the tech giants to remove 13 face-swapping apps from their app stores. These apps allow users to generate AI-powered nonconsensual nude images, and the letters, viewed by WIRED, accuse the companies of profiting from harmful technology. The legal notices demand that Apple and Google stop what Chiu calls “aiding and abetting” the sale of explicit deepfake images and “sever” business ties with the app developers.

“Generating non-consensual intimate images is illegal, harmful, and completely unacceptable,” Chiu told WIRED. His office previously took legal action against 16 popular deepfake websites. Now, Chiu argues that Apple and Google have likely collected “millions of dollars in fees” from apps offering nudification features, and they must improve moderation to prevent such apps from appearing in their stores.

“These companies have responsibility to ensure that apps on their platforms do not facilitate sexual abuse,” Chiu said. The letters cite California law, which prohibits supporting services that create deepfake pornography. The apps use in-app payments, and the tech companies take a cut of those transactions, according to the letters. “The fact that some of the world’s largest and most established technology companies are facilitating this has to stop.”

Researchers have repeatedly flagged apps in both Apple’s App Store and Google’s Play Store that enable users to generate AI-generated sexual images, including some rated suitable for children. Despite new laws and bans targeting explicit deepfakes, technology and social media companies continue to direct millions of people toward this harmful technology. Both Apple and Google have developer policies prohibiting pornography, abuse, and harassment, and they have removed dozens of nudify and deepfake apps after reports from researchers and journalists.

Google spokesperson Dan Jackson told WIRED that the company has deleted “hundreds” of apps with nudifying features for policy violations, including the five Android apps flagged by Chiu’s office. “Google Play does not allow apps that contain sexual content, and we continually take proactive steps to detect and remove apps with harmful content,” Jackson said in a statement. “When violations are reported to us, we investigate and take swift action, which in the case of these apps has included suspending hundreds of violating apps and restricting related search terms like ‘nudify’ on our store.” Apple did not provide comment ahead of publication.

Over the past five years, a lucrative market for deepfake nudification technology has emerged online. In January, xAI’s Grok was used to create millions of sexualized images. A range of apps, websites, and bots allow users,mostly men,to upload photos of people, overwhelmingly women and girls, and digitally “remove” clothing or place them into graphic sexual scenarios. Often, creating a sexual deepfake requires only a reference photo and a few clicks, with some results available in seconds. As underlying generative AI technology improves, images and videos become more realistic, with some services offering free results or charging small fees. Previous reporting by WIRED and Indicator Media has uncovered incidents in at least 90 schools where deepfake sexual abuse images have been created of minors.