The Marketplace Selling Custom AI Deepfakes of Women
▼ Summary
– The AI platform Civitai hosts a marketplace where users can buy and sell specialized instruction files (LoRAs) to generate deepfakes, with 86% of requests being for these files.
– Researchers found users posted bounties requesting high-quality deepfake models of real people, often public figures, with nearly 92% of these bounties being awarded and paid for.
– Civitai provides educational resources and articles that teach users how to customize AI outputs, including generating pornography, with most weekly requests now for NSFW content.
– Despite announcing a ban on all deepfake content in May 2025, the site still hosts many pre-ban requests and the AI models created to fulfill them remain available for purchase.
– Researchers criticize Civitai for not only providing the infrastructure for creating deepfakes but also explicitly teaching users how to exploit it.
The online marketplace Civitai has become a focal point in the complex debate over AI-generated deepfakes, particularly those targeting women. While platforms often grapple with blocking explicit content, Civitai’s model presents a more intricate challenge. It operates as a hub where users can buy and sell specialized instruction files known as LoRAs. These files are designed to “coach” mainstream AI image generators, such as Stable Diffusion, into producing content they were not originally trained to create. Researchers discovered that a staggering 86% of all deepfake requests on the platform were specifically for these LoRAs, which users then combine with other tools to create graphic or sexualized deepfakes.
A detailed study of the platform’s “bounty” system revealed how these requests function. Users would post financial incentives, often between fifty cents and five dollars, for the creation of “high quality” models of specific individuals. Targets frequently included public figures like influencer Charli D’Amelio or singer Gracie Abrams, with requesters often linking directly to the subjects’ social media profiles to facilitate image scraping. Specifications went beyond simple face-swapping, with bounties asking for models that accurately rendered a person’s entire body, captured specific tattoos, or allowed for customizable hair color. Some requests targeted niche online communities, such as ASMR artists, while one disturbing bounty sought a deepfake model of a woman the user claimed was his wife. The system proved highly effective, with nearly 92% of all posted deepfake bounties being successfully fulfilled and awarded.
While it is theoretically possible that some purchasers use these tools for non-explicit purposes, such use would still violate Civitai’s own terms of service and raise serious ethical questions. The platform’s role, however, extends beyond mere hosting. It provides educational resources that instruct users on how to use external software to further manipulate AI outputs, such as altering a person’s pose. Furthermore, the site hosts user-written articles detailing methods for instructing AI models to generate pornography. Research indicates a clear trend: the volume of pornographic content on Civitai has increased significantly, with the majority of weekly requests now being for NSFW material.
Matthew DeVerna, a postdoctoral researcher at Stanford’s Cyber Policy Center and a co-author of the study, emphasizes the platform’s active role. “Not only does Civitai provide the infrastructure that facilitates these issues; they also explicitly teach their users how to utilize them,” he states. This creates an ecosystem where harmful content is not just available but actively enabled.
In response to growing scrutiny, Civitai announced a policy shift in May 2025, moving from a ban solely on sexually explicit deepfakes of real people to a prohibition on all deepfake content. Despite this new rule, a major loophole remains. Countless deepfake requests submitted before the ban are still active on the site, and the AI models created to fulfill those bounties remain available for purchase. This backlog of existing content means the marketplace continues to host and profit from the very material its new policy claims to forbid. Neither Civitai nor its major investor, Andreessen Horowitz, responded to requests for comment on these findings.
(Source: Technology Review)
