Patreon drops request, starts blocking AI scrapers
▼ Summary
– Patreon is partnering with Cloudflare to actively block AI bots that scrape creator content for AI training, moving beyond passive requests via robots.txt files.
– The crackdown follows increased AI scraping sophistication since 2023, and new Patreon discovery tools like the Home Feed and Quips that could expose more content to crawlers.
– Cloudflare offers tools for publishers to restrict AI bots, including a “Pay Per Crawl” marketplace and a policy blocking mixed-use crawlers on ad-hosted pages by default.
– Testing showed that AI training crawlers were making thousands of weekly attempts to access Patreon, ignoring its robots.txt file, but the new measures reduced attempts to zero.
– Patreon will still allow bots that index pages to send users back to the platform, emphasizing that creators should control how their work is used by AI.
Patreon is taking a stronger stand against unauthorized AI data scraping, announcing on Thursday that it has partnered with Cloudflare to actively block AI bots from using creators’ content for model training. The membership platform, which helps creators monetize their work, says the move is a direct response to increasingly aggressive scraping tactics that have evolved since it first tried to curb AI crawlers in 2023.
While Patreon’s paywall has historically kept most creator content out of reach for scrapers, the company has recently rolled out new discovery features, including a revamped Home Feed and Quips, a tweet-like posting tool. These additions, designed to boost visibility, also create more potential entry points for AI bots seeking to harvest content without permission.
The crackdown places Patreon among a growing list of publishers and creators grappling with how AI companies ingest their work to improve models. Cloudflare has responded to this trend with tools that let websites restrict AI bots, including the Pay Per Crawl marketplace, where sites can charge for scraping access. Earlier this month, Cloudflare also updated its policies to block mixed-use crawlers by default on ad-supported pages, meaning bots that both index content and train AI models are now locked out.
Patreon is extending its existing Cloudflare partnership to deploy the company’s AI Crawl Control technology, updating its own AI policies and enforcement tools. The key difference now is that Patreon is moving beyond the robots.txt file, a standard but voluntary request, to actively block AI training bots.
“Consent shouldn’t depend on whether a scraper chooses to behave,” Patreon wrote in a blog post, underscoring the shift to mandatory controls.
During testing, individual AI training crawlers saw their weekly attempts to access Patreon drop from thousands to zero, suggesting the scrapers had been ignoring the robots.txt file all along. However, Patreon will still allow bots that index pages and help direct users back to the platform.
“As AI agents become increasingly powerful and popular, creators deserve a meaningful say in how their work is used by AI companies,” said Drew Rowny, Patreon’s product chief. “On most of the Internet, creators have to accept AI training on their work just to reach and grow an audience. Patreon has a different vision: creators should be able to grow their audience and control how their work is used.”
(Source: TechCrunch)
