Topic: ai bot blocking
-
ChatGPT Index, GA Benchmarks, Google Refiles: SEO Pulse
Cloudflare will treat Googlebot and Bingbot as dual-purpose crawlers starting September 15, 2026, meaning sites with AI bot blocking enabled will inadvertently block search indexing unless they opt out beforehand. An analysis by Resoneo found that ChatGPT's in-house index serves pages from publis...
Read More » -
Cloudflare AI Bot Blocking May Also Stop Googlebot Indexing
Cloudflare's "Block AI Training" setting can inadvertently block legitimate search engine bots like Googlebot and Bingbot, causing HTTP 403 errors on sitemap fetches, as reported by an r/SEO user; disabling the setting restores access immediately. The issue stems from Cloudflare classifying mixed...
Read More » -
AI Crawlers Blocked by More News Sites by Default
Reuters and Time have joined a growing number of major publishers switching to "allowlists" that block all AI crawlers by default, only granting access to approved bots, a reversal of the traditional robots.txt system. A Tollbit report found 30% of AI bot scrapes ignore robots.txt rules, and swit...
Read More » -
Is Your Managed WordPress Silently Blocking AI Bots?
An AI citation monitoring tool revealed stark disparities in platform presence for searchinfluence.com, with Google AI Mode at 37.8% while Claude and Meta AI sat at 0%, despite identical content quality and topical authority across all platforms. Cloudflare logs showed 65.8% of 29,099 bot request...
Read More » -
Cloudflare Blocked 416 Billion AI Bot Requests Since July
Cloudflare has blocked over 416 billion AI bot requests since July, revealing the massive scale of web data harvesting for AI training and a significant imbalance in access, with Google's crawlers reaching far more webpages than competitors like OpenAI. The situation creates a dilemma for publish...
Read More » -
Kohler's Toilet Cameras Lack True End-to-End Encryption
Kohler's Dekota smart toilet falsely advertised end-to-end encryption, as user health data is decrypted and accessible on company servers, leading to the removal of that claim. A major Chinese cyberespionage campaign, Salt Typhoon, infiltrated US telecom networks and accessed communications of po...
Read More » -
Brands Unite to Monetize AI Training Data with New Licensing Standard
The Really Simple Licensing (RSL) framework offers publishers a structured way to monetize content used for AI training by embedding licensing and royalty terms directly into their materials. RSL is supported by a coalition of major publishers and aims to democratize compensation, moving beyond t...
Read More »