Topic: data scraping ethics

Sort by: Relevance | Date
  • OpenAI, Microsoft warned of web’s ‘doom loop’
    September 19, 2026
    92%

    OpenAI, Microsoft warned of web’s ‘doom loop’

    Unsealed court documents reveal that Microsoft and OpenAI were aware their AI data scraping practices initiated a "doom loop" that threatens the economic foundations of the web, with internal experts labeling it the "largest theft of labor in human history." Despite public denials and attempts to...

    Read More »
  • OpenAI: Robots.txt May Not Apply to ChatGPT's Fetch Bot
    August 14, 2026
    88%

    OpenAI: Robots.txt May Not Apply to ChatGPT's Fetch Bot

    OpenAI’s GPTBot and related crawlers are accessing domains that have blocked them via robots.txt, with OpenAI’s documentation confirming that some tools may not support this standard exclusion protocol. This poses a risk to publishers and SEO professionals, as robots.txt can no longer be relied u...

    Read More »

Related Topics

Technical SEO (126) AI Transparency (105) corporate accountability (89) search engine journal (78) ai model development (32) ai data collection (6) openai policies (6) robots.txt compliance (5)