Google’s LLMs.txt Guidance Varies by Product

Google’s own product teams are sending mixed signals about the value of llms.txt, leaving publishers with conflicting advice depending on whether they prioritize Search visibility or agentic browser readiness.

A newly published Google Search optimization guide explicitly advises that llms.txt is not needed for generative AI features. The document groups it alongside content chunking, AI-specific rewriting, and specialized schema as strategies to skip. This is the Search team’s clearest statement yet.

Just days earlier, Chrome’s Lighthouse tool moved in the opposite direction. Version 13.3 introduced an Agentic Browsing category that includes an llms.txt audit. This audit checks whether a site serves the file and flags any server errors during retrieval.

Lighthouse’s documentation frames llms.txt as a way to provide “a machine-readable summary of a website’s content, specifically designed for LLMs and AI agents.” Without it, the docs warn that “agents may spend more time crawling the site to understand its high-level structure and primary content.”

Google Search’s Position Has Been Consistent

For over a year, Google’s Search team has made clear that llms.txt is not a Google initiative and will not be adopted by the company. John Mueller compared it to the keywords meta tag, noting that no AI services used it and bots never requested the file. He called building separate Markdown pages for bots “a stupid idea.”

At Search Central Live Deep Dive Asia Pacific, Gary Illyes and Amir Taboul confirmed Google was not pursuing llms.txt. The optimization guide’s explicit instruction to skip it reinforces that stance.

Chrome’s Lighthouse Takes a Different Path

Lighthouse 13.3 ships with the Agentic Browsing category enabled by default. It evaluates WebMCP integration, agent accessibility, layout stability, and llms.txt. The audit only marks a site as “Not Applicable” if the file returns a 404; any other error triggers a flag.

Lighthouse describes llms.txt as an “emerging convention” from llmstxt.org and advises site owners to place it in their root directory. This category is separate from SEO audits, signaling that llms.txt helps browser-based agents, not search rankings or AI citations.

Google Has Sent Mixed Signals Before

This isn’t the first time Google’s internal teams have disagreed. In December, Lidia Infante spotted an llms.txt file on Google’s Search Central developer documentation. Mueller responded on Bluesky with “hmmn :-/” and offered no further explanation.

Dave Smart observed that the file appeared on multiple Google developer properties, including developer.chrome.com and web.dev. The pattern suggested an internal CMS platform update that automatically deploys llms.txt files, not a Search team decision. The Search Central file was removed within hours, but files on other Google properties remained.

What This Means for Publishers

Google’s answer on llms.txt depends entirely on the use case. For Google Search, it’s unnecessary for AI Overviews, AI Mode, or other generative AI Search features. For browser-based agents, Lighthouse considers it optional in an experimental machine interaction category.

Guidance is split between different Google developer sites, which can lead to conflicting instructions when comparing Lighthouse or its llms.txt documentation with Google’s Search docs.

What Comes Next

Google has not commented on the documentation gap between the two product teams. For many sites, creating a basic llms.txt file is simple, but maintaining it is questionable given that Google Search states it’s unnecessary for AI Search visibility.