Llms.txt V2 Adds Markdown Links for AI Agents
▼ Summary
– Jeremy Howard released llms.txt version 2 on August 10, the first update since the format launched, adding standard ways for agents to discover Markdown versions of pages.
– V2 supports two URL patterns for Markdown pages: appending .md to the full filename or replacing the file extension, and introduces rel=”alternate” and rel=”describedby” link relations to connect pages to their Markdown versions and llms.txt files.
– Sites can implement these link relations via HTML elements in the page head or through an HTTP Link header, the latter working on non-HTML files without changing site templates.
– The update stems from two years of adoption, with thousands of sites, platforms like Mintlify, and companies like Anthropic, OpenAI, and Google publishing llms.txt files, revealing a gap in where Markdown versions lived.
– Google’s Search team still ignores llms.txt files, and the update doesn’t affect that stance; the spec’s real value is for coding agents and documentation tools, not for search rankings or AI citation influence.
Jeremy Howard, the Answer.AI developer behind llms.txt, rolled out a version 2 update to the specification on August 10. This marks the first major revision since the format launched, and it introduces standardized methods for AI agents to locate Markdown versions of web pages.
The original 2024 proposal included both the llms.txt file and Markdown versions of individual pages, published at the same URL with .md appended to the filename. V2 retains this approach but adds a second URL pattern for Markdown pages and formal mechanisms for agents to discover them. For sites already using the file, upgrading to V2 requires only a minor adjustment rather than a full overhaul.
What Changed In V2
V2 now supports two distinct URL patterns for Markdown pages. Sites can either append .md to the full filename, which was the only pattern previously supported, transforming /docs/tutorial.html into /docs/tutorial.html.md, or they can swap out the extension entirely, converting the same page to /docs/tutorial.md.
To simplify agent discovery of Markdown pages, the V2 update introduces two link relations. A rel=”alternate” attribute with type=”text/markdown” provides a direct link from a page to its Markdown counterpart. A rel=”describedby” attribute points to the llms.txt file that governs it, since a single llms.txt file can describe all pages beneath its directory path.
Sites can implement these link relations either as standard HTML link elements in the page head or through an HTTP Link response header. The header option offers flexibility, as it works on non-HTML files such as Markdown pages themselves. It can also be configured via server or CDN settings, eliminating the need to modify a site’s templates or codebase.
How V2 Connects Pages, Markdown, And llms.txt
An HTML page uses two link relations to reference its Markdown version and the llms.txt file that covers it. The page /docs/tutorial.html points to rel=”alternate” type=”text/markdown” which leads to either /docs/tutorial.html.md or /docs/tutorial.md. It also uses rel=”describedby” to reference the /llms.txt file. These can be delivered through HTML elements or HTTP Link headers, as outlined in the llms.txt v2 specification.
Why Howard Made The Change
Howard explained that the update stems from two years of real-world usage. Thousands of sites now publish an llms.txt file, and platforms like Mintlify automatically generate one for every site they host. Google’s Chrome team has even integrated a check for this file into Lighthouse’s Agentic Browsing category, which was introduced in May. Anthropic, OpenAI, and Google’s Gemini team all publish their own llms.txt files for developer documentation.
As adoption expanded, a genuine gap emerged. The llms.txt file pointed agents toward pages, but the spec offered no guidance on where the Markdown versions actually resided.
Where This Leaves Google’s Guidance
The update does not shift the positions of Google’s two product teams. Google’s Search team has stated for over a year that llms.txt support is not on their roadmap, and the company’s AI optimization guide explicitly notes that Google Search does not use these files. The guide also clarifies that maintaining such a file “will neither harm nor help your site’s visibility or rankings” since Google Search ignores them entirely.
Chrome’s Lighthouse tool includes an Agentic Browsing check that attempts to fetch the llms.txt file and flags server errors. A missing file returning a 404 is marked as N/A instead, since providing the file is optional. The tool frames this as a way to help browser agents understand a site’s structure more efficiently. However, it does not specifically test for the new link relations or Markdown-page discovery mechanisms.
Why This Matters
The V2 update technically involves adding two link relations as HTML elements or using a single HTTP header. What truly matters is what this revision signals about the format’s intended audience.
Coding agents and documentation tools already read llms.txt files to accelerate API lookups. This use case has nothing to do with whether Google treats the file as a ranking factor. V2 clarifies this behavior rather than making a fresh argument for search visibility.
Websites maintaining an llms.txt file primarily to influence AI Overviews or AI Mode citations still lack any data or official Google statement supporting that objective.
Those who keep the file because coding agents, IDE tools, or documentation platforms already rely on it now have a more precise spec to follow and a defined pathway for directing tools to Markdown content when it exists.
Looking Ahead
Howard’s changes page documents every addition to the llms.txt spec along with the rationale behind it. V2 remains open for feedback on GitHub, so if you’re implementing these link relations now, keep in mind that the syntax could still shift before the final stable version is locked in.
(Source: Search Engine Journal)
