Google AI Director’s New Content Strategy: Agentic Engine Optimization
▼ Summary
– Addy Osmani introduced Agentic Engine Optimization (AEO), a new model for optimizing content to be used by autonomous AI agents, parallel to SEO.
– AI agents process information in single requests without traditional browsing, making conventional engagement metrics like clicks irrelevant.
– Token limits are a critical constraint, so content must be concise with key answers placed early to avoid truncation or agent errors.
– Serving clean Markdown versions of pages is recommended, as it removes layout noise and makes parsing easier and cheaper for agents.
– New discovery files like `llms.txt` and `AGENTS.md` help agents efficiently find and use content within AI workflows.
A new framework for optimizing content specifically for AI agents is gaining attention, introduced by a senior Google engineering leader. Agentic Engine Optimization (AEO) represents a parallel discipline to traditional SEO, designed for autonomous systems that fetch, parse, and act on information without human intervention. This approach fundamentally changes how we think about content structure and delivery in an AI-driven ecosystem.
The core insight is that AI agents operate differently than human users. They collapse multi-step browsing tasks into single requests, extracting needed data instantly without scrolling or clicking. This renders conventional engagement metrics largely irrelevant for these automated systems. A primary technical constraint is the agent context window, with token limits posing a significant challenge. Excessively long pages risk being truncated or skipped entirely, potentially leading to incomplete or even hallucinated outputs. Consequently, token count emerges as a critical performance metric, directly influencing whether content is fully processed.
To adapt, content must be restructured for machine parsing efficiency. Key recommendations include placing answers early in the content, ideally within the first 500 tokens, and maintaining a compact, focused page structure. Long introductory sections and buried key insights are discouraged, as agents exhibit what the guidance terms “limited patience” for such formats. The goal is clarity and conciseness to fit within an agent’s operational parameters.
Format also plays a crucial role. The guidance advocates for serving clean Markdown versions alongside traditional HTML pages. Markdown strips out navigational elements, scripts, and complex layouts, reducing parsing noise and computational cost for AI systems. Making these .md files directly accessible and discoverable is part of the strategy.
For discovery, new conventions are emerging to help agents navigate. These include an llms.txt file acting as a structured index for documentation, skill.md files to define capabilities, and AGENTS.md as a machine-readable entry point for codebases. These files serve as efficient shortcuts, helping agents decide what content to read and utilize.
The strategic importance is clear: this adds a new optimization layer beyond human-centric SEO. If agents cannot efficiently parse content due to token limits, poor structure, or an incompatible format, they may skip, truncate, or misinterpret it. This directly impacts whether information is successfully used, cited, or acted upon within AI-powered workflows. The ultimate goal shifts from merely driving website visits to enabling successful outcomes inside autonomous agent processes.
It is important to distinguish this concept from traditional search optimization. The AEO framework discussed is unrelated to Google Search ranking algorithms. In fact, Google’s search team has previously advised against using markdown pages for SEO and does not utilize the llms.txt standard. This guidance specifically addresses how standalone AI systems interact with web content, outlining what “optimized” means in that distinct, emerging environment.
(Source: Search Engine Land)
