Google Agent Gets Its Own Web Identity

On March 20, 2026, Google quietly introduced a new addition to its official list of web fetchers. This isn’t a crawler or a training bot. It’s an agent.

Google-Agent serves as the user agent string for AI systems operating on Google infrastructure that browse websites on behalf of users. When someone asks an AI assistant to research a product, fill out a form, or compare options across different sites, Google-Agent is the entity that actually visits the page. Project Mariner, Google’s experimental AI browsing tool, is the first product to use it.

This is not Googlebot. Googlebot continuously crawls the web, indexing pages for search. Google-Agent only appears when a human requests it. That distinction fundamentally changes how it functions.

Robots.txt Does Not Apply

Google classifies Google-Agent as a user-triggered fetcher. This category includes tools like Google Read Aloud (text-to-speech), NotebookLM (document analysis), and Feedfetcher (RSS). They all share a common property: a human initiated the request. Google’s stance is that user-triggered fetchers “generally ignore robots.txt rules” because the fetch was requested by a person.

The reasoning: If you type a URL into Chrome, the browser fetches the page regardless of what robots.txt says. Google-Agent operates on the same principle. The agent acts as the user’s proxy, not an autonomous crawler.

This marks a significant departure from how OpenAI and Anthropic handle similar traffic. ChatGPT-User and Claude-User both function as user-triggered fetchers, but they respect robots.txt directives. If you block ChatGPT-User in robots.txt, ChatGPT won’t fetch your page when a user asks it to browse. Google made a different choice.

Website owners who relied on robots.txt as a universal access control mechanism now face a gap. To restrict access from Google-Agent, you’ll need server-side authentication or access controls. The same tools you’d use to block a human visitor.

Cryptographic Identity: Web Bot Auth

The more significant development lies in a single line of Google’s documentation: Google-Agent is experimenting with the web-bot-auth protocol using the identity https://agent.bot.goog.

Web Bot Auth is an IETF draft standard that functions like a digital passport for bots. Each agent holds a private key, publishes its public key in a directory, and cryptographically signs every HTTP request. The website verifies the signature and knows, with cryptographic certainty, that the visitor is who it claims to be.

User agent strings can be spoofed by anyone. Web Bot Auth cannot. Google adopting this protocol, even experimentally, signals where agent identity is heading. Akamai, Cloudflare, and Amazon (AgentCore Browser) already support it. Google brings the critical mass.

This matters because the web is about to face an identity problem. As agent traffic increases, websites need to distinguish between legitimate AI agents acting on behalf of real users and scrapers pretending to be agents. IP verification helps, but cryptographic signatures scale better and are harder to fake.

What This Means For Your Website

Google-Agent creates a three-tier visitor model for the web:

• Human visitors browsing directly.Each tier has different access rules, different intentions, and different expectations. A crawler wants to index your content. An agent wants to complete a task. It might be reading a product page, comparing prices, filling out a contact form, or booking an appointment.Here’s what to do now:
• Monitor your logs. Google-Agent identifies itself with a user agent string containing `compatible; Google-Agent`. Google publishes IP ranges for verification. Start tracking how often agents visit, which pages they hit, and what they attempt to do.

The Hybrid Web Isn’t Coming. It’s Logged

A year ago, the idea that AI agents would browse websites alongside humans was a conference talk prediction. Today, it has a user agent string, published IP ranges, a cryptographic identity protocol, and an entry in Google’s official documentation.

The web didn’t split into human and machine. It merged. Every page you publish now serves both audiences simultaneously, and Google just made it possible to see exactly when the non-human audience shows up.