This Tool Reveals When ChatGPT Secretly Searches Google

When your customers interact with AI assistants like ChatGPT or Gemini, the model isn’t just pulling answers from memory. It quietly dispatches a series of traditional web searches in the background, retrieves the top-ranking pages, and synthesizes a response from that data. The websites that rank for those hidden queries get cited. The ones that don’t, remain invisible. QueryFan generates persona-specific prompts, runs them through both models, and captures the exact searches each one triggered. That list represents your real AI visibility target, and it’s free to use.

Keyword lists are useful, but they only tell half the story.

Let me clarify before anyone fires off a heated response.

I’m using “keywords” here to mean the direct, one-shot queries you type into a traditional search engine. Yes, we’ve been living in a “semantic” world for over a decade, but let’s agree on terms that everyone can follow for now.

The core problem with keyword lists in the context of AI search is threefold:

Typically, the prompts users feed into large language models (LLMs) are longer, more conversational, and multifaceted. Traditional searches are narrower and more direct. A traditional search is a one-shot event: you search, get information, then move on to an independent query. But LLM interactions are conversational, carrying the context of previous exchanges. The mechanisms LLMs use for web search also factor in personalization. If a user has mentioned they are vegan and then asks about “running shoes,” the model will likely perform a search tailored to that preference.

In essence, AI search has become a universal intent decoder for users. Those sprawling, contextual conversations with the AI are broken down into smaller, solvable queries. These are then run in the background as traditional searches on Google or Bing, and the resulting pages are used to generate the final answer. This process is called Retrieval Augmented Generation (RAG) .

The optimization target has shifted. You are no longer optimizing solely for what a human types into a chat box. You are optimizing for what the AI agent quietly searches for on their behalf, in the background, without the user ever knowing it happened.

Those background queries are exactly what QueryFan captures. They are often strikingly different from the user’s original question. And they are the precise list of terms you need to rank for to appear in AI-generated answers.

Exhibit A: Reddit Fell Off a Cliff on a Tuesday

The full scope of this hidden relationship became painfully clear when Reddit, which had been enjoying a meteoric rise in Google visibility, faced a sudden collapse. According to citation tracking data from PromptWatch, Reddit’s citation rate in ChatGPT responses plummeted almost overnight on September 10th, 2026. It had been running as high as 15% of all citations. Within days, it fell below 2%.

The cause was unglamorous: Google quietly removed the ability to request 100 search results at once (the `num=100` parameter) from its search API on that date.

Think about what that reveals. Reddit’s visibility in ChatGPT responses was tied directly to Google’s bulk search capabilities, not to anything Reddit did, not a training data update, not an alignment tweak. The implication is blunt: ChatGPT was bulk-pulling Google search results. Reddit dominated those results at the time. When the bulk-pull disappeared, so did Reddit’s citations.

AI search surfaces are, in large part, wrappers around traditional search. The “AI” part is real (the synthesis, personalization, and conversational flow), but the information retrieval step is remarkably familiar. Google indexes and ranks the web; the AI consults that index. Your content still needs to rank.

How QueryFan Works

Step 1: Your ‘Traditional’ Keywords

Your existing keyword list for “running shoes” might include various suggested variations from a tool like Google Suggest. For QueryFan, we simply take the topic of “running shoes” as our starting point, as we will generate prompts around it.

Step 2: Define Personas

Personas are how we customize the prompts we generate. This alters our traversal of the token space, aligning us with training data from millions of communities, forum posts, Reddit threads, and internet discourse where real users ask real questions with these identities.

QueryFan sends your persona plus topic combination to the LLM to generate the kinds of questions that persona would actually ask an AI tool. Not keywords. Questions. Real, conversational, context-laden questions. For the “middle-aged vegan man who just started running,” it will produce things like:

• “Which vegan running shoes are good for middle-aged men just starting to run?”Step 3: LLM Selection and AlsoAsked EnrichmentAI conversations branch. Someone who asks about vegan running shoes will ask follow-up questions about cost, brands, and injury prevention. QueryFan passes the generated prompts through the AlsoAsked API to capture the nearest-intent follow-up questions around each one. People Also Ask data is the right tool here because it was built to model question proximity, which is exactly what you need when predicting where a conversation will go next.For example, a UK search for “running shoes” will surface follow-up questions on specific brands, how to pick a shoe, and even common medical queries.You can also choose whether to use ChatGPT, Gemini, or both. Each LLM handles and fans out queries slightly differently, so if you are optimizing for a specific platform, it is best to get the data from there.Step 4: Query Fan-OutQueryFan sends the enriched prompt list to GPT-5 with web search enabled (via the OpenAI Responses API) and to Gemini with Google Search grounding active (via the Gemini Grounding API). Both models, when they decide a prompt requires current information, perform actual Google searches behind the scenes.This process captures the fan-out queries because both APIs are, fortunately, transparent about what they searched. The Gemini API returns a `webSearchQueries` array in the `groundingMetadata` field of every grounded response. OpenAI’s Responses API logs the actual search queries in the `websearchcall` output. QueryFan harvests both.The result is a table: persona-specific prompts in, the actual Google search queries the AI fired out. Not what your customer typed. What the AI searched for on their behalf. Those are your new SEO targets, and until now there has been no free tool that surfaces them at scale.The Grounding Question: Not Every Prompt Triggers a SearchA brief but important caveat before you sprint off to classify everything as an SEO opportunity.Not every prompt causes the AI to perform a web search. The models make a decision based on the consensus of token prediction on whether live information is required.For example, the prompt “What do red blood cells do?” does not trigger a search. There is a very steep bell-curve of which tokens are likely to appear next. In the billions of training documents, the answer has remained very stable, so an “in-model” answer can be confidently generated.At the opposite end, a prompt like “What happened in the news today?” would trigger a web search. There is a very flat curve of “what tokens are next?” because there is no stable answer within the training data; it always changes and requires live data. This is another version of the Query Deserves Freshness (QDF) concept that SEOs have used for years.If you are interested in grounding, Dan Petrovic has done excellent work in this area and even released trained models on Hugging Face to predict whether queries will be grounded when they hit a confidence threshold.QueryFan surfaces which prompts triggered searches and which did not. Only the grounded ones (those that actually caused a Google search to happen) are actionable through SEO. The in-model answers are, for now, largely outside your reach. You would need to influence training data to move the needle there, which is a different project with a much longer horizon.What You Do With the ResultsYou now have a list of actual search queries that AI tools fire when answering questions from your specific personas. Run a standard gap analysis:
• Which of these queries do you have content for?The first two categories are diagnostic. The third is your action list.One important distinction from traditional SEO: Your own ranking is not the only path to AI visibility. LLMs scan the top 10, 20, sometimes 50 results for a grounded query and synthesize across them. A trusted review site ranking at position 3 is a legitimate route to appearing in an AI-generated answer, even if your own domain never makes the first page. Getting a product reviewed on a high-authority specialist site, earning a mention in a roundup article, or appearing in relevant community content all count.LLM visibility is a multi-site focus. This means the gap analysis has two outputs: content to create on your own site and placements to earn on other people’s sites.The PunchlineCast your mind back to that Reddit citation graph. The one that fell off a cliff when Google changed a single API parameter. An entirely independent company’s AI visibility tracked the behavior of a search API it did not control and probably did not know existed.That is the shape of the dependency. And the implication is not that SEO is dead; it is almost the opposite. SEO is now operating at one additional remove: instead of optimizing for the human query, you need to optimize for the AI-translated query that happens between the human and Google.QueryFan gives you a way to see what that translation actually produces. Your keyword list tells you what people typed into a search bar. QueryFan tells you what ChatGPT and Gemini searched for on their behalf, in the background, without anyone asking them to announce it.Those are different lists. The gap between them is not a minor refinement to your content strategy. It is the part of AI search that nobody has been measuring because nobody has had a free tool to measure it with.