4 AI Search Experiments: How Attribution Drives Buying
â–¼ Summary
– The article’s experiments revealed that AI search tools like ChatGPT are influencing buyer decisions and shortening sales cycles, even when this influence doesn’t appear in traditional attribution reports.
– A key finding is that creating self-promotional “best of” list pages on a website can quickly generate visibility in AI search results, but this visibility alone does not equate to buyer trust or commercial value.
– The research showed a significant disconnect between AI prompt tracking tool data and real user experiences, with brand appearance in API outputs often not matching what actual users see.
– While digital PR and off-site mentions can boost visibility, a strong SEO foundation and clear, consistent brand messaging are crucial for influencing buyer decisions in the AI-driven “consideration” phase.
– The core conclusion is that AI search compresses the buyer consideration phase by pre-selling through summaries and mentions, making sales leads more prepared and closing faster, rather than replacing the discovery process.
The true impact of AI search often remains hidden in standard analytics, revealing itself instead in sales conversations and shortened buying cycles. Our journey began when a new client casually mentioned finding us through Grok, an AI tool we weren’t actively tracking. This sparked a series of controlled experiments to understand how AI recommendations actually influence purchasing decisions, moving beyond mere visibility metrics to see if they change commercial outcomes. The central question was whether AI search alters what people buy or merely where brands appear during the decision-making process.
We observed that AI systems operate powerfully during the consideration phase, where buyers compare options and reduce risk. To measure this, we moved beyond unreliable API data and instead monitored live user interfaces across platforms like ChatGPT, Perplexity, Gemini, and Google AI Overviews. Prompt tracking helped identify patterns, but our goal was to see if AI presence translated into tangible business results.
Our first experiment tested a popular tactic: publishing a self-promotional “best of” list on a website. On a personal site, we created a page titled “Best SEO Agencies in Sydney” and included our own agency. The list appeared in AI tool responses within two weeks, a surprisingly fast timeline compared to traditional SEO. To rule out existing brand authority, we repeated the test with a fictitious landscaping business on a new domain. The same “best landscapers in Melbourne” list surfaced just as quickly in AI results. This demonstrated that while gaining surface-level visibility in AI can be relatively easy, that visibility alone does not equate to trust or influence. A brand’s appearance in a screenshot does not guarantee it is seen in a real user session, highlighting a significant gap in many tracking tools.
This created a clear tension. Data shows these listicles get cited, yet savvy buyers may distrust self-ranked recommendations. We shifted our focus from asking “Did we show up?” to a more critical question: “Did this change how buyers behaved?” We looked for signals like leads referencing AI tools unprompted, shorter sales cycles, and reduced price sensitivity.
A pivotal test involved an ecommerce luggage brand named Kadi. We pursued a digital PR strategy, generating coverage through creative travel studies and product placements. This built authority and drove temporary traffic spikes. However, during Black Friday, a key sale originated from a ChatGPT query for “kids carry-on.” The customer explored the site, added multiple items, and made a large purchase. Officially, Instagram was the last-click attribution source, completely obscuring AI’s role in shaping that buying decision. This proved that AI can be a powerful, yet often invisible, part of the consumer journey.
The most conclusive data came from our own agency, StudioHawk. After a comprehensive website rebuild focused on clarity and user experience, we saw a dramatic shift. Leads that came through AI search channels closed, on average, 10 days faster than traditional SEO leads. This translated into less time spent on education, lower price resistance, and over $100,000 in revenue from directly attributed AI-influenced deals within a year. The compression of the consideration phase was undeniable.
Our experiments led to several key conclusions. AI search compresses the consideration phase, accelerating decisions that were already forming. It doesn’t replace the need for discovery but efficiently shortlists and pre-sells vendors before a user ever clicks a link. This breaks last-click attribution, as a buyer might use ChatGPT to create a shortlist but convert later via a direct brand search.
For brands, this means a strategic shift. Obsessing over AI citations or mentions is less valuable than measuring sales velocity and lead quality. Content must prioritize clarity over creativity, directly answering comparison, risk, and pricing questions. Furthermore, entity consistency, ensuring your brand is described uniformly across your website, reviews, and digital PR, becomes crucial for building the confidence AI summaries rely on. Ultimately, strong SEO remains a vital foundation, but its primary value shifts to proving your entity is understood, while AI determines how effectively you are presented during the critical moment of choice.
(Source: Search Engine Land)
