SERP Providers Claim Google Throttles Goto Links
▼ Summary
– Google has implemented a throttling mechanism that restricts the speed at which users can follow /goto redirects for search results on a single page.
– This restriction causes SERP data providers like DataForSEO, Serper, and SerpApi to encounter delays or receive incomplete redirect resolutions when scraping results rapidly.
– The limit is based on the specific results page rather than IP address or search query, making common bypass techniques ineffective against the new system.
– Providers are adapting by modifying their API modes; for instance, DataForSEO’s Standard mode now resolves all URLs with added latency while High Priority modes resolve fewer links.
– Serper initially struggled with partial resolution but later found a temporary workaround to retrieve real URLs, highlighting the ongoing challenge of accessing accurate search data.
Google search result providers are facing significant hurdles as the tech giant implements stricter limits on following redirect links. Services like DataForSEO and Serper, which supply access to Google’s search data via APIs, have reported that google.com/goto redirects are being throttled. This restriction prevents these tools from instantly resolving the final destination URLs, leading to reports where rank data displays incomplete or generic redirect paths instead of actual website addresses.
The issue began surfacing around late August when Google introduced the www.google.com/goto mechanism. These links act as intermediaries that cannot be decoded directly; a request must follow the link to reveal the true destination. The new limitation specifically targets the speed at which these requests are processed. Neither DataForSEO nor Serper has linked to an official Google announcement regarding this change, meaning the details currently come from the providers themselves.
How The Throttling Mechanism Functions
DataForSEO outlined the mechanics of this restriction in a November 2nd article. The system triggers when multiple links from the same results page are accessed in rapid succession. Each link contains an identifier tied to its source results page, which serves as the basis for the throttle.
When the limit is hit, Google stops sending requests directly to the destination. Instead, it returns a page featuring a “Continue” button. This button remains disabled for a specific waiting period. The duration of this wait increases based on how quickly previous links were resolved. For instance, if a user attempts to open 18 links from one page quickly, the first five or six might resolve immediately. However, the seventh link could require a one-second delay, the eighth three seconds, and subsequent links could face waits extending up to 40 seconds or more. DataForSEO noted instances where users waited up to 120 seconds for a single URL resolution.
Crucially, this limit applies per results page rather than per IP address or search query. Changing IPs or queries does not bypass the restriction. Furthermore, delaying requests by an hour or more does not reset the counter; the limit is strictly bound to the specific results page session.
Provider Workarounds And Status Updates
SERP data providers have implemented various strategies to mitigate the impact of these delays. DataForSEO adjusted its Standard Priority mode to resolve all goto URLs, accepting longer execution times. In contrast, its High Priority and Live modes currently resolve only about half of the goto URLs due to the imposed delays.
Serper identified the issue on its status page at 1:30 a.m. UTC on October 30. Initially, their fix allowed resolution of the top three organic links while returning results four through ten as goto links, which would work approximately 30 seconds after the search. They also added a /resolve endpoint to fetch the remaining data. By October 31, Serper claimed to have found another method to retrieve real URLs, restoring full functionality, though they described this solution as “likely temporary.”
SerpApi, another major provider, logged similar issues starting October 30. Their status page indicated that their Google Search API returned goto links instead of standard organic results. While they resolved the initial incident by November 2nd, a second report on November 3rd highlighted intermittent problems with the same issue. No cause was provided for either incident.
Impact On Rank Tracking And Reporting
This technical shift has tangible consequences for SEO professionals and data analysts. Many rank tracking services and internal reports rely on URL-level reporting. Since approximately October 30th, some services may display Google goto redirects in place of true destination URLs, depending on the provider and request methodology.
If a specific URL appears missing from a URL-level report, it may still be ranking but simply have an unresolved redirect. Before concluding that a site has lost a ranking position, experts should verify whether the stored URL is actually a Google redirect. Contacting service providers to understand their data collection methods is essential for accurate interpretation.
As of November 4, SerpApi’s incident report from November 3 remained listed as “Identified.” Serper noted that if Google further restricts their URL retrieval methods, they will revert to automatically resolving the top three results and using the /resolve endpoint for the rest after a thirty-second delay. DataForSEO stated it will continue to adjust its processing based on observed changes in Google’s handling of goto resolutions.
Providers advise reviewing reports from late October through early November again once these adjustments stabilize. The evolving nature of these restrictions means that current data snapshots may require re-evaluation as providers refine their approaches to handle the ongoing limitations.
(Source: Search Engine Journal)
