Google sues Chinese scam network using Gemini AI automation

▼ Summary
– Google filed a legal action against Chinese group Outsider Enterprise for running a phishing-as-a-service scam campaign via Telegram.
– The group used Google’s Gemini AI to create nearly 300 scam website templates imitating Google, YouTube, and government agencies.
– The scam sent over 2.5 million text messages to Android users, with 55,000 occurring in a two-week period last month.
– The text messages lured users to fraudulent websites that stole personal data and banking details, though the total money stolen is unestimated.
– Google worked with AT&T, Verizon, and T-Mobile to block malicious texts, and its on-device scam detection in Google Messages stops 10 billion scam texts monthly.
Google frequently highlights how its generative AI tools are being used to build businesses, drive innovation, and even tackle global challenges. But the company is now confronting a darker reality: the same technology is powering organized crime. In a new legal action, Google has filed a lawsuit against a Chinese-operated network known as Outsider Enterprise, which it accuses of orchestrating a massive AI-driven scam operation. The tech giant says it is coordinating with law enforcement and mobile carriers to dismantle the scheme.
According to the complaint, Outsider Enterprise operates primarily through Telegram, offering what amounts to phishing-as-a-service. The group targets individuals who lack the technical skills to build fraudulent websites and launch text-based attacks on their own. Within its Telegram channels, the organization allegedly provided step-by-step guidance on leveraging Google’s Gemini AI to craft convincing replicas of legitimate sites, including those for Google, YouTube, and government services like New York’s E-ZPass. In total, the group reportedly offered nearly 300 different scam templates for its customers.
Google’s filing reveals that the scams enabled by Outsider Enterprise generated more than 2.5 million malicious text messages sent to Android users. A concentrated burst of activity saw roughly 55,000 of those messages sent within a two-week span last month. The investigation has so far uncovered 9,000 fraudulent websites and over 1 million URLs linked to the network.
The text messages typically warned recipients about account issues or package delivery problems. Clicking the embedded links directed users to those Gemini-generated fake websites, which appeared authentic enough to steal personal information and banking credentials. While the legal filing does not specify the total financial losses, a related blog post from Google acknowledges that hundreds of individuals have suffered monetary harm.
To combat the threat, Google partnered with AT&T, Verizon, and T-Mobile to block many of the malicious texts. The company also credits its on-device scam detection feature within Google Messages for reducing the success rate of these phishing attempts. That AI-powered tool already intercepts an estimated 10 billion scam texts each month, making it likely that it caught at least a portion of Outsider Enterprise’s activity before it reached users.
(Source: Ars Technica)




