Google: Hackers Use Gemini AI for Every Attack Phase
▼ Summary
– State-backed hackers from China, Iran, North Korea, and Russia are using Google’s Gemini AI to assist in various attack stages, including reconnaissance, phishing, and coding.
– Specific malicious uses include Chinese actors employing Gemini for vulnerability analysis and Iranian actors leveraging it for social engineering and tool development.
– Malware like HonestCue and phishing kits like CoinBait show evidence of AI-assisted creation, with attackers using AI to generate and refine malicious code.
– Google reports attempts at AI model extraction, where actors systematically query Gemini to replicate its functionality, posing an intellectual property threat.
– Google has responded to this abuse by disabling associated accounts, implementing defensive classifiers, and reinforcing the model’s security guardrails.
State-sponsored cyber attackers are now leveraging Google’s Gemini artificial intelligence to power every step of their malicious campaigns, from initial reconnaissance to final data theft. According to a new report from Google’s own threat intelligence team, hacking groups linked to China, Iran, North Korea, and Russia are actively using the AI model. Their activities include profiling potential targets, crafting convincing phishing messages, writing and debugging malicious code, and even planning sophisticated vulnerability tests against specific organizations.
These advanced persistent threat (APT) actors are integrating AI into their core workflows. For instance, Chinese hackers adopted a cybersecurity expert persona when interacting with Gemini. They fabricated scenarios to have the AI automate vulnerability analysis and develop targeted testing plans, specifically requesting analysis of techniques like remote code execution and SQL injection against U.S.-based targets. Another China-linked group routinely used the model to troubleshoot code, conduct research, and gather technical advice to improve their intrusion capabilities.
The Iranian group known as APT42 utilized Google’s large language model to enhance social engineering efforts. They employed it as a development platform to accelerate the creation of custom malicious tools, using it for debugging, code generation, and researching new exploitation methods. This trend extends beyond reconnaissance, with threat actors using AI to implement new features into existing malware families.
Two specific examples highlighted are the CoinBait phishing kit and the HonestCue malware framework. CoinBait is a fraudulent cryptocurrency exchange site designed to steal login credentials. Its code contains artifacts strongly suggesting it was developed with the help of AI code generation tools. HonestCue, observed as a proof-of-concept, uses the Gemini API to dynamically generate C# code for second-stage malware, which it then compiles and executes directly in a system’s memory, a technique that helps evade detection.
Cybercriminals are also deploying AI in more direct attack campaigns. In so-called “ClickFix” operations, malicious actors place ads in search results for common tech support issues. When users click these ads, they are tricked into running commands that install info-stealing malware like AMOS on macOS devices. This method leverages AI to create convincing lures and streamline the attack process.
A separate but significant threat involves attempts to steal the AI models themselves. Google reports that Gemini has been the target of systematic model extraction and knowledge distillation attacks. In these incidents, entities use authorized API access to query the model with thousands of prompts, methodically mapping its decision-making processes to create a functional replica. One large-scale attack involved 100,000 prompts aimed at replicating the model’s reasoning, particularly for tasks in non-English languages.
While this does not directly threaten user data, it represents a major commercial and intellectual property risk. It allows attackers to rapidly develop their own AI capabilities at a fraction of the cost, undermining the business model of AI-as-a-service platforms. Google has responded by disabling accounts and infrastructure linked to documented abuse and has enhanced Gemini’s built-in classifiers to make such exploitation more difficult. The company states it builds AI systems with strong safety guardrails and continuously tests its models to improve security.
(Source: Bleeping Computer)
