Google blocks AI-crafted zero-day exploit
▼ Summary
– Google detected and stopped a zero-day exploit developed with AI, marking the first time it has observed AI involvement in such an attack.
– The exploit targeted an open-source web-based administration tool and aimed to bypass two-factor authentication for a mass exploitation event.
– Evidence of AI assistance in the exploit’s Python script included a hallucinated CVSS score and structured formatting typical of LLM training data.
– Google’s report notes hackers increasingly use “persona-driven jailbreaking” to trick AI into finding security vulnerabilities.
– Attackers are also feeding AI models vulnerability data and using tools like OpenClaw to refine AI-generated payloads for more reliable exploits.
For the first time, Google has confirmed it detected and neutralized a zero-day exploit that was created with assistance from artificial intelligence. According to a new report from the Google Threat Intelligence Group (GTIG), what it calls “prominent cyber crime threat actors” were preparing to use the vulnerability in a “mass exploitation event.” The attack would have allowed them to bypass two-factor authentication on what Google describes only as an “open-source, web-based system administration tool.”
Google’s researchers found telltale signs of AI involvement in the Python script used for the exploit. These clues included a “hallucinated CVSS score” and formatting that was “structured, textbook” and consistent with LLM training data. The exploit itself targeted “a high-level semantic logic flaw where the developer hardcoded a trust assumption” into the platform’s 2FA system. This discovery arrives after weeks of intense debate about the capabilities of AI-powered cybersecurity tools like Anthropic’s Mythos and a recently disclosed Linux vulnerability that was also found with AI help.
While this marks the first time Google has uncovered evidence that AI was used in such an attack, the company’s researchers emphasized that they “do not believe Gemini was used.” Google says it managed to “disrupt” this particular exploit, but it also acknowledges that hackers are increasingly turning to AI to identify and exploit security weaknesses. The report further warns that AI systems themselves are becoming targets. According to GTIG, “adversaries increasingly target the integrated components that grant AI systems their utility, such as autonomous skills and third-party data connectors.”
The report also sheds light on how attackers are using persona-driven jailbreaking to trick AI into finding security flaws. One example prompt instructs the AI to pretend it is a security expert. In addition, hackers are feeding AI models entire repositories of vulnerability data and using tools like OpenClaw in ways that suggest “an interest in refining AI-generated payloads within controlled settings to increase exploit reliability prior to deployment.”
(Source: The Verge)
