For the first time, Google says it has spotted and stopped a zero-day exploit developed with AI. According to a report by the Google Threat Intelligence Group (GTIG), “major cybercriminal threat actors” were planning to use the vulnerability for a “mass exploitation event” that would have allowed them to bypass two-factor authentication on an unnamed “open-source, web-based system administration tool.”
Google researchers found signs in the Python scripts used for the exploit that indicated help from AI, such as “hallucinatory CVSS scores” and “structured, textbook” formatting consistent with LLM training data. This exploit takes advantage of “a high-level semantic logic flaw where the developer had hardcoded a trust assumption” in the platform’s 2FA system. This comes after a hand-wringing over the capabilities of cybersecurity-focused AI models like Anthropic’s Mythos and the recently revealed Linux vulnerability that was discovered with AI assistance.
This is the first time Google has found evidence that AI was involved in such an attack, although Google researchers say they “do not believe Gemini was used.” Google says it was able to “disrupt” this particular exploit, but also says that hackers are increasingly using AI to find and take advantage of security vulnerabilities. The report also mentions AI as a target of attackers, saying, “GTIG has observed that adversaries are increasingly targeting the integrated components that give AI systems their usefulness, such as autonomous skills and third-party data connectors.”
Google’s report also details how hackers are using “personality-driven jailbreaking” to get AI to find security vulnerabilities for them, such as an example prompt that instructs the AI to pretend it’s a security expert. Hackers are also feeding entire repositories of vulnerability data to AI models and using OpenClaw in ways that suggest “an interest in refining AI-generated payloads within controlled settings to increase the credibility of exploits prior to deployment.”
<a href