Google says an AI framework turned security patches into exploits with no human in the loop
A new report out today from Google LLC’s Google Threat Intelligence Group finds that for the first time, attackers have used an artificial intelligence framework running with no human in the loop to turn security patches into exploits.
Called exploit_agent, the framework turned up on an exposed staging server, where it was wired into several frontier and locally hosted large language models. Its job, according to GTIG, is continuous comparison of popular web application code before and after each patch announcement. Once a likely flaw is identified, the exploit it drafts is executed automatically.
Data taken from victims was sitting on the same server, alongside scripts for brute-forcing, installing webshells and even removing webshells left by rival attackers. Victim telemetry suggests the operator was most interested in French websites tied to media, entertainment and education. Written in Russian, one comment in the framework’s configuration file tells whoever set it up to enter the keys “right here, not in the chat.”
In the previous edition of the quarterly tracker two months ago, GTIG said it had not observed fully autonomous attack pipelines deployed against targets in the wild. That changed over the past quarter, according to the new report, titled “Agentic Threat Activity Crosses a Significant Threshold.” Because the source material for a patched flaw is finite, GTIG believes attackers may find it more efficient to point LLMs at n-day vulnerabilities than to hunt for zero-days.
How quickly the gap between disclosure and a working exploit can close shows in the case of two flaws in print management servers from PaperCut Software Pty Ltd., chained together and exploited as zero-days in late August. Security researchers and Huntress Labs Inc. used large language models to generate working exploits within a day of disclosure.
A campaign tracked by GreyNoise Intelligence Inc. four days after disclosure is believed to have relied on LLMs to build its exploits before it hit hundreds of organizations. One foothold in that campaign reached domain administrator within minutes, GreyNoise found.
Elsewhere, open-source agentic penetration testing frameworks are being used to automate discrete steps of live intrusions. During one intrusion against the U.S. aerospace sector, a threat actor was observed using the multi-agent framework CAIRN.
Another actor ran a customized platform built on PentAGI between late August and early October to scan a wide range of organizations in Asia for vulnerabilities. For now, the noisy forensic trail left by unrestricted autonomous execution is said to make the frameworks easier to detect.
According to GTIG, influence operations now use models for far more than writing posts. Using cloud phone farms, a turnkey platform that one commercial contractor was building with help from the Gemini API is designed to run more than 100,000 social media accounts. To make generated photos pass as smartphone shots, prompts behind a separate platform, which targeted political sentiment across Europe and the Middle East, are said to have called for deliberate camera flaws.
Among government-backed actors, photos on the fake recruiter profiles that a suspected North Korean group tracked as UNC6836 used to approach cryptocurrency organizations in North America and Europe were found to carry Google’s SynthID watermarks. By way of altered .claude/settings.json files in internal code repositories, the group later rigged each launch of Anthropic PBC’s Claude Code to download its SOMBERMEME backdoor.
Google said it disabled the accounts and assets tied to the Gemini misuse it found. Google DeepMind has also used the findings to strengthen its classifiers.
Image: SiliconANGLE/GPT Image 2.6
A message from John Furrier, co-founder of SiliconANGLE:
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.