AI Hacking Skills Near Critical ‘Inflection Point’
▼ Summary
– RunSybil’s AI tool, Sybil, discovered a novel, complex security vulnerability in a customer’s GraphQL deployment, a flaw requiring deep system knowledge that hadn’t been publicly documented.
– The incident illustrates a growing risk: increasingly intelligent AI models are becoming more capable of finding critical software vulnerabilities, which can be exploited for both defense and attack.
– AI researcher Dawn Song notes a recent inflection point, with frontier models’ cybersecurity capabilities drastically improving due to techniques like simulated reasoning and agentic AI, enabling them to find zero-day bugs at low cost.
– To counter this, Song proposes measures like having AI companies share models with security researchers pre-launch and using AI to generate inherently more secure “secure-by-design” code.
– In the near term, the RunSybil team warns that AI’s accelerating ability to generate code and perform computer actions could give hackers the upper hand by accelerating offensive security actions.
The rapid advancement of artificial intelligence is pushing its ability to discover software vulnerabilities toward a crucial threshold, a development with profound implications for global cybersecurity. Experts warn that AI models are approaching an inflection point where their skill in finding security flaws could soon outpace traditional defensive measures. This dual-use nature of the technology means the same powerful tools designed to protect systems can be repurposed to attack them with alarming efficiency.
Consider the experience of cybersecurity startup RunSybil. Last November, their AI tool, Sybil, identified a previously unknown weakness in a client’s deployment of federated GraphQL, a system for managing data access. The flaw was exposing confidential information. What surprised the founders was the sophisticated reasoning required to find it; the AI needed a deep understanding of how multiple complex systems interacted. The team confirmed this was a novel discovery, not a known issue documented anywhere online. For them, it signaled a leap in AI’s analytical capabilities.
This incident underscores a broader, accelerating trend. The intelligence that enables an AI to detect a vulnerability is precisely what a malicious actor could use to exploit it. According to computer scientist Dawn Song, who specializes in AI and security at UC Berkeley, recent breakthroughs have dramatically enhanced models’ cyber abilities. Techniques like simulated reasoning, breaking problems into smaller parts, and agentic AI, which allows models to perform actions like running software tools, have been key drivers.
“The cybersecurity capabilities of frontier models have increased drastically in the last few months,” Song states. “This is an inflection point.” Her research provides concrete evidence. Using a benchmark called CyberGym, which contains over 1,500 known vulnerabilities in open-source projects, her team tracked significant progress in a short time. In mid-2025, a leading model could find about 20 percent of these flaws. Just a few months later, an updated version identified 30 percent. “AI agents are able to find zero-days, and at very low cost,” she notes.
This accelerating capability creates an urgent need for new defensive strategies. Song advocates for a multi-pronged approach. One proposal is for AI developers to share advanced models with security researchers before public release, allowing those experts to use the AI to find and patch bugs proactively. Another, more foundational idea is to reinvent how software is built. Song’s lab has demonstrated that AI can be used to generate code that is inherently more secure than standard programmer output, a “secure-by-design” philosophy that could strengthen defenses over the long term.
However, in the immediate future, the balance of power may shift toward attackers. The RunSybil team observes that AI excels at the core tasks of hacking: generating code and executing actions on computer systems. If those capabilities accelerate, that means offensive security actions will also accelerate, giving hackers a potent new advantage. The race is on to ensure that defensive applications of AI can keep pace with its offensive potential, a challenge that will define cybersecurity for years to come.
(Source: Wired)
