Hey there, have you heard about the latest research findings on AI defenses? It turns out that many security teams are investing in AI defenses that may not be as effective as they claim to be. A recent study by researchers from OpenAI, Anthropic, and Google DeepMind revealed some eye-opening results that every CISO should take note of.
In their paper “The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections,” the research team tested 12 AI defenses that boasted near-zero attack success rates. However, they found that most of these defenses could be bypassed with success rates above 90%. This highlights a critical issue – most AI security products are being tested against unrealistic attacker behaviors.
The team specifically looked at prompting-based, training-based, and filtering-based defenses under adaptive attack conditions, and all of them failed to stand up against the attacks. Prompting defenses had success rates of 95% to 99%, while training-based methods had bypass rates of 96% to 100%. The research methodology was rigorous, involving 14 authors and a $20,000 prize pool for successful attacks.
Why are WAFs failing at the inference layer?
Web application firewalls (WAFs) are designed to be stateless, but AI attacks are not. This fundamental difference is why traditional security controls are failing against modern prompt injection techniques.
The researchers tested known jailbreak techniques against these defenses, such as Crescendo and Greedy Coordinate Gradient (GCG), which exploit conversational context and automate attacks, respectively. These attacks are not theoretical – they have working code and can bypass stateless filters.
Each attack exploited different blind spots, but all succeeded due to the assumption of static behavior in the defenses.
Carter Rees, VP of AI at Reputation, highlighted the impact of semantic layer attacks on AI applications, emphasizing the need for detection methods beyond signature-based approaches.
Why is AI deployment outpacing security?
The failure of current defenses is concerning, especially given the rapid pace of AI deployment in enterprise applications. Gartner predicts a significant increase in the integration of AI agents by 2026, while security measures are struggling to keep up.
Adversaries are becoming more sophisticated, with hands-on keyboard techniques that bypass traditional defenses. CrowdStrike’s 2025 Global Threat Report found that the majority of detections were malware-free, indicating a shift in attacker tactics.
Anthropic’s disruption of an AI-orchestrated cyber operation in 2025 highlighted the efficiency and speed of AI-enabled attacks, with traditional campaigns compressing into a matter of hours.
Jerry Geisler, EVP and CISO of Walmart, warned of new security threats introduced by agentic AI, which could disrupt operations and violate regulations.
Four attacker profiles exploiting AI defense gaps
The research identified four attacker profiles exploiting the inference layer, highlighting the need for adaptive security measures. Security through obscurity is no longer effective when attackers can learn and adapt to defenses in real-time.
These attacker profiles range from external adversaries operationalizing attack research to negligent insiders compromising security from within. Each profile exploits different vulnerabilities in AI defenses, emphasizing the need for dynamic security measures.
Key questions for AI security vendors
Before engaging with AI security vendors, security leaders should ask critical questions to assess the effectiveness of their defenses against adaptive attacks. These questions directly address the vulnerabilities identified in the research and can help organizations make informed decisions when procuring AI security solutions.
Conclusion
The research findings from OpenAI, Anthropic, and Google DeepMind shed light on the gaps in AI defenses and the urgent need for adaptive security measures. As enterprises rapidly deploy AI technologies, it is crucial to reevaluate current security controls and ensure they can withstand real-world attacks. The security curve must align with the deployment curve to prevent breaches and safeguard sensitive data effectively.
