Do you remember the terrifying shark from Jaws? It struck suddenly, using chaos to cause deadly harm to its victims. According to Forrester, today’s apex predator, generative AI, is similar to that shark when it falls into the wrong hands: always active, always executing, and on a massive scale.
During the IT consultancy firm’s 2025 Security and Risk Summit, Forrester principal analyst Allie Mellen compared generative AI to the chaos agent in Jaws. She emphasized the significant flaws and uncertainties in AI systems, stating that AI is often wrong, and wrong by a large margin.
Failure Rates and Risks
Mellen referenced a study by the Tow Center for Digital Journalism at Columbia University, which revealed that AI models were wrong 60% of the time, resulting in more failed queries than successful ones. Additionally, research from Carnegie Mellon found that AI agents fail 70 to 90% of the time on real-world corporate tasks, with 45% of AI-generated code containing known vulnerabilities.
These alarming statistics highlight the dangers of generative AI as a chaos agent in cybersecurity. The widespread use of unauthorized AI, known as shadow AI, further exacerbates these risks, with 88% of security leaders admitting to incorporating unauthorized AI into daily workflows.
The Impact on Security
AI’s inaccuracies can have severe consequences during security incidents, as demonstrated by an AI-generated map that mistakenly placed shark attacks in Wyoming, a landlocked state far from the ocean. This type of false positive can lead to confusion and mismanagement during critical situations.
Challenges and Solutions
As AI continues to evolve, security professionals must adapt to the changing landscape. By treating AI agents as mission-critical identities, developing AI red team capabilities, and designing security controls to match the speed of AI, organizations can better protect themselves against the risks posed by generative AI.
Ultimately, the key to mitigating these risks lies in eliminating blind trust in automation and legacy infrastructure. By continuously verifying, auditing, and challenging automated systems, organizations can stay one step ahead of potential breaches and attacks.
Forrester’s insights provide a roadmap for navigating the complexities of generative AI in cybersecurity, ensuring that organizations are prepared to face the challenges posed by this emerging technology.
Source: 2025 Security & Risk Summit.
