There is a palpable sense of apprehension among cybersecurity professionals and scholars that AI will revolutionize cyber conflict. A slew of recent reports conjure images of supercharged, AI-automated cyber attacks. The worst of these fears seemed to come true last month when OpenAI’s latest model escaped a supposedly secure sandbox and, unbeknownst to its handlers, hacked into machine learning platform HuggingFace to cheat on a test. Not only did the AI model autonomously discover a zero-day vulnerability in the sandbox that was supposed to keep it locked down, but it also successfully breached a large corporate network. However, one important twist in this story is that HuggingFace almost instantly detected the intruders—with the help of AI agents.
This episode highlights an important aspect: cyber offenders do not exist in a vacuum; rather, they are constantly competing with cyber defenders. How that competition plays out will shape the future of cyber conflict, and its impact both in regular and irregular warfare. Unfortunately, there is still little systematic analysis of the new challenges and opportunities AI automation brings for both offense and defense.
On the one hand, AI automation could massively enhance the effectiveness of cyber operations and campaigns below the threshold of war as some have predicted. If true, states would be able to achieve even more strategic gains through irregular warfare than U.S. national defense strategy assumes. The old dream of cyberwar, throwing the enemy without firing a shot, might finally come true. But the rise of AI makes that dream ever harder to attain.
AI Enhances Efficiency but not Effectiveness
In a new article published in International Security, I argue that these expectations are likely misplaced. Contrary to prevailing expectations, cyber defense likely has more to gain from AI automation. While AI-powered attacks make for dramatic headlines, a level-headed examination shows core offense challenges reflect the weaknesses of AI models whereas defense tasks speak to its strength. Cyber offense requires creativity and deception. Cyber defense, on the other hand, requires accurate and speedy detection and mitigation. AI excels at detection but notoriously struggles with deception and creativity. Consequently, there is an automation gap between offense and defense. AI automation makes offensive cyber operations more efficient, but not necessarily more effective. Sometimes AI automation may even make offensive cyber operations less effective.
AI automation may make operations faster yet it also adds a new element of uncertainty as AI may use generic tools or heavy-handed tactics that are easy to detect and are prone to unpredictable behavior. In addition, large language models tend to hallucinate. They make up things that aren’t there. Indeed, the OpenAI case illustrates just this unpredictability: an AI model tasked with demonstrating their capabilities within a secure environment decided to cheat and escape. In this case, the unpredictable behavior involved breaching a network, demonstrating the model’s capabilities.
Crucially, however, this behavior did not at all align with the objectives of the model’s handlers. Hence, it is just as possible that a model tasked with breaching a specific network to obtain specific data unexpectedly targets a third party, such as a cloud provider or software vendor used by the targeted organization. Or what if the model cheats and generates an approximation of the data it is supposed to steal? How and when would its handlers find out? Even if none of these pathologies happen and the model tamely stays on the assigned task, it may behave in a way that increases failure risks.
The reason why HuggingFace quickly detected OpenAI’s rogue agent was the spray-gun approach the model took of attempting to detect and exploit vulnerabilities at scale with 17,600 actions executed within four days. AI models far outmatch the efficiency and speed of any human team, yet this approach is also all but sure to alert the victim to one’s presence, just as happened in the OpenAI case.
AI agents are rapidly getting better at breaching systems, but when it comes to going after specific systems for specific goals, using AI may make it less likely one achieves those goals. In short, offensive AI automation injects even more uncertainty into a process already full of it. Human intervention can help prevent such errors and guide deceptive efforts, but forfeits the speed advantage of automation. Crucially, creativity and deception become increasingly essential for success as stakes increase in cyber conflict. Indeed, they are the hallmarks of the most advanced, state-sponsored actors.
The Automation Gap Favors the Defense
Consequently, AI automation has the least revolutionary potential at the high end of cyber conflict, namely state-sponsored cyber operations. In other words, state-sponsored actors may be able to operate faster, but at a higher risk of getting caught or of something going wrong. Those sacrifices in effectiveness likely outweigh the efficiency gains. In short, AI automation in cyber conflict will probably not make a “cyber Pearl Harbor” or other damaging cyber attacks more likely. It is also unlikely to revolutionize cyber campaigning short of war for the same reasons outlined above. Actors may be able to run operations more efficiently. If these operations are not significantly more effective, however, they will continue to struggle to make a difference strategically.
In fact, the opposite outcome is more likely due to significant opportunities for the defense. AI models excel at pattern recognition at speed and scale. This capacity provides significant potential to improve vulnerability and intrusion detection, two key defensive tasks. Moreover, the larger their training datasets, the better AI models tend to perform. Hence, the largest organizations that have the most to lose from cyber attacks also have the most to gain from AI-automated defense. In particular, cybersecurity vendors with huge datasets of telemetry gathered from their clients have a key competitive advantage in training dedicated defensive models.
Implications: An Offensive Window of Opportunity and How to Close It
As both offense and defense adopt AI, there are three consequences for cyber conflict. First, there is a significant window of opportunity for offenders to exploit delayed adoption by defenders. Cybersecurity and cyber risk management has become heavily bureaucratic with a focus on compliance over practical security and correspondingly slow and costly compliance processes. This approach will struggle to adapt to rapid change, providing significant opportunities for AI-enhanced offenders to exploit victims lagging behind in adoption. The likely short-term result is a target-rich environment full of low-hanging fruit for cyber campaigning short of war. One important dimension of this mismatch is the unintended impact of cybersecurity safeguards built into frontier models. When HuggingFace detected the intrusion into its networks, its attempts to analyze the latter with Anthropic’s Claude and Fable got blocked by those models’ safeguards. Instead, it had to fall back to using Chinese open-weight models. Ironically, the safeguards intended to prevent offensive actors from harnessing the power of leading edge models not only ended up hampering defenders, they also handed China a victory in the AI race.
Second, and counterintuitively, AI disruption may increase long-term stability. The short-term offensive window of opportunity will likely give way to a long-term defensive advantage as adoption becomes more universal. As defenders reap the benefits of AI-automated defense, cyber offense against large and capable organizations will become even harder than it already is. The short-term offensive bonanza will likely fizzle out, making way for a significantly more challenging environment with fewer and fewer vulnerable targets. That means interstate cyber conflict short of war will likely become even lower in intensity than it already is.
There is a flipside to this situation, however, as offenders may take more risks and attempt to go all-in. To avoid having their tradecraft profiled and neutralized by defensive AI models across multiple operations, offensive actors might pour all their skills and efforts into one massive attack to maximize impact while they can. Sponsors of cyber operations face a trilemma between speed, intensity, and control. The more they improve one or more of these factors, the more they tend to lose across the others. Consequently, such ‘intensity-maxing’ also brings maximum risks of losing control, leading to unintended consequences and corresponding instability. Identifying such attempts before adversaries can execute should be a priority for intelligence collection efforts.
In short, there is a potential golden age for cyber defense—but it will not come by itself, nor will it ‘solve’ cybersecurity. Determined and well-resourced threat actors will still find vulnerabilities and get into systems, harming U.S. national interest. To make it as hard as possible for them, policy should focus on three priorities. First, move from speculation about possibilities to systematic testing of how AI automation impacts real-world outcomes in cyber conflict. Assessments about model capabilities by frontier labs themselves are necessarily biased. Independent assessments are crucial, whether done internally in government or in collaboration with academia.
Second, facilitate and support the adoption of AI to augment defenses along the lines described above. Not only the protection of government systems is important, but rollout across the private sector. The most attractive attack surface for AI augmented offenders is likely small and medium enterprises, which already struggle to keep up with traditional threats. Domestically, reducing red tape and compliance burdens will be key to support them. Internationally, U.S. allies with weaker cyber defenses will also become even more vulnerable than they already are. Two-pronged security cooperation can help reduce that vulnerability. Targeted support of allies with AI-enabled defensive solutions will improve not only their security, but also make them a less attractive staging ground for operations targeting the U.S. ‘hunt forward’ operations to detect and disrupt adversary activities in allied states can further reduce this risk. Of course, a precondition for this to work is a constructive relationship with allies built on mutual trust.
Third, build active defenses that exploit the vulnerabilities of AI agents themselves. Precisely because AI models are prone to fall for deception, a defense based around it is a winning strategy. That means building systems that mislead intruders, bog them down with token-burning tasks, and trap them with techniques like prompt injection. Meanwhile, it is crucial to test one’s own defensive agents against such techniques to limit their vulnerability as much as possible. In the era of Agentic AI, the deception skills already concentrated in the Irregular Warfare and Intelligence communities will become a crucial defensive asset.
Lennart Maschmeyer is an Assistant Professor of Cybersecurity Policy at the Georgia Institute of Technology, Jimmy and Rosalynn Carter School of Public Policy. His research has been published in leading journals in Political Science and International Relations. Dr. Maschmeyer is the author of the award-winning book "Subversion: From Covert Operations to Cyber Conflict" (2024, Oxford University Press).
Main image: Military cyber operators at work. As AI reshapes the offense-defense balance, the detection edge increasingly favors the defender. (U.S. Space Force photo by David Grim, U.S. Space Force Combat Forces Command)
The views expressed are those of the author(s) and do not reflect the official position of the Irregular Warfare Initiative, Princeton University’s Empirical Studies of Conflict Project, the Modern War Institute at West Point, or the United States Government.
If you value reading the Irregular Warfare Initiative, please consider supporting our work. And for the best gear, check out the IWI store for mugs, coasters, apparel, and other items.
