Alibaba researchers discover a DoS vulnerability in reasoning AI models

Researchers from Alibaba Discovered DoS Vulnerability in Reasoning AI Models

At the ICML 2026 conference in Seoul in early July 2026, researchers from Zhejiang University and tech giant Alibaba presented a new type of attack on reasoning AI models: specially distorted prompts drive DeepSeek-R1, OpenAI's GPT-o3, and Google's Gemini 2.5 Flash into infinite reasoning loops, inflating response sizes by dozens of times and creating a denial-of-service risk for commercial AI services.

How the Attack Works

Modern reasoning models don't produce an answer instantly — they generate an internal monologue, step by step analyzing the task before a final answer. Researchers turned exactly this feature into a vulnerability.

The team took 940 tasks from three mathematical benchmarks and decomposed each one using an LLM into logical premises and a final question. A genetic algorithm applied "mutations": shuffled premises between tasks, added unrelated conditions, removed key data without which the task becomes unsolvable, swapped final questions. After each round, the system selected variants that maximized response bloat and triggered uncertainty markers: "but," "wait," "maybe," "alternatively." Five iterations — and the algorithm produces a set of prompts specifically tuned to each model.

Key facts:

• Tested models: DeepSeek-R1, Qwen3-Thinking (Alibaba), GPT-o3 (OpenAI), Gemini 2.5 Flash (Google)
• Maximum response length increase: 26.1× — DeepSeek-R1 on the MATH dataset
• Original dataset: 940 tasks from three mathematical benchmarks
• Attack works through public API — access to model weights not required
• Prompts created by a cheap auxiliary model work against expensive closed systems

Why Reasoning Models Turned Out to Be Vulnerable?

Reasoning models are priced by token count: the longer the reasoning chain, the higher the server load and the more computational resources the provider spends on each request. If such an attack is run at industrial scale, legitimate users will experience sharp slowdowns or complete service unavailability. The effect is reproduced not only on mathematics — authors tested tasks in programming, scientific reasoning, and dialogue scenarios, and in all cases recorded significant response lengthening.

"Our results show that overthinking is not an isolated phenomenon of specific models, but a general vulnerability of modern reasoning systems,"

Wei Cao, a graduate student at Zhejiang University, wrote in a letter to IEEE Spectrum.

An additional risk is the portability of the attack across models. Malicious prompts generated by a cheap open model work effectively against expensive closed systems, reducing attack costs to a practically feasible level.

What This Means

The vulnerability turned out to be systemic: it reproduces on all four tested reasoning models regardless of developer and architecture. The authors emphasize that the goal of the work is to document the existence of the vulnerability, not create a ready-made DoS tool. Rate limiting, pricing policies, and existing filters contain the threat but do not eliminate it. A systemic solution will require work at the architecture level: limiting reasoning chain lengths, detecting "empty loops," and filtering logically contradictory input data.

Frequently Asked Questions

Which models turned out to be vulnerable to the attack?

The study tested four reasoning systems: DeepSeek-R1, Qwen3-Thinking from Alibaba, GPT-o3 from OpenAI, and Gemini 2.5 Flash from Google. All four showed significant response lengthening on distorted prompts.

How realistic is the threat of industrial DoS?

The authors acknowledge limitations: rate limiting, pricing, and existing provider filters reduce the practical effect. The study documents the existence of the vulnerability and the attack vector — but does not demonstrate a ready-made tool with a guaranteed result.