Model Collapse

Model collapse is the progressive degradation of an AI model's output quality and diversity when trained on AI-generated data, causing successive model generations to lose coverage of rare but important patterns present in the original human-authored data distribution.

Model collapse—also termed "the curse of recursion"—is a failure mode documented formally in a 2023 paper by Shumailov and colleagues at the University of Oxford and Google DeepMind. When AI models are trained on datasets that include significant proportions of content produced by earlier AI models rather than human-generated originals, each successive training generation progressively narrows the statistical distribution the model has learned. Rare patterns, minority viewpoints, and low-frequency but accurate information are the first casualties; model outputs converge toward a more homogeneous, central mode.

Two compounding mechanisms drive the collapse. First, model-generated data approximates but compresses the training distribution—generative models are biased toward probable outputs and systematically underrepresent the tails. Second, estimation errors accumulate across generations: small biases introduced in generation 1 become the training signal for generation 2, amplifying those biases further. In language models, the downstream result is a drift toward bland, repetitive prose; in image generators, toward visually uniform compositions.

Model collapse matters because the internet—the primary training corpus for frontier models—is rapidly filling with AI-generated content. Multiple analyses published between 2023 and 2025 found measurable and growing fractions of AI-generated text across forums, social media, and news aggregators, with some content domains showing particularly high saturation. If left unaddressed, future training runs risk systematic contamination, producing models with narrowed capability and output diversity even if benchmark scores on common tasks remain stable.

Researchers are exploring several mitigations: watermarking AI-generated content so it can be filtered from future training sets, maintaining curated high-quality human-authored corpora, and developing data-mixing strategies that preserve rare examples. As of 2026, no training organization has announced a complete solution; the problem is widely regarded as one of the primary long-term structural risks to the quality of foundation models.