AI 101 — What is model collapse?

Model collapse is what happens to a generative AI model that learns mostly from other AI models: it slowly stops representing the full range of real data, and the loss is not recoverable by training longer.

That is the plain version. The precise version, from the 2024 study that named the effect, is that "indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear" — published in Nature by Ilia Shumailov and colleagues at Oxford, Cambridge and Imperial College London.

Why it matters right now

Every frontier lab now trains on at least some synthetic data, because the public internet has been scraped about as thoroughly as it can be. Synthetic data is cheap, endless, and legally clean. It is also a copy of a copy, and copies degrade.

The clearest evidence of the cost came this month. AI labs have started buying the data remains of dead startups — old code, emails, chat logs, documents — because a real archive preserves the messy judgment calls that synthetic data smooths away, MIT Technology Review reported on October 13. Wind-down firms like SimpleClosure and Sunset now broker those archives; SimpleClosure's Asset Hub has handled more than 150 deals and expects to return over $10 million to startups by the end of 2026, with 47% of its deals now including code or data sales. Google paid $10 million for Spirit Airlines' bankruptcy archive of 100 million emails and 500 million Teams messages, Reuters reported in August.

Nobody is buying bankruptcy emails because they are precious. They are buying them because real data is the anchor that keeps a model from drifting off the distribution it was supposed to learn.

The mental model

Picture any training dataset as a shape with a fat middle and long, thin tails. The middle is the common case: ordinary sentences, normal photos, typical code. The tails are the rare stuff — odd phrasing, minority dialects, the weird edge case that only shows up once in ten thousand examples.

A model trained on human data learns both, because both are in there. A model trained on another model's output learns mostly the middle, because that is what a model produces most readily: the most likely next token, the most typical image. Its rare examples were already compressed away by the previous model. Train on that output, and the tails get thinner again.

The researchers describe two stages. In early collapse, the model loses the tails and keeps looking fine — benchmarks barely move, because benchmarks test the middle too. In late collapse, the model's outputs converge: same phrasings, same images, then outright errors, as it starts misreading its own compressed version of the world as the world.

The photocopier analogy

A photocopy of a photocopy. The first copy is slightly softer than the original, and nobody notices. By the tenth copy the fine print is gone, but the bold headline — the high-contrast part — is still perfectly legible, so the page looks fine at a glance.

That is exactly the failure signature. The model still answers obvious questions well. What it has silently lost is the rare, low-contrast detail: the unusual name, the minority usage, the long-tail case your product actually gets asked about. And unlike a photocopier, you cannot un-copy it by running the machine longer — the information is already gone from the training set.

Common misconceptions

"Training on AI data destroys any model." Not quite, and this is the important correction. A 2024 paper by Matthias Gerstgrasser and colleagues showed the outcome depends on one design choice: whether synthetic data replaces real data or accumulates alongside it. When synthetic generations are accumulated with the original real data, test error stays bounded no matter how many generations you add; when real data is replaced, error grows without bound (arXiv). The rule is accumulate, don't replace.

"It is sudden." It is usually gradual and invisible for several generations, then accelerates. The Nature team found the tails were gone well before the headline numbers moved.

"It only affects chatbots." The original study reproduced collapse in image models and simple statistical models too. Any generative model trained on its own output is exposed.

"So synthetic data is unusable." No. Labs use it heavily, with filtering, deduplication and a real-data floor. What they rarely publish is the ratio — which is why the buying spree for genuine archives is the most honest signal we have about how much real data they think they still need.

Where to learn more

The Nature paper is open access and readable if you skip the proofs. The Gerstgrasser paper carries the accumulate-versus-replace result that practitioners actually act on, and the MIT Technology Review report shows what the economics look like when real data gets scarce. If you want a plain-language tour of the wider phenomenon first, IBM's explainer is a decent starting point: What is model collapse?.

Related reading: the data side of this story is covered in What is synthetic data?, the training side in What is continual learning?, and the thing that actually degrades — the weights — in What are model parameters?.

Have you seen a model get noticeably blander after a version update? Write to us via our contact page.

AI 101 — What does GPT stand for?

GPT stands for Generative Pre-trained Transformer — the three-word recipe behind OpenAI's language models, from GPT-2 to today's GPT-6. It is also the most-abbreviated phrase in AI: most people who type "GPT" into a search box are really asking what the letters mean and why they keep appearing in the news. Why it matters right now Every model name in the headlines is a small riddle.

AI 101 — How do AI video generators work?

An AI video generator is a model that starts with a grid of random noise and repeatedly refines it into a moving picture that matches your prompt — the same trick image generators use, stretched across time. Nothing is filmed, and nothing is stitched together from clips the model has seen. The pixels in the finished video were invented, frame by frame, all at once. Why it matters right now Kling 4.0, Kuaishou's flagship video model, went fully live on October 15 with up to 30 seconds of…

AI 101 — What is AGI?

AGI — short for artificial general intelligence — is a hypothetical AI system that could do any intellectual job a person can do, instead of being good at one narrow task. Every AI you can actually use today is narrow: it writes, translates, spots patterns in scans, plays games — each system built for its lane. AGI names the destination where one system covers all the lanes.

AI 101 — What is reinforcement learning?

Reinforcement learning is how a machine learns by trial and error: it tries things, notices which attempts earned a reward, and drifts toward the behaviour that earns more of it. There is no answer key involved. In the supervised learning most people picture, someone hands the model millions of labelled examples and it learns to copy them. In reinforcement learning, nobody says what the right move is — the machine acts, gets a number back telling it how well that went, and adjusts.

AI 101 — What is LLM-as-a-judge?

If you read a week of AI news, you'll notice almost nobody grades models by hand any more. When a lab says its new model is better at writing, summarising or handling customers, the number usually came from another AI. An LLM judge is a language model used to grade the work of another model — or of your own AI feature — when no piece of code can decide whether the answer was good.

AI 101 — What is multimodal AI?

Multimodal AI is a model that handles more than one kind of input — text, images, audio, video — inside the same system, rather than one model per sense stitched together. "Modality" is just the academic word for a channel of information. Text is a modality. A photograph is a modality. A voice recording is a modality. Multimodal AI reads more than one of them at once and reasons across them. That last part is the whole trick.

AI 101 — AI vs machine learning: what's the difference?

Artificial intelligence is the goal — getting a machine to do things we would call intelligent — and machine learning is the technique the field now overwhelmingly uses to get there: instead of a programmer writing the rules, the system is handed examples and adjusts itself until its answers get better. The two terms get swapped for each other in almost every story you read, including ours. That is mostly harmless, and occasionally it hides the interesting part.

AI 101 — What is continual learning?

Continual learning — also called lifelong learning — is the ability of an AI model to keep learning new information over time without losing the skills it already has. Instead of being frozen after training, or wiped and rebuilt whenever the world changes, a continually learning model gets smarter as it goes, accumulating knowledge the way a person does. Most AI models today do the opposite.