AI 101 — What is a reasoning model?
A reasoning model is a language model trained — usually with reinforcement learning — to write a long chain of thought before it answers, so it can spend more computation on a hard problem instead of firing back the most likely next sentence.
The label arrived with OpenAI's o1 in September 2024. Before it, a chatbot answered in one pass: read the question, predict the reply token by token. o1 added a visible delay and a trail of working behind the answer. OpenAI described it as trained with reinforcement learning to perform complex reasoning, able to "produce a long internal chain of thought before responding" — good enough to reach the 89th percentile on Codeforces and the top 500 US students in the USA Mathematical Olympiad qualifier, per that same post.
Two years later, "does it have a thinking mode" is a standard question about any new model, and the answer is where labs put their best benchmark numbers.
Why it matters right now
Reasoning is the part of the stack you now pay for by the token. Thinking happens in tokens like everything else, they are billed like output, and several APIs let you dial the budget up or down. That single knob — how long the model is allowed to think — changed the economics of AI products: a support bot reading a receipt should think for zero seconds, and a coding agent untangling a failing test suite should think for a while.
It is also what made agents viable: a model that can plan three steps, notice that step two failed, and try a different route is using its own intermediate output as working memory.
And it is the clearest example of a bigger shift. For a decade, better AI meant a bigger model trained on more data. Reasoning models added a second axis — the same model thinking longer at the moment you ask.
The mental model: two knobs, not one
A base language model has one job — predict the next token. Its "thinking" is the text it produces, and that text is the only scratch space it has.
Reasoning models exploit that. If the model writes out intermediate steps, those steps become part of its input for the next token, so it can build on its own work: set up the equation, compute a partial result, check it, correct it. Chain-of-thought prompting showed in 2022 that asking any model to "think step by step" improves accuracy on multi-step problems. That was a prompt trick. A reasoning model is trained to do it by default, and — this is the important part — trained to do it well.
The training is where the reinforcement learning comes in. Give the model a few thousand math or coding problems with checkable answers. Let it attempt each one, reward it when the final answer is right, and let the optimization push the probability of the successful thinking patterns up. Nobody writes those patterns by hand. DeepSeek's R1 paper showed the extreme version: pure reinforcement learning, no human-written reasoning traces at all, and self-checking behaviour — going back, verifying, trying another approach — emerged on its own.
The second knob is at inference time: how many tokens the model is allowed to spend. OpenAI reported in the o1 launch that performance improves with both more training compute and more time spent thinking at test time. More thinking costs more money and more seconds. That is the trade you are making every time you pick a model or set an effort level.
The exam analogy
Picture two students who have read exactly the same books. One has to answer instantly, out loud, the moment the question is read — a quiz-show contestant. The other gets scratch paper and ten minutes.
The second student will do better on a multi-step problem, and not because they know more. They can write the intermediate values down, spot that an answer doesn't fit, and start again. That is test-time compute, and it is why a reasoning model can beat a larger model on a math problem it was never explicitly trained on.
Now add the coach. During training, the coach only marks the final answer right or wrong — never the working. Over thousands of attempts, the student discovers working habits that reliably land on correct answers. Those habits are sometimes odd, sometimes longer than a human's, and sometimes wrong in ways a teacher would have caught on day one. The model learned what scores, not what a textbook would approve of.
This is also why the gains cluster in math, code and logic: those have checkers. For "write a warmer reply to this customer," there is no automatic right answer to reward, so the training signal is weaker and the improvement is smaller.
Common misconceptions
"It's just 'think step by step' with a longer prompt." Prompting helps any model, but a reasoning model has been trained against a reward for correct answers, and it decides on its own how long to think. You can prompt a standard model to ramble; you cannot prompt it into the reliability.
"More thinking tokens means a better answer." Apple's study of reasoning models found accuracy collapsing past a certain problem complexity, and — the counterintuitive part — reasoning effort rising with difficulty only up to a point, then falling even when the token budget was still available. On low-complexity tasks, standard models actually beat the reasoning ones, the researchers reported. Overthinking a simple question can also talk a model out of an answer it already had right.
"The visible thinking trace is what the model actually did." The trace is generated text, like the answer. It is a summary of the work, not a log of the computation inside the network, and models do not always report accurately what changed their mind. Treat it as useful evidence, not an audit record.
"Reasoning models are strictly better." For classification, extraction, reformatting and short replies, they are slower, pricier and sometimes worse. Most production systems route: a small fast model handles the routine traffic, and the reasoning model is called only when the task actually has steps.
Where to learn more
Start with OpenAI's o1 announcement for the clearest plain statement of the two knobs, then the DeepSeek-R1 paper for what pure reinforcement learning produces, and Apple's "Illusion of Thinking" for the limits. If you want the mechanics of what those thinking tokens are, we covered them in what reasoning tokens are; for how models are trained to reason, see what GRPO is; for the prompting idea underneath it all, what chain of thought is.
Have you caught a reasoning model talking itself out of an answer it had right? Write to us via our contact page.
