AI 101 — What are reasoning tokens?

Share
AI 101 — What are reasoning tokens?

Reasoning tokens are the hidden text a model generates to think through a problem before giving you its answer — invisible, billed to your account, and the reason modern AI can suddenly solve hard questions.

When you ask a regular chatbot a question, it reads your words and starts writing a reply immediately. When you ask a reasoning model — OpenAI's o3, DeepSeek's R1, or Google's Gemini with "thinking" enabled — something different happens first: the model spends time generating an internal monologue. It breaks the problem down, tries approaches, catches its own mistakes, and only then speaks to you. Those hidden words are reasoning tokens.

Why it matters right now

Reasoning tokens are everywhere in 2026, even if you have never heard the term. Every major API provider now charges separately for them: OpenAI bills reasoning tokens at a different (often higher) rate than regular output tokens. When you use a coding agent like Cursor or GitHub Copilot in "think" mode, or when Anthropic's Claude works through a multi-step analysis, the invisible thinking that happens behind the scenes is made of reasoning tokens. They are the single biggest reason AI got dramatically better at math, code, and logic in the past two years — and the single biggest reason API bills went up for hard problems.

Understanding them matters because they sit at the intersection of two things readers care about: how AI actually reasons, and what you are paying for. If you have ever looked at a usage dashboard and wondered why a "simple" question cost ten times more than usual, reasoning tokens are probably why.

The mental model

Think of a reasoning model like a student allowed to think on scratch paper before writing the final answer on the exam sheet. The scratch paper is the reasoning tokens — the model's visible work. You, the teacher, only see the final answer on the sheet. But the scratch paper existed, it took time, and (in the AI version) it costs money.

Here is the key insight: reasoning tokens are not a different kind of token. They are the same tokens the model always generates — the same word-prediction machinery that produces your visible reply also produces the invisible thinking. The difference is architectural: reasoning models are trained to produce a hidden block of text first (sometimes called a "thinking block" or "reasoning trace"), and then a final answer after. The thinking block can range from a single sentence to thousands of tokens, depending on how hard the problem is.

In practice, this means reasoning tokens scale with difficulty. Ask a reasoning model "What is 2 + 2?" and it might burn two tokens thinking. Ask it to prove a theorem, debug a complex codebase, or plan a multi-step business strategy, and the thinking trace can stretch to thousands of tokens — all invisible, all billed.

An everyday analogy

Imagine a chef who, before plating your meal, first talks through the entire recipe out loud: "OK, I need to sear the salmon skin-side down for four minutes, check that the pan is hot enough, oh wait I should have started the rice first..." That narration is the reasoning tokens. You never hear it — it happens in the kitchen, not at your table. But it is what makes the dish good. And if the chef were billing you by the word, that off-menu monologue is where your bill spikes.

The crucial detail: a basic model skips the monologue entirely. It just plates. That works fine for simple orders (salad, toast), but for complex dishes (a twelve-course tasting menu), the chef who thinks aloud produces a better result — at the cost of more words and more time.

Common misconceptions

"Reasoning tokens are just chain of thought." Close, but not quite. Chain of thought is a technique — you prompt the model to show its work, and it does. Reasoning tokens are an implementation: the model was trained to produce a hidden thinking trace automatically, without you asking. Think of chain of thought as "ask the student to show their work," and reasoning tokens as "the student has been trained to always show their work — but then the teacher hides it from you." We covered what is chain of thought in an earlier explainer.

"I'm paying for extra tokens for no reason." Not exactly. Reasoning tokens demonstrably improve accuracy on hard problems — the o1 launch showed accuracy on the AIME math competition jump from 13 percent (GPT-4o) to 83 percent (o1) when reasoning was enabled. The cost is real, but so is the capability. For easy questions, most reasoning models skip or minimize the thinking block automatically.

"Reasoning tokens are always hidden." By default, yes — OpenAI and Anthropic hide the thinking from the user. But DeepSeek's R1 showed its full reasoning trace to users by default, and some tools let you toggle visibility. The trend is toward hiding: labs worry that showing the thinking leaks training signals, allows prompt injection through the reasoning trace, and gives competitors a shortcut to copying the model's abilities.

"Longer thinking always means better answers." Not necessarily. Models can think in circles, reason confidently toward wrong conclusions, or spend hundreds of tokens re-examining an already-solved step. Research from 2025 showed that beyond a certain thinking length, accuracy plateaus or even drops. More thinking is a tool, not a guarantee.

The cost equation

Reasoning tokens directly affect what you pay. On OpenAI's API, o3-mini reasoning tokens are billed at a higher per-token rate than standard output tokens. A 500-token reasoning trace on a complex coding problem might cost more than the final answer itself. For consumer-facing products, this translates to subscription pricing tiers, usage limits, and the occasional surprise bill. As one recent analysis put it, reasoning tokens have become a "model-specific API contract" — the terms under which a model agrees to think on your behalf.

For builders, this creates a real design decision: enable reasoning for accuracy, or disable it for speed and cost. Most products now offer a toggle, and the best ones auto-select based on question difficulty.

Where to learn more

OpenAI's reasoning guide explains how their models use reasoning tokens and how the billing works. The original o1 economics post breaks down when extra thinking actually pays off. And our own What is chain of thought? covers the technique that reasoning tokens automate.

When a model bills you for thinking you cannot see, should you be able to read the scratch paper? Write to us via our contact page.

Get SiliconNoon in your inbox

One email a day with the AI stories that matter — what happened, the numbers, and why it matters. No spam, unsubscribe in one click.

Read more

AI 101 — What is model quantization?

AI 101 — What is model quantization?

A large language model is a giant bag of numbers. Quantization is the trick of shrinking those numbers so the model takes up less memory and runs faster — usually with only a small, barely-noticeable drop in quality. If you have ever tried to run a powerful open-weight model on your own laptop and watched it choke, quantization is the reason some people can and some can't. It is also why a phone can now hold an assistant that, a couple of years ago, needed a rack of servers.

AI 101 — What is synthetic data?

AI 101 — What is synthetic data?

Synthetic data is information generated by a computer — not collected from the real world — that mimics the patterns, structure, and statistical properties of real data well enough to train an AI model. It is one of the most important tools in modern AI development, and one of the least understood. Every major lab is using it, many products depend on it, and the debate over whether it is good enough to replace real-world data is shaping where the field goes next. Why it matters right…

AI 101 — What is a token in AI?

AI 101 — What is a token in AI?

A token is the chunk of text an AI model actually reads and writes — not a word, not a character, but a piece of a word: "un", "believ", "able". Every language model, from the smallest phone-side model to the biggest frontier system, does its work by predicting one token after another. If you understand tokens, you understand why AI bills look the way they do, why some languages cost more than others, and why a model can count the letters in "strawberry" and still get it wrong. It is also the…

AI 101 — What is an embedding?

AI 101 — What is an embedding?

An embedding is a list of numbers that represents the meaning of something — a word, a sentence, a photo, a song — so a computer can compare meanings using arithmetic. The rule is simple and surprisingly powerful: things that mean similar things get similar lists of numbers. That's the whole idea, and it's the foundation of semantic search, recommendation feeds, and most of what people call "AI memory." Why it matters right now Embeddings are the invisible layer under nearly every AI feature…

AI 101 — What is a diffusion language model?

AI 101 — What is a diffusion language model?

A diffusion language model writes text the way an image generator paints a picture: it starts with a block of noise and refines the whole thing over several passes, instead of typing one word at a time left to right. Almost every AI you have used does the typing. This is the main rival design, and in the last week it went from research curiosity to something running real phone calls and coding agents. The reason it is in the news is Inception's Mercury 2.5 release, which the company calls the…