AI 101 — What is a context window?

Share
AI 101 — What is a context window?

A context window is the amount of text a language model can look at at one time — its working memory. It is measured in tokens (roughly three-quarters of an English word each), and it caps how much of your prompt, your conversation, or your document the model can actually see when it writes a reply. Everything you want the model to consider must fit inside that window, or it gets cut off, summarized, or ignored.

Why it's in the news

Context size has quietly become one of the biggest selling points in AI — and this week is a good example. Upstage shipped Solar Pro 4 with a 512K-token context window, and DeepSeek pushed V4 Pro to general availability with a 1-million-token window. Those numbers sound like marketing trivia, but they decide what you can actually do with a model: a 200K window holds roughly 150,000 words — about 500 pages — enough for a whole codebase directory, a full legal contract, or a long technical manual. A million tokens is a bookcase. The industry moved from GPT-4's 8,000 tokens in March 2023 to a million in less than a year, and every jump has unlocked a new class of task: chat became document review, then document review became "read my entire repository and fix the bug."

Businesswoman in formal attire reviewing and organizing documents at an office desk.

The mental model

Think of the model as a worker with a finite desk. Your prompt is everything you lay on the desk: the question, the attached files, the conversation history, the instructions. The model can only work with what is physically on the desk — it reads across all of it, weighs it, and writes its answer using it. Anything that doesn't fit never gets seen. When the desk is nearly full, the worker has options: you can summarize what's there to make room (condensing earlier messages), you can shove a file off the edge (it's gone from the model's view), or you can move to a bigger desk — a model with a larger context window. The catch is that a bigger desk is more expensive to maintain and slower to search, which is why context size is a genuine engineering trade-off, not just a spec-sheet number.

An everyday analogy

Imagine a chef cooking from a stack of recipes. The recipes she can lay out on the counter at once are her context window: with a small counter, she can follow one dish at a time and must put completed recipes away before starting the next. Give her a huge counter, and she can have the appetizer, the main course, the dessert, the wine notes, and the guest list all in front of her — she can cross-reference them, keep the whole meal consistent, and remember that the guest of honor is allergic to garlic. But a kitchen with unlimited counter space still has limits: the ingredients in the pantry (the model's training knowledge) are not on the counter, and the more recipes she spreads out, the harder it is to notice she's misread one in the middle. The context window is the counter, not the pantry — and most AI misunderstandings come from confusing the two.

Common misconceptions

"A bigger context window means a smarter model." No. It means a model that can see more at once — but research has shown that models recall details best from the start and end of long inputs and often miss things buried in the middle. A huge window doesn't guarantee careful attention to everything inside it; it just makes it possible.

"The model remembers what I told it." Not after the session. The context window is working memory, not long-term memory: when a conversation ends, the window is discarded, and the model retains nothing unless it was saved into its training data. You can paste a project's history into a new session and the model will act as if it knows it — but that's the window, not a memory.

"Tokens are words." Roughly three-quarters of a word, on average — and the conversion varies wildly: "the" might be one token while a long technical term eats several. That's why context sizes in tokens (200K, 1M) translate to fewer words than they sound like.

"Long context is free now." It's cheaper than it was — early long-context models charged premium rates for the extra tokens — but every token in your prompt costs money and compute time. Filling a million-token window on every request is expensive, which is why real products summarize aggressively instead of dumping everything in.

Where to learn more

IBM has a clean primer on context windows and tokenization if you want the mechanics. Taskade's history of the context window is the best single page for the milestone numbers — when each lab hit 100K, 200K, and 1M tokens. Anthropic's original 100K announcement and Google's Gemini 1.5 launch post are the primary sources for the two biggest jumps. And this week's launches — Upstage's 512K Solar Pro 4 and DeepSeek's 1M-token V4 Pro — show where the race stands right now.

What's the longest document you'd trust a model to read in one go — and where do you draw the line? Write to us via our contact page.

Source: Anthropic — 100K context windows

Get SiliconNoon in your inbox

One email a day with the AI stories that matter — what happened, the numbers, and why it matters. No spam, unsubscribe in one click.

Read more

AI 101 — What is chain of thought?

AI 101 — What is chain of thought?

Chain of thought is the step-by-step reasoning a language model generates before it gives a final answer — the "show your work" part of AI problem-solving. Instead of jumping straight from question to answer, the model writes out its intermediate steps, and that simple change measurably improves how well it handles math, logic, and anything with multiple stages. The idea surfaced in a January 2022 paper from Google Research: the authors found that asking a model to work through a problem aloud…

AI 101 — AI vs machine learning: what's the difference?

AI 101 — AI vs machine learning: what's the difference?

Artificial intelligence is the goal — getting a machine to do things we would call intelligent — and machine learning is the technique the field now overwhelmingly uses to get there: instead of a programmer writing the rules, the system is handed examples and adjusts itself until its answers get better. The two terms get swapped for each other in almost every story you read, including ours. That is mostly harmless, and occasionally it hides the interesting part.

AI 101 — What is reinforcement learning?

AI 101 — What is reinforcement learning?

Reinforcement learning is how a machine learns by trial and error: it tries things, notices which attempts earned a reward, and drifts toward the behaviour that earns more of it. There is no answer key involved. In the supervised learning most people picture, someone hands the model millions of labelled examples and it learns to copy them. In reinforcement learning, nobody says what the right move is — the machine acts, gets a number back telling it how well that went, and adjusts.