AI 101 — What is context engineering?
Context engineering is the work of deciding what text a model sees before it answers: the instructions, the retrieved documents, the tool descriptions, the conversation so far, the notes from last week — and, just as importantly, what to leave out. Prompt engineering is one slice of it. Context engineering is the whole tray.
Every time a model writes a reply, it sees exactly one flat block of text and nothing else. It does not remember your last session, cannot see inside your database, and has no idea which file you have open. If a fact matters for the answer, something has to put that fact into the block first. That "something" is the job: choosing, ordering, compressing and refreshing the material so the model has what it needs and as little else as possible.
Why it matters right now. The term went mainstream when Anthropic's engineering team published a September 2025 post describing it as "the natural progression of prompt engineering," on the grounds that building with models is now less about finding the right words and more about answering "what configuration of context is most likely to generate the desired behavior," Anthropic wrote. The reason is agents. A chatbot answers one question from one prompt; an agent loops — call a tool, read the result, decide the next step — and every turn drops more text into the same window: file contents, search results, error messages, half-finished plans. Left alone, the context fills with things that were useful twenty steps ago.
And filling it is not free. In July 2025, Chroma published a technical report that ran 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, through tasks at increasing input lengths; it found that performance grows "increasingly unreliable as input length grows," even on simple tasks, according to the report. Models do not read all tokens equally. Adding material can make an answer worse, not better — which turns "what goes in the window" into an engineering decision rather than a housekeeping one.
The mental model. Treat the context window as a budget of attention, not a bucket with spare room. Every token you spend — a full API response, three near-duplicate search results, a system prompt that restates itself — is a claim on the model's limited focus, and it competes with the sentence that actually contains the answer. Anthropic's guidance reduces to one line: find the smallest set of high-signal tokens that maximizes the chance of the outcome you want. Engineers apply that at four points. Compaction: when a conversation nears the limit, summarize it and start fresh with the summary, keeping decisions and open problems while dropping raw tool output. Notes: have the agent write progress to a file outside the window and read it back later. Just-in-time retrieval: keep a file path or a query instead of the whole document, and load the contents only when the task needs them. Sub-agents: hand a narrow job to a separate loop with its own small window, and return only the answer.
The analogy. Picture a kitchen counter at dinner service. The counter is the window; the ingredients are tokens. A cook with a clean counter and the six things this dish needs works fast. A cook who has piled every ingredient in the fridge onto the counter has all the same food and no room to cut anything — and will eventually knock the salt into the sauce. More on the counter is not more capability. It is less space to work, and a higher chance the wrong thing ends up in the pan. Context engineering is mise en place for models: decide the dish, lay out only what it needs, clear as you go.
Common misconceptions. First, "it's just prompt engineering with a new name." A prompt is one input; context includes tool definitions, retrieved documents, memory, and the accumulating history of a loop — much of it assembled by software, not typed by a person. Second, "a bigger window fixes it." Long-context models do hold more, but the Chroma results show the decline tracks input length itself, so a million-token window buys you room to be careful, not permission to be sloppy. Third, "more context is always safer." Irrelevant text is not neutral: it competes for attention and can pull an answer toward the wrong passage. Fourth, "this is only for people building agents." Anyone who pastes a document into a chat box, or starts a fresh thread because the old one went strange, is already doing a manual version of it.
Where to learn more. Read Anthropic's post for the working definitions and the compaction, note-taking and retrieval patterns in their own words. Read the Chroma report if you want the measurements behind "long context degrades." Then watch any agent you use: the moment it starts repeating itself or losing the thread, you are usually watching a context problem, not a model problem.
Ever had an assistant go vague halfway through a long task — did you blame the model or the pile of context? Write to us via our contact page.
Related reading: AI 101 — What is prompt engineering? · AI 101 — What is AI memory? · AI 101 — What is RAG?
