AI 101 — What is AI memory?

AI memory is a feature of the app, not of the model: it stores notes about you somewhere outside the model and feeds the relevant ones back into the prompt on later turns, so a new conversation can start with some of the old one still attached. The model underneath remembers nothing. Its weights are frozen after training, and between requests it holds no state at all.

That split is the whole concept, and almost every confusion about AI memory comes from missing it. When ChatGPT says it remembers that you are vegetarian, nothing inside the network changed. A sentence to that effect was written into a store, and on your next question the app pasted it back in.

Why this is a live question right now

For the first two years of chatbots, forgetting was the defining experience. You explained your project on Monday and started from zero on Tuesday, because the only thing the model could see was the text in the current window.

OpenAI's first fix, in April 2024, was the obvious one: save notes. You said "remember I'm travelling to Singapore in July" and it kept a list. The company's own retrospective is blunt about how that felt — like talking to someone who took a few notes but still forgot everything that wasn't written down, and whose notes went stale, OpenAI wrote when it shipped the current system. A year later it added a background process that reads chat history and writes the memory itself, which OpenAI calls dreaming.

The version running now, rolled out from June 4, 2026, is a larger rebuild of that background process. OpenAI reports factual recall on its memory tasks at 82.8%, up from 67.9% the year before and 41.5% in 2024, and says it costs about five times less compute to serve, which is what let it reach free users. It also doubled saved-memory capacity for paid plans. Treat those as vendor numbers on vendor benchmarks; the direction is what matters. Memory went from a notes list you maintained to a summary the system maintains about you.

The other reason this matters: agents. A chat that lasts ten minutes can survive on a big context window. An agent that works a job for six hours cannot — it has to write down what it found, drop what stopped mattering, and come back for it later. Long-running agents are memory systems with a language model attached, which is why what an AI agent actually is gets easier to reason about once you see the memory layer underneath.

A mental model: the desk and the cabinet

Three things are involved, and they are constantly confused with each other.

The weights are the model's skill — everything it learned in training. They do not change when you talk to it. Then there is the context window: everything the model can see on this one request, which is your message, the system prompt, and anything the app chose to add. That is the only thing it is actually reading, and what is a context window covers how that limit works. Finally there is memory: text kept outside the model, in a database, a file, or a notes page.

Memory only exists for the model when something carries it back across the gap. Three mechanisms do that. The app retrieves stored notes by keyword or similarity and prepends them. The app writes a summary and injects it every time. Or the model is given tools to read and write its own notes mid-task — the design in the 2023 MemGPT paper, which treated the context window like RAM and external storage like a disk and let the model page its own memory in and out, the paper's authors described.

The kitchen

Picture a short-order cook working a counter. The counter is the context window: there is only so much room, and whatever is not on it is not being cooked. The pantry behind is memory — effectively unlimited, and useless unless someone walks over and brings something back.

The interesting character is the prep sheet. Somebody decides each morning what gets written down: "table four is allergic to nuts", "the Tuesday regular wants it mild". Get that right and the kitchen runs. Get it wrong and you get a cook confidently plating almonds for a man who mentioned a holiday in Singapore, because a note that said "going to Singapore" became a note that said "lives in Singapore".

Notice what a forty-foot counter would buy you: more room for ingredients, and a cook who still has to decide what to fetch. That is the honest state of the art. Capacity is not the same as attention.

What people get wrong

"It learns from me." It does not. No commercial assistant retrains its weights on your chats mid-conversation. What you call learning is text being stored and re-inserted — which is also why a memory can be deleted, audited, and wrong.

"A bigger context window is memory." They solve different problems. Chroma's July 2025 study of long inputs found model performance degrading as input length grows, unevenly and unpredictably, even on simple tasks — the effect the researchers named context rot, per Chroma's report. Stuffing more history into the window is not the same as remembering well.

"More memory is always better." Memory goes stale, and stale memory is worse than none: it arrives with the model's full confidence behind it. OpenAI's own framing of the problem is freshness and contradiction, which is why the newer system tries to distinguish a passing state ("I'm in Singapore this week") from a durable fact.

"Memory is a transcript." It is a summary of you, written by a model, and summaries drift — which is also why deleting a chat does not reliably delete what it learned. The useful habit is the boring one: occasionally ask what it remembers, and correct it.

Where to learn more

Read OpenAI's write-up of the memory system for how the background synthesis works and what it claims. Read the MemGPT paper for the architectural idea — memory as a file system the model manages. Read the Chroma report before you assume a million-token window means nothing gets lost.

Related reading: What is a context window? on the only thing a model can actually see · What is RAG? on retrieving documents instead of personal notes · What is an AI agent? on why long tasks need a memory layer.

Would you rather your assistant forget you every morning, or remember you slightly wrong? Write to us via our contact page.