AI 101 — What is a transformer?

A transformer is the neural network design almost every modern AI system is built on — an architecture that lets a model look at every word in its input at once and decide which other words matter for understanding it. That mechanism is called attention, and it is the reason AI went from clunky to conversational.

The name comes from a 2017 paper by eight researchers at Google, titled "Attention Is All You Need." Before it, language models read text one word at a time, in order, like someone running a finger along a line. The transformer threw that out. It reads the whole passage in parallel and lets every word consult every other word directly. The paper's authors reported 28.4 BLEU on the English-to-German translation task — better than anything before it, including ensembles — and reached a new single-model record of 41.8 BLEU on English-to-French after just 3.5 days of training on eight GPUs.

Why it matters right now

Every model in the news is a transformer or a close descendant. GPT, Claude, Gemini, Llama, Qwen, DeepSeek, Mistral — different companies, different sizes, same underlying design. When you read about a new model shipping this week, you are reading about a new transformer.

It matters for a second, more practical reason: the transformer's core habit explains nearly everything else about modern AI. Its appetite for data is why labs scraped the internet. Its cost structure is why local models versus cloud APIs is a real decision rather than a formality. And the fact that attention compares every word to every other word is exactly why a context window is a scarce, priced resource instead of unlimited memory.

The one-paragraph mental model

A transformer processes text in layers. First it breaks your input into tokens — rough word-pieces — and turns each into a list of numbers capturing something about its meaning. Then it runs those tokens through a stack of blocks, and each block does the same two things. The attention step lets every token look around at all the others and pull in whatever context it needs — in "the cat sat on the mat, and it was warm," the token "it" gathers meaning from "cat" and "mat." The feed-forward step then thinks about each token on its own, using what attention just collected. Stack dozens or hundreds of these blocks, and by the top the model has built up a rich, context-sensitive understanding of every position in the text. That final representation is what it uses to predict what comes next.

The crucial property is that attention works on the whole passage at once. Nothing has to wait for the word before it to finish.

A room full of people, not a conveyor belt

Picture an old translation machine as a conveyor belt. Words arrive one at a time; the machine reads each, updates its memory, and moves on. By the time it reaches the end of a long sentence, its memory of the beginning has faded. And it cannot start on word fifty until it has finished forty-nine.

Now picture a round table. Everyone is seated at once. Each person can turn to anyone else in the room and ask, "how do you relate to me?" — and everyone does this simultaneously. Nobody waits. That is attention, and the round table is the transformer.

The round table has one obvious cost, though. Add a person and they must now consider everyone else — so doubling the table more than doubles the work. That is the tradeoff baked into the design: transformers are fast because they work in parallel, but long inputs get expensive. It is the mechanical reason behind the context-window limits you run into with every chatbot.

Common misconceptions

"A transformer is the same thing as ChatGPT." No. ChatGPT is a product; the transformer is the architecture underneath it. The same relationship holds between a car engine and a car. Every major AI assistant today runs on transformers, in the way nearly every car runs on an internal combustion engine or an electric motor.

"Transformers understand language like people do." They do something narrower and stranger. A transformer is very good at learning statistical relationships between tokens across enormous amounts of text. Whether that adds up to understanding is genuinely unsettled — and it is the argument underneath most of the AI regulation debate. What is not in dispute is that the results often look like understanding.

"Attention means the model is paying attention to what matters." It is a mechanism, not a mental state. Attention weights are numbers learned during training; they tell the model where to route information, not what it cares about. A model can attend very strongly to a word and still get the answer wrong.

"The 2017 design is unchanged." It has been rebuilt in place. Modern models use variations on attention, different placement of the feed-forward layers, and add-ons like mixture of experts — a way of making each layer hold many sub-networks and only activate a few per token. The core idea — parallel attention over the whole sequence — is still the one from 2017.

Where to learn more

The original "Attention Is All You Need" paper is short, famous, and surprisingly readable in its early sections. For the visual intuition, Jay Alammar's "The Illustrated Transformer" walks through the architecture a diagram at a time and is the single best starting point for non-programmers.

Related reading: What is a token in AI? · What is a context window? · What is a mixture of experts?

Does knowing that attention is just weighted averaging make modern AI seem more or less mysterious to you? Tell us in the comments.