AI 101 — What is an AI data center?
An AI data center is a building full of specialized computers — mostly GPUs — designed from the power feed to the cooling pipes to train and run AI models at massive scale. From the outside it looks like any windowless warehouse. Inside, the difference is density: racks of chips packed so tight, and drawing so much power, that ordinary air conditioning and ordinary grid connections no longer do the job.
Why it matters right now. These buildings have become the physical front line of AI. Between October 8 and 11, drone strikes hit three Yandex data centers in four days; the third site — reported at roughly 50 megawatts and about 2,880 server racks — stopped operating completely, and YandexGPT went offline with it. The second pressure is electrical: the International Energy Agency reckons around 20% of planned data center projects are at risk of delays from grid bottlenecks, and global investment in data centers reached about half a trillion dollars in 2024, nearly double the 2022 figure. Chips grab the headlines, but substations, cooling loops, and siting decide how fast AI actually grows. We've covered the electricity side separately in AI 101 — How Much Power Does AI Actually Use?; this one is about the anatomy of the building itself.
The mental model. A data center is a machine for turning electricity into computation. In a classic one, work is fragmented — one rack handles email, another a database, another a website — so the facility is built for balance: steady loads, air cooling, everything in moderation. An AI data center is built around one repeated operation, matrix multiplication, run around the clock by thousands of identical accelerators. Every subsystem — power delivery, cooling, networking — is therefore sized for a single extreme, uniform workload. And when you ask a chatbot a question, the reply is computed in a hall like this: that step is AI inference, and it is what these buildings do all day.
The analogy: a kitchen. Think of a traditional data center as a big hotel kitchen — many stations, each doing something different, sized so no station ever runs hot. An AI data center is a kitchen built around one appliance: a blast furnace that must never stop burning. You don't cope by adding extension cords and opening a window; you build the room around the furnace — its own gas main, a water-cooling loop, a reinforced floor. The GPU racks are the furnace; everything else in the building exists to feed them and carry the heat away.
What's actually inside. The rack is the unit of design. NVIDIA's reference configuration for one GB200 NVL72 rack — 72 Blackwell GPUs and 36 Grace CPUs — is planned at roughly 120 kilowatts, an order of magnitude denser than a conventional server rack. (For contrast: Yandex's struck facility worked out to well under 20 kilowatts per rack across the site, cooling included.) At that density, fans are useless, so coolant runs through cold plates bolted directly onto the chips. One such rack holds about 30 terabytes of memory and connects its GPUs over a 130 terabyte-per-second NVLink fabric so the 72 chips behave as a single giant GPU. Memory is the scarce resource in this design — the chips of this class rely on a special kind of it called high-bandwidth memory. Between racks, the networking matters as much as the chips: thousands of processors must stay in lockstep, or a training run stalls waiting for the slowest one.
Common misconceptions. First: it's just a normal data center with fancier servers. The power and cooling math above says otherwise — this is a different engineering problem, not a bolt-on. Second: data center means the cloud. The cloud is the service sold from buildings like these; the data center is the building itself. Third: it's one giant computer. It is thousands of separate machines that must act as one — which is why the interconnect gets as much engineering attention as the processors. And it isn't exclusively Nvidia hardware: Google's equivalent facilities run on its own TPUs.
Where to learn more.
Is physical security now the most underestimated constraint on AI capacity — after chips and power? Tell us in the comments.
