AI 101 — What is liquid cooling for AI data centers?

Liquid cooling means the heat from AI chips is carried away by fluid running through metal plates clamped to the processors, instead of by air blown past them. It is the plumbing change behind almost every new AI data center, and the reason a modern AI hall looks less like a server room and more like a boiler room.
For about thirty years a data center was a windy place: rows of servers, fans at the front, cold air pushed up through a raised floor, hot air pulled out the back. That design works up to a point. One Nvidia GB200 NVL72 rack, which holds 72 Blackwell GPUs and 36 Grace CPUs, draws 120 kilowatts — the heat of roughly a hundred electric space heaters packed into a cabinet the size of a refrigerator, according to Nvidia's own description of the rack. No fan array moves that much heat out of that little volume. So the heat leaves in water.
Why it matters right now
Two things changed at once. Chips got hotter — a top-end AI accelerator burns 700 watts and up — and racks got denser, because the fastest way to train a model is to wire hundreds of accelerators into one machine that fits in one cabinet. Air cooling ran out of headroom somewhere in the 20 to 35 kilowatt range per rack, per engineering work published by Schneider Electric. Everything above that needs a coolant loop.
That is why cooling became a supply-chain story with its own headlines: LG signing on to supply cooling and power for Microsoft's next 26 GW of capacity, and LG's 2.6-megawatt coolant distribution unit clearing Nvidia's reference-design qualification. It is also why the buildout, not the model release, is the slow part — we walked through the electricity side in this AI 101 on how much power AI actually uses.
The mental model: three loops
Think of the heat as water flooding a basement, moved out in three stages.
The first loop is inside the rack. A cold plate — a flat block of copper or aluminium with narrow channels machined through it — is bolted onto each GPU and CPU, with thermal paste or a pad in between so heat crosses the metal. Coolant runs through those channels and picks the heat up by conduction. Nothing is sprayed. The fluid never touches the silicon; it touches the lid of the chip package.
The second loop is the coolant distribution unit, or CDU: a heat exchanger, usually a box at the end of the row, that hands the heat from the rack's fluid to the building's water. The two fluids stay separate, so the rack loop can be filtered and treated like a sealed system while the building loop stays the facility team's problem. The third loop rejects the heat outdoors, through dry coolers, cooling towers, or chillers when the weather will not cooperate.
The detail that surprises people is the temperature. Nvidia's GB200 coolant enters the rack at about 25°C and leaves around 45°C, moving two litres a second, Jensen Huang said at the launch, as The Register reported. Warm water is the point, not a compromise: if the fluid leaving the building is already hot, a mechanical chiller can often stay switched off, and that chiller is one of the largest single loads in the building. Nvidia claims the GB200 is 300 times more water-efficient than a traditional air-cooled hall — a company number, not an independent measurement. Its roadmap pushes further: the Rubin-generation rack is specified for 45°C liquid, and the Kyber rack planned for Rubin Ultra is a 600-kilowatt cabinet, per Data Center Dynamics.
The analogy: the pot on the stove
You are boiling a big pot of soup and the kitchen is unbearable. Option one is the extractor fan: cheap to install, and if the pot gets twice as big the fan has to move twice as much air, which eventually means a wind tunnel and a lot of noise. Option two is pressing a coil of cold water pipe against the side of the pot and running the other end to the drain. Almost none of the heat reaches the room, because it is captured at the source and carried out through a pipe a few centimetres wide.
A cold plate is that coil; the CDU is where your coil meets the building's plumbing. And as with the pot, the reason to bother is density: you can put a much bigger burner under a much smaller pot if the heat has somewhere to go.
Common misconceptions
"They run water through the servers." No. Fluid runs through sealed plates and hoses; the electronics stay dry. Leak detection, drip trays and quick-disconnect fittings are standard, because a leak in a 120-kilowatt rack is an expensive service call.
"Liquid cooling saves energy." It saves the wrong energy. The chips draw the same watts either way. What goes away is the fan power inside the rack and the chiller load outside it, plus floor space — you fit far more compute in the same hall. Efficiency improves; consumption does not fall.
"It is new." IBM water-cooled mainframes in the 1960s and Cray supercomputers in the 1980s. What is new is doing it at commodity scale, in racks a cloud provider buys by the thousand.
"It is one thing." Direct-to-chip, where plates sit on the hottest components, is what nearly all AI racks use. Immersion cooling — lowering whole servers into a tank of non-conducting fluid — exists, works, and is rare outside specialised high-performance computing sites.
The honest trade-off is the retrofit. An air-cooled hall has no manifolds, no CDUs, and no staff trained to service a wet rack, which is why new AI capacity is designed liquid-first and old capacity mostly is not. The other trade-off is lock-in: Nvidia now publishes a reference design for the whole thermal chain, and suppliers that qualify against it land on a very short list.
Do you think cooling, rather than chips, is the real limit on how fast AI can grow? Write to us via our contact page.
Related reading: AI 101 — What is an AI data center? · AI 101 — How Much Power Does AI Actually Use? · AI 101 — What is a GPU, and why does AI need one?







