A server cabinet that no longer fits in the air
A classic data centre rack draws between 5 and 15 kilowatts (kW). The tightest ones, at 20 or 30 kW, already need sealed aisles and fans running flat out. Now look at the cabinet NVIDIA designed to train artificial intelligence models: the GB200 NVL72 fits 72 GPUs and 36 Grace processors into a single rack and draws around 120 kW, a figure its newer version, the GB300, pushes up to 142 kW. That is 42 rack units (roughly 1.8 metres tall) swallowing, at full load, the annual electricity of some 300 Spanish households. All in one cabinet.
The problem is not getting that power in: it is getting the heat out. Every watt that goes in ends up as heat inside the chip, and it has to leave or the GPUs drop their clock speed and the training run is over. That is where air, the universal coolant of data centres since the 1960s, runs out of road. The answer the industry has settled on over the past two years is simple to state and hard to execute: put water in the rack.
Why air has a ceiling (it is physics, not laziness)
Air has a heat capacity of about 1,005 joules per kilogram per degree (the joule is the International System unit of energy) and weighs 1.2 kilograms per cubic metre. Water absorbs 4.18 kilojoules per kilogram per degree and weighs 1,000 kilograms per cubic metre. In other words, one litre of water carries away the heat of more than 3,000 litres of air for the same temperature rise.
The concrete numbers are brutal: to remove 120 kW while warming the air by 15 degrees you would have to move some 26,000 cubic metres per hour, eight kilograms of air every second, split across several high-flow fans and large-section ducting. With water and a 10-degree rise, 3.1 litres per second is enough, about 187 litres per minute: 2,300 times less flow, in pipes you can hold in your hand. And one detail settles the argument: a fan’s power grows with the cube of the flow rate, so doubling the air to cool the same load costs eight times the energy. That is why air copes fine up to 20 or 30 kW per rack, with aisle containment and good thermal design. At 100 kW there is simply no fan that will do it sensibly.
Classic cooling and its breaking point
Traditional room cooling is organised around alternating cold and hot aisles: servers pull cold air in at the front and push hot air out at the back, and doors or panels (the so-called containment) keep the two from mixing. A raised floor distributes the cold air, and air handling units (CRAH) chill it with water from the facility loop. When the outside temperature is low, the mechanical chiller is switched off and outside air is used instead: that is free cooling, standard practice in northern Europe and in the cold months.
All of this is measured with PUE (power usage effectiveness, total energy divided by IT equipment energy). A PUE of 1.25 means that for every useful watt you spend 0.25 on cooling, pumping and distribution. An air-cooled data centre running 100 kW racks does not just make that number worse: it enters a spiral where the fans themselves dump heat into the room.
The winning approach: cold plates on the chip
Direct liquid cooling (DLC) places a copper or aluminium plate with internal microchannels, the cold plate, directly on top of the GPU and the CPU. Between chip and plate sits a thermal interface material (TIM), the same idea as the thermal paste in a home PC, but formulated to survive years of thermal cycling without drying out. Water mixed with glycol flows through those microchannels at 2-3 litres per minute per module.
The scale of the challenge is in the density. A Blackwell B200 GPU dissipates around 1,000 watts over an area of roughly 1,600 square millimetres: about 60 watts per square centimetre on average, with far worse local hotspots in the HBM memory stacks, where heat is concentrated in a very small space. Air at that scale is impossible; a cold plate with the right flow rate is not.
The loop: primary, secondary and the CDU in between
A rack like this does not plug into a tap. It carries a manifold that distributes coolant to every tray and blind-mate quick disconnects that let you pull a server drawer while it is running without draining the loop or spilling a drop. That, in fact, is the hard part: the thermal hardware is solved, but a drip on 120 kW of electronics is expensive. Hence leak sensors, drainage trays and redundant seals.
The loop is split into two circuits separated by the CDU (coolant distribution unit): the primary brings water from the facility system, and the secondary carries the glycol mixture that enters the servers. Keeping them apart lets the electronics run on a controlled fluid while the primary side can use dirtier water or a different temperature. The numbers in NVIDIA’s GB200 NVL72 reference architecture are specific: with facility water at 45 degrees, the rack needs 177 litres per minute and shows a pressure drop of 18.4 pounds per square inch. CDUs on the market range from 250 kW units inside the rack itself, with twin hot-swap pumps, to in-row units of 1.3 megawatts, plus liquid-to-air variants of 180 and 240 kW for sites with no cooling tower or water supply.
That 45-degree detail is the heart of the energy saving. The warmer the supply water, the easier it is to reject the heat to the atmosphere with a dry cooler (a radiator with fans, no compressor) or an adiabatic tower. An air-cooled data centre needs cold water, 10 to 15 degrees, and that means running compressor chillers, the biggest consumers in the room. With cold plates you can cool with warm water and skip much of that cost: it is the difference between paying to move heat and paying to create it.
Immersion: the server inside the liquid
The other family is immersion cooling, with the server submerged in a tank. In the single-phase version, a synthetic hydrocarbon heats up without changing state and is pumped to a heat exchanger; in the two-phase version, the liquid boils on contact with the chip, the vapour rises and a condenser at the top of the tank turns it back into liquid. Two-phase moves the most heat per volume and allows over 100 kW per tank with no fans, at the cost of expensive fluids, more maintenance and the discomfort of pulling out a dripping server. There is also a nuance that is often overlooked: cold plates cool the GPU and CPU, but main memory, SSDs, network cards and optical modules still need air, so most trays are hybrids with small fans for the rest of the components.
What changes in the room: weight, cables and electricity
A 120 kW rack with manifolds, CDU and hoses weighs more than 1.5 tonnes, and the floor has to carry it. The electrical side is stretched too: the GB200 carries eight 33 kW power shelves. And there is a copper limit that explains the next generation: with 54-volt DC distribution, a 1-megawatt rack would need up to 200 kilograms of copper busbar per cabinet, and a 1-gigawatt data centre some 200 tonnes of busbar alone. That is why NVIDIA and the rest of the industry are pushing the 800-volt DC architecture, which reaches production with the Kyber racks and Rubin Ultra GPUs in 2027 to support 1 MW racks on the same infrastructure.
Measuring properly: PUE is not enough
Alongside PUE come two more metrics: WUE (water usage effectiveness, litres of water per kilowatt-hour of computing) and the energy reuse factor, which measures how much of the heat ends up warming something useful. Reusing heat is the outstanding task: return water at 45 or 60 degrees can feed a district heating network, and cities such as Stockholm or Zurich have been doing it for years. It does require the municipal network to be nearby and willing to accept modest temperatures. In the European Union there is also a transparency obligation: under the energy efficiency directive, data centres above 500 kW must report their energy and water consumption to the European database, which publishes the figures.
Air is not retiring: it is specialising
It is worth not overstating this: below 20 kW per rack, air remains the cheapest and simplest option, and most of the world’s installed base will keep blowing air for years. What has changed is that at the high end, liquid cooling has stopped being an exotic exception and become the default design, with hybrid racks, warm water and cold plates from day one. In a modern data centre, the question is no longer whether water will run through the cabinets, but when and at what temperature.
