Liquid Cooling Explained: Why the AI Rack Is a Plumbing Problem
AI rack density is pushing data centres past air cooling. Cold plates, immersion, the CDU gate, and why cooling can cap live GPUs.
Every watt of compute becomes heat that has to leave the building. As AI racks densify, cooling stops being a facilities detail and becomes the thing that decides how many GPUs actually run at full power.
In short: cooling is a bottleneck when racks make more heat than the hall can remove. GPUs may sit in boxes yet run throttled or stay offline. Liquid cooling raises the ceiling, but plant, plumbing, and power still set the pace.
What does a cooling bottleneck mean?
A bottleneck is the scarcest step that limits the whole system. When cooling is that step:
- New GPUs cannot turn on at full power
- Racks must thin out or throttle
- Build schedules wait on pipes, pumps, or heat-rejection gear
- Chip supply alone does not raise live capacity
The story is physics first, headlines second.
Why is heat the twin of power?
Raise rack power and you raise heat load by the same amount. The two are coupled, and cooling design has to match both.
A hall may have enough grid feed but not enough heat rejection. Or enough chillers but not enough rack-level liquid distribution. Either mismatch caps useful GPUs while the electricity bill continues.
Illustrative example: if a rack's power doubles versus a prior design in the same footprint, airflow that worked before may fail. The operator then needs cold plates, rear-door exchangers, or a full liquid loop.
Where does air cooling hit a wall?
Air cooling blows cool air across hot sinks and removes warm air. It is familiar, easy to service, and works well at moderate density.
Limits appear when required airflow becomes huge and noisy, hot spots form inside dense GPU servers, fan energy climbs, or inlet temperatures cannot stay low enough.
Past that wall, adding more fans is a weak fix. Air simply cannot carry heat away fast enough.
How does liquid cooling work?
Liquid carries far more heat per unit volume than air. The common patterns:
| Approach | How it works | Trade-off |
|---|---|---|
| Cold plates, direct-to-chip | Liquid loops run over the hottest components | Highest density per retrofit effort, needs plumbing to each rack |
| Rear-door heat exchangers | Liquid coil on the rack door cools exhaust air | Easier retrofit, lower ceiling |
| Immersion | Servers sit in dielectric fluid | Very high density, biggest change to operations and servicing |
All three still need somewhere to reject the heat outside the building. The rack is only the visible end of the loop.
Why do CDUs and facility loops gate liquid halls?
This is the part most coverage misses, and it is where schedules actually slip.
A CDU — coolant distribution unit — moves heat between the building's cooling plant and the liquid loops that touch the GPUs. Cold plates on a server do very little if there is no CDU capacity, no pipes, and no heat rejection outside.
Dense AI halls therefore buy plant, not only better chassis. Server makers can ship liquid-ready boxes while CDU slots and install crews lag, and the earn date follows the late part.
Illustrative example: 100 liquid-ready servers arrive. 40 CDU ports are live. Sixty servers wait. The chip story looked fine; the plant story was short.
Four questions for any cooling procurement headline:
- Is the order for rack kits, CDUs, or full facility loops?
- Are lead times about equipment or skilled install crews?
- Does the hall already have power, or is cooling waiting on watts?
- Is this a retrofit of an air hall or a liquid-native build?
Who gets paid when cooling binds?
- Cooling equipment suppliers when CDU orders and lead times stretch
- Landlords and hyperscalers who must fund plant upgrades
- Neoclouds only after cooled megawatts exist to rent
- Power teams, because more cooling electricity rides with more IT load
The sequencing matters: plant spend comes before rent, and the gap between them is carried by whoever owns the hall.
How does this link to power?
Cooling and power are the same constraint viewed from two ends. Energised megawatts still need heat rejection before racks run at full clock — see speed-to-power explained and behind-the-meter power.
A hall is not "done" on power alone.
What would change this view?
The cooling gate loosens if CDU and facility-loop lead times normalise, install crews scale with demand, liquid-native builds outpace retrofits, and accelerator designs improve performance per watt enough to slow density growth.
It stays binding while liquid-ready servers ship faster than the plant that cools them.
Practical takeaways
- Every watt of compute becomes a watt of heat to remove
- Air cooling has a density ceiling that more fans cannot fix
- The CDU and the facility loop, not the cold plate, usually set the schedule
- Cooling capex lands before rent does
FAQ
Why do AI data centres need liquid cooling?
Because rack power density has risen past what airflow can remove. Liquid carries far more heat per unit volume than air.
What is a CDU?
A coolant distribution unit. It moves heat between the building's cooling plant and the liquid loops that touch the servers.
What is the difference between cold plates and immersion?
Cold plates run liquid over the hottest components inside an otherwise normal server. Immersion submerges whole servers in dielectric fluid, allowing higher density but changing how the hall is operated and serviced.
Can you retrofit an air-cooled hall for liquid?
Often partially, using rear-door heat exchangers or direct-to-chip loops, but full conversion needs plant, piping, and heat rejection that many older halls lack.
Does cooling or power bind first?
Either can. They are coupled — a hall with enough grid feed can still lack heat rejection, and a well-cooled hall can still wait on energised megawatts.
