- Nvidia’s closed-loop liquid cooling system virtually eliminates water waste
- Direct-to-chip cooling transfers heat more effectively than air
- It enables higher performance-per-watt and higher rack densities
Data centers aren’t without their fair share of criticism – energy-intensive compute raises temperatures and giant campuses consume considerable amounts of air and/or water to keep them running optimally.
Land scarcity and financial incentives have also been pushing new developments closer to high-risk areas, including drought-prone regions, ultimately leading to even higher cooling requirements.
But Nvidia knows this, and it knows traditional air cooling has pretty much reached its limits as AI hardware becomes denser and denser.
Closed-loop cooling virtually eliminates water waste
With cooling now a core part of AI infrastructure design, Nvidia’s latest liquid-cooled AI systems promise higher thresholds to reduce the burden, driving down water and energy consumption as a result.
By running coolant at higher temperatures – 45°C or 113°F, to be specific – it enabled simpler cooling systems and lowers electricity consumption. Nvidia’s concept uses 75% water, 25% glycol as a coolant, noting that it can run about 5-7°C higher than hot tubs.
Compared with traditional evaporative cooling towers, Nvidia’s latest proposition involves a closed-loop system where coolant continuously circulates through servers to remove excess heat from chips. The warm coolant then cycles through external dry coolers leaving virtually no water evaporation.
The company boasted that cooling-related water consumption can be reduced by as much as 100% in suitable climates subject to the occasional extreme day, with cooling towers totally eliminated.
“The Nvidia DSX reference design for AI factories has zero water consumption – we have eliminated massive amounts of power usage and pretty much all water usage,” Data Center Cooling and Infrastructure Director Ali Heydari said.
The system’s efficiency primarily comes from direct-to-chip liquid cooling, where liquid flows directly through cold plates attached to CPUs and GPUs. This captures and expels the heat from exactly where it’s produced.
Not only is this more effective than cooling entire rooms, but liquid also promises to transfer heat thousands of times more effectively than air.
Major improvements across water consumption, energy efficiency and Power Usage Effectiveness (PUE) all help on the sustainability front, but there’s another layer to the benefits.
Nvidia says it can increase rack density and performance
Nvidia acknowledged that chip power consumption and rack densities continue to rise, so by implementing liquid cooling data center companies can add more GPUs per rack, use higher rack power and ultimately pack in larger AI clusters within the same building footprint.
The company explained that its Rubin systems now fit inside two racks, instead of six, marking a major space saving.
Simultaneously, air cooling has become ineffective. “Once the watts per chip crossed a certain level, liquid cooling became mandatory,” Motivair’s CEO Richard Whitmore said.
Independent testing totally separate from Nvidia’s latest announcement shows that its H100 systems delivered around 17% higher performance when water cooled, compared with air cooling. Under sustained AI workloads, GPU temperatures fluctuated between 41-50°C when water cooled, and 54-72°C when air cooled.
Besides improving immediate and sustained performance, greater thermal efficiency could also boost longevity.
The new, higher-temperature closed-loop water cooling model is set to be used in upcoming Rubin deployments this year.
