Google's Liquid Cooling

https://chipsandcheese.com/p/googles-liquid-cooling-at-hot-chips

by giuliomagnifico • 11 months ago

400 167 11 months ago

chipsandcheese.com

betaby• 11 months ago

Relevant.

OVH has been using liquid cooling for many years:

https://blog.ovhcloud.com/new-hybrid-immersion-liquid-coolin...

https://www.youtube.com/watch?v=S_DHJcQpumI

jeffbee5• 11 months ago

And yet their claimed PUE is 1.26 which is pretty bad. One way to characterize that overall PUE figure is they waste 3x as much on overhead as Google (claimed 1.09 global PUE) or Meta (1.08).

Loic• 11 months ago

10 years ago, OVH datacenter SBG1 was below 1.1 with a mixture of natural convection + liquid cooling. But I have not been following in the latest years.

bri3d6• 11 months ago

Your linked articles are about immersion cooling, which is "liquid cooling," I suppose, but a very different thing. Do OVH actually use immersion cooling in production? This seems like a "labs" product that hasn't been fully baked yet.

OVH _do_ definitely use traditional water-loop/water-block liquid cooling like Google are presenting, described here: https://blog.ovhcloud.com/understanding-ovhclouds-data-centr... and visually in https://www.youtube.com/watch?v=RFzirpvTiOo , but while it looks very similar architecturally their setup seems almost comically inefficient compared to Google's according to their PUE disclosures.

TiredOfLife• 11 months ago

Based on how their data centers burn down, they don't even use liquid in their fire suppression systems.

michaelt4• 11 months ago

TPU chips are hooked up in series in the loop, which naturally means some chips will get hotter liquid that has already passed other chips in the loop. Cooling capacity is budgeted based on the requirements of the last chip in each loop.

Of course, it's worth noting that if you've got four chips, each putting out 250W of power, and a pump pushing 1 litres of water per minute through them, water at the outlet must be 14°C hotter than water at the inlet, because of the specific heat capacity of water. That's true whether the water flows through the chips in series, or in parallel.

foota2• 11 months ago

Hm... but in the case when the chips are in serial, the heat transfer from the last chip will be less than when the chips are in parallel, because the rate of heating is proportional to the difference in temperature, and the water starts at a lower temperature for the parallel case for this last theoretical chip.

fraserphysics• 11 months ago

One way to characterize the cost of cooling is entropy production. As you say, cooling is proportional to difference in temperature. However, entropy production is also proportional to temperature difference. It's not my field, but it looks like an interesting challenge to optimize competing objectives.

0x4572• 11 months ago

Yes, but water is constantly moving in a loop. It's not like you use water to cool chip #1, and then it moves to chip #2, it's constantly moving, so temperature delta isn't that much.

chickenbig• 11 months ago

In steady state the power put into the chip is removed by the water (neglecting heat transfer away from the cooling system). The increased water temperature on entering into the cooling block is offset by a correspondingly higher chip temperature.

idiotsecant• 11 months ago

It just means in series that some of your chips get overcooled in order to achieve the required cooling on the hottest chip. You need to run more water for the same effect.

k7sune• 11 months ago

I can imagine a setup where multiple series of slower cooler water converging into a faster warmer stream, and the water will extract an equal amount of heat away from all the chips whether upstream or downstream.

friendzis• 11 months ago

While there is some truth to your comment, it has no practical engineering relevance. Since energy transfer rate is proportional to temp difference, therefore you compute the flow rate required, which is going to be different if the chips are in series or in parallel.

Dylan168073• 11 months ago

If you're heating up the water by 10 or so degrees on typical computer hardware, I bet you're not bottlenecked by energy transfer. Your flow rate is based on how hot you want the water to get, so series or parallel goes back to not mattering.

JonChesterfield• 11 months ago

The equations all work out OK though since one can go substantially above a litre a minute at pressures which are still viable. The standard 18W thing for desktops is about ten times that.

jonathaneunice4• 11 months ago

It’s very odd when mainframes (S/3x0, Cray, yadda yadda) have been extensively water-cooled for over 50 years, and super-dense HPC data centers have used liquid cooling for at least 20, to hear Google-scale data center design compared to PC hobbyist rigs. Selective amnesia + laughably off-target point of comparison.

jeffbee5• 11 months ago

Hyper-scale data centers normally need not be concerned with power density, and their designers might avoid density because of the problems it causes. Arguably modern HPC clusters that are still concerned about density are probably misguided. But when it comes to ML workloads, putting everything physically close together starts to bring benefits in terms of interconnectivity.

jonathaneunice4• 11 months ago

LOLWUT? Hyperscalers and HPC data centers have been very concerned about power and thermal density for decades IME. If you're just running web sites or VM farms, sure keep racks cooler and more power efficient. But for those that deeply care about performance, distance drives latency. That drives a huge demand to "pack things closely" and that drives thermal density up, up, up. "Hot enough to roast a pig" was a fintech data center meme of 20 years go, back at 20kW+ racks. Today you're not really cooking until you get north of 30kW.

spankalee• 11 months ago

From the article:

Liquid cooling is a familiar concept to PC enthusiasts, and has a long history in enterprise compute as well.

And the trend in data centers was to move towards more passive cooling at the individual servers and hotter operating temperatures for a while. This is interesting because it reverses that trend a lot, and possibly because of the per-row cooling.

echelon• 11 months ago

This abundance of marketing (not necessarily this blog post) is happening because of all the environmental chatter about AI and data centers recently.

Google wants you to know it recycles its water. It's free points.

Edit: to clarify, normal social media is being flooded with stories about AI energy and water usage. Google isn't greenwashing, they're simply showing how things work and getting good press for something they already do.

dekhn4• 11 months ago

We've basically been watching Google gradually re-discover all the tricks of supercomputing (and other high performance areas) over the past 10+ years. For a long time, websearch and ads were the two main drivers of Google's datacenter architecture, along with services like storage and jobs like mapreduce. I would describe the approach as "horizontal scaling with statistical multiplexing for load balancing".

Those style of jobs worked well but as Google has realized it has more high performance computing with unique workload characteristics that are mission-critical (https://cloud.google.com/blog/topics/systems/the-fifth-epoch...) their infrastructure has had to undergo a lot of evolution to adapt to that.

Google PR has always been full of "look we discovered something important and new and everybody should do it", often for things that were effectively solved using that approach a long time ago. MapReduce is a great example of that- Google certainly didn't invent the concepts of Map or Reduce, or even the idea of using those for doing high throughput computing (and the shuffle phase of MapReduce is more "interesting" from a high performance computing perspective than mapping or reducing anyway).

legulere• 11 months ago

It's not so surprising when considering googles history coming from inexpensive commodity hardware. It's pretty similar to how it took decades for x86 servers and operating systems to gain mainframe functionality like virtualisation.

https://blog.codinghorror.com/building-a-computer-the-google...

mr_toad4• 11 months ago

Once again IT comes full circle.

liquidgecka7• 11 months ago

[bri3d pointed out that I missed an element of this. There is a transfer between rack level and machine level coolant which makes this far less novel than I had initially understood. See their direct comment to this]

I posed this further down in a reply-to-a-reply but I should call it out a little closer to the top: The innovation here is not “we are using water for cooling”. The innovation here is that they are direct cooling the servers with chillers that are outside of the facility. Most mainframes will use water cooling to get the heat from the core out to the edges where traditional where it can be picked up by the typical heatsink/cooling fans. Even home PCs do this by moving the heat to a reservoir that can be more effectively cooled.

What Google is doing is using the huge chillers that would normally be cooling the air in the facility to cool water which is directly pumped into every server. The return water is then cooled in the chiller tower. This eliminates ANY air based transfer besides the chiller tower. This is one being done a server or a rack.. its being done on the whole data center all at once.

I am super curious how they handle things like chiller maintenance or pump failures. I am sure they have redundancy but the system for that has to be super impressive because it can’t be offline long before you experience hardware failure!

[Edit: It was pointed out in another comment that AWS is doing this as well and honestly their pictures make it way clearer what is happening: https://www.aboutamazon.com/news/aws/aws-liquid-cooling-data...]

ambicapter• 11 months ago

So every time they plug in a server they also plug in water lines?

nitwit005• 11 months ago

This was before I was born, so I'm hardly an expert, but I've heard of feeding IBM mainframes chilled water. A quick check of wikipedia found some mention of the idea: https://en.wikipedia.org/wiki/IBM_3090

ChuckMcM3• 11 months ago

Much of the Google use of liquid chillers was protected behind NDAs as part of its "hidden advantage" with respect to the rest of the world. It was the secret behind really low PUE numbers.

bri3d6• 11 months ago

I don't think this comment is accurate based on the article, although you cite personal experience elsewhere so maybe your project wasn't the one that's documented here?

What Google is doing is using the huge chillers that would normally be cooling the air in the facility to cool water which is directly pumped into every server.

From the article:

CDUs exchange heat between coolant liquid and the facility-level water supply.

Also, I know from attaching them at some point that plenty of mainframes used this exact same approach (water to water exchange with facility water), not water to air to water like you describe in this comment and others, so I think you may have just not had experience there? https://www.electronics-cooling.com/2005/08/liquid-cooling-i... contains a diagram in Figure 1 of this exact CDU architecture, which it claims was in use in mainframes dating back to 1965 (!).

I also don't think "This eliminates ANY air based transfer besides the chiller tower." is strictly true; looking at the photo of the sled in the article, there are fans. The TPUs are cooled by the liquid loop but the ancillaries are still air cooled. This is typical for water cooling systems in my experience; while I wouldn't be surprised to be wrong (it sure would be more efficient, I'd think!), I've never seen a water cooling system which successfully works without forced air, because there are just too many ancillary components of varying shapes to successfully design a PCB-waterblock combination which does not also demand forced air cooling.

jwr• 11 months ago

they are direct cooling the servers with chillers that are outside of the facility

That is exactly what the Cray Y-MP EL that I worked with in the 90s/2000s did.

bluedino• 11 months ago

super-dense HPC data centers have used liquid cooling for at least 20

Hasn't this just been for things like rack doors and such?

In the last ~two generations of servers it seems like now there's finally DLC (direct liquid cooling) into the actual servers themselves (similar to the article). Intel kind of forced that one on us, with their high-end SKUs. This has been a pain becuase it doesn't fit into legacy datacenters as easily as the door/rack-based systems.

I won't say which server vendor it is, but I've put in more than one service ticket for leaking coolant bags (they hang them on the racks).

BoppreH• 11 months ago

I see frequent mentions of AI wasting water. Is this one such setup, perhaps with the CDU using the facility's water supply for evaporative cooling?

bri3d6• 11 months ago

The CDU is inside the datacenter and strictly liquid to liquid exchange. It transfers heat from the rack block's coolant to the facility coolant. The facility then provides outdoor heat exchange for the facility coolant, which is sometimes accomplished using open-loop evaporative cooling (spraying down the cooling towers). All datacenters have some form of facility cooling, whether there's a CDU and local water cooling or not, so it's not particularly relevant.

The whole AI-water conversation is sort of tiring, since water just moves to more or less efficient parts or locations in the water cycle - I think a "total runtime energy consumption" metric would be much more useful if it were possible to accurately price in water-related externalities (ie - is a massive amount of energy spent moving water because a datacenter evaporates it? or is it no big deal?). And the whole thing really just shows how inefficient and inaccurately priced the market for water is, especially in the US where water rights, price, and the actual utility of water in a given location are often shockingly uncorrelated.

maartin0• 11 months ago

Slightly old but interesting https://blog.equinix.com/blog/2024/09/19/how-data-centers-us...

bjackman3• 11 months ago

Even without AI the whole water conservation topic is kinda silly.

The margin at which it makes sense to save water varies wildly by location, but the cultural dominance of the western USA infiltrates everything.

Here in Zürich, my office briefly installed water saving taps. This is a city of less than half a million where the government maintains 1,200 fountains that spew drinkable water 24/7. But someone at the California HQ said saving water is important and someone at the Swiss subsidiary said yes sir we'll look into it right away sir.

Lerc2• 11 months ago

I have encountered a lot of references to AI using water, but with scant details. Is it using water in the same way a car uses a road? The road remains largely unchanged?

The implication is clear that it is a waste, but I feel like if they had the data so support that, it wouldn't be left for the reader to infer.

I can see two models where you could say water is consumed. Either talking about drinkable water rendered undrinkable, or turning water into something else where it is not practically recaptured. Tuning it into steam, sequestering it in some sludge etc.

Are these things happening? If it is happening, is it bad? Why?

I'd love to see answers on this, because I have seen the figures used like a kudgel without specifying what the numbers actually refer to. It's frustrating as hell.

jeffbee5• 11 months ago

The water is "used" in the sense that it evaporates. At a global average rate of 1 liter per kilowatt-hour of energy, Google claims.

tony_cannistra• 11 months ago

This article will help you. https://www.construction-physics.com/p/how-does-the-us-use-w...

...actual water consumed by data centers is around 66 million gallons per day. By 2028, that’s estimated to rise by two to four times. This is a large amount of water when compared to the amount of water homes use, but it's not particularly large when compared to other large-scale industrial uses. 66 million gallons per day is about 6% of the water used by US golf courses, and it's about 3% of the water used to grow cotton in 2023.

quickthrowman2• 11 months ago

I have encountered a lot of references to AI using water, but with scant details.

Here you go: https://en.m.wikipedia.org/wiki/Cooling_tower

Data centers use evaporative cooling towers which evaporate water to reject heat to the atmosphere.

sleepydog• 11 months ago

AWS had a similar article a couple months ago:

https://www.aboutamazon.com/news/aws/aws-liquid-cooling-data...

In either case I cannot find out how they dump the heat from the output water before recycling it. That's a problem I find far more interesting.

esaym• 11 months ago

Yes see: https://www.austinchronicle.com/news/2025-07-25/texas-is-sti...

m463• 11 months ago

I wonder what the economics of water cooling really is.

Is it because chips are getting more expensive, so it is more economical to run them faster by liquid cooling them?

Or is it data center footprint is more expensive, so denser liquid cooling makes more sense?

Or is it that wiring distances (1ft = 1nanosecond) make dense computing faster and more efficient?

summerlight• 11 months ago

Not sure about classical computing demands, but I think wiring distances definitely matter for TPU-like memory heavy computation.

moffkalast• 11 months ago

It's more of a testament to inefficiency, with rising TDP year after year as losses get larger with smaller nm processes. It's so atrocious, even in the consumer sector Nvidia can't even design a connector that doesn't melt during normal usage because their power draw has become beyond absurd.

People don't really complain about crappy shovels during a gold rush though unfortunately, they're just happy they got one before they ran out. They have no incentive to innovate in efficiency while the performance line keeps going up.

MurkyLabs• 11 months ago

It's a mixture of both 2 and 3. The chips are getting hotter because they're compacting more stuff in a small space and throwing more power into them. At the same time, powering all those fans that cool the computers takes a lot of power (when you have racks and racks those small fans add up quickly) and that heat is then blown into hot isles that need to then circulate the heat to A/C units. With liquid cooling they're able to save costs due to lower electricity usage and having direct liquid to liquid cool as apposed to chip->air->AC->liquid. ServeTheHome did a write up on it last year, https://www.servethehome.com/estimating-the-power-consumptio...

mikepurvis2• 11 months ago

I've never done DC ops, but I bet fan failure is a factor too— basically there'd be a benefit to centralizing all the cooling for N racks in 2-3 large redundant pumps rather than having each node bringing its own battalion of fans that are all going to individually fail in a bell curve centered on 30k hours of operation, with each failure knocking out the system and requiring hands-on maintenance.

xadhominemx• 11 months ago

It’s because the chips are networked with very high bitrates and need to be physically densely packed together.

jabl4• 11 months ago

Or is it that wiring distances (1ft = 1nanosecond) make dense computing faster and more efficient?

Contrary to other posters, I'd argue this effect is relatively small. A really good interconnect fabric might give you ping-pong times on the order of 1 microsecond, which is still 1000 times larger than a nanosecond. Most of the delay will be in the switches and the end nodes, not in the signal traveling over the wire or fiber. Say for a large-ish cluster with a diameter of, say, 100 feet (something like 7 rows of racks, each row 100 feet long, give or take), if liquid cooling allows you to double the density, you could condense it to a diameter of 100/sqrt(2) = 70 ft (about 5 rows of 70 ft each). As a ping-pong involves a signal going both ways, the worst-case increase in signal delay would be (100-70)*2 = 60 ft or 60 nanoseconds (in reality somewhat more since cables have to be routed). So about a 6% increase if we assume the baseline is 1 microsecond. Measurable, yes, but likely very small effect on application performance vs. a ping-pong microbenchmark.

Now where it can matter is that by packing the components more closely together, you can connect more chips via backplane and/or copper connectors vs. having to use optics.

mnw21cam3• 11 months ago

It's also the fact that a large portion of the power used by a data centre is actually spent on cooling, so anything that will make that more efficient is a cost saving.

hnburnsy2• 11 months ago

"Google extensively validates components with leak testing, uses alerting systems to discover problems like leaks, and takes preventative measures like scheduled maintenance and filtration. They also have a clear set of protocols to respond to alerts and issues, allowing a large workforce to respond to problems in a consistent fashion. It’s a far cry from the ad-hoc measures enthusiasts take to maintain their water cooling setups."

How about the author compare it to MS, Meta, or AWS instead of Joe Blow buying parts online? I would hope that Google had extensive validation and clear protocols. [Roll-Eyes]

MBCook• 11 months ago

My guess would be the average reader is probably far more familiar with people water cooling their own machines than the intricacies of how various large companies are doing water cooling (or not) in their data centers.

hnburnsy2• 11 months ago

Kind of related from B1M...

The Paris Olympic Pool is Heated by the Internet
https://www.youtube.com/watch?v=2gWudPtN6z4&t=4s

miohtama• 11 months ago

How much moat Google has in the AI race due to its in-house expertise in data centers and in-house IPR in TPUs? Can any company on Earth match them?

owebmaster• 11 months ago

Not enough to prevent open source models making it all useless, I think.

ykl• 11 months ago

I once saw an interview with the SVP who oversees Azure datacenter buildout or something like that and a thing that stuck with me was that he said his job got a lot easier when he realized he was no longer in the computer business, he was now in the industrial cooling business.

Reading this article immediately made me think back to that.

kak9• 11 months ago

curious to see that interview, do you remember link?

nwmcsween• 11 months ago

What I don't understand is why not use what I've seen in industrial processes? Why not a low boiling point liquid and a heat exchanger? Or even better power a turbine like biogas plants do

ianpenney• 11 months ago

Between 2006 and 2012 I was in DCs. I didn’t live there but I had to visit at stupid o’clock too often.

It’s not a hospitable place.

I would have loved it if cooling was quieter and less extreme.

The reason interfaces are on the back is that’s the intake side. So, bring a sweater.

Or just go to the other side of the rack to warm your hands back up.

pjdesno• 11 months ago

In theory data center cooling is simple - CPUs run at 60-70C, while the outside ambient is usually 30C or less, so heat should just "flow downhill" with a bit of help from fans, pumps, etc.

The problem with using air cooling to get it there is that the humans who run the data center have to enter and breathe the same air that's used to cool the computers, and if your working fluid gets too hot it's quite unhealthy for them. (we run our hot aisles at 100F, which is a bit toasty, and every third rack is a heat exchanger running off the chilled water lines from the outside evaporative cooler, modulo a heat exchanger to keep the bird shit out)

We're not going to be able to pump much heat into the outside world unless our working fluid is a decent amount hotter than ambient, so when it gets reasonably warm outside we need to put chillers (water-to-water AC units) in the loop, which consume energy to basically concentrate that heat into a higher-temperature exhaust. When it's really hot outside they consume quite a bit of energy.

If the entire data center was liquid cooled we could have coolant coming from the racks at a much higher temperature, and we'd be able to dump the heat outside on the hottest days without needing any chillers in the loop. As it is we have some liquid cooled racks running off the same chilled water lines as the in-aisle heat exchangers, but the coolant temp is limited by the temperature of our hot aisles, which are quite hot enough already, thank you.

fanf2• 11 months ago

15 years ago IBM installed a supercomputer at ETH Zurich that used 60C hot water as its coolant, with a heat exchanger to the building’s hot water system (which is typically somewhat less than 60C) https://en.m.wikipedia.org/wiki/Aquasar

saagarjha• 11 months ago

I wonder if we will eventually have people entering datacenters in heatsuits to eke out some extra efficiency gains from just needing to cool people.

eternauta3k• 11 months ago

CPUs run at 60-70C, while the outside ambient is usually 30C or less, so heat should just "flow downhill" with a bit of help from fans, pumps, etc.

This isn't how you think of heat flow. The CPUs run at a given power. Their temperature will depend on the ambient temperature, and on the thermal impedance between them and the ambient. If the thermal impedance is too high, they'll be too hot and die.

mr_toad4• 11 months ago

CPUs run at 60-70C

When properly cooled. If cooling is insufficient they can easily get much hotter.

adolph• 11 months ago

Seeing in "installation & commissioning" mention of "Leak testing" reminds me of the PowerMac G5 liquid cooling leaks:

https://www.xlr8yourmac.com/systems/G5_CoolantLeak_Repair/G5...

tyingq• 11 months ago

OVH has been doing liquid cooling for quite a while, and is now using immersion also. Somewhat surprising to see at that dirt cheap end of the market.

ehnto• 11 months ago

I wish I could be as excited as I used to be about Google's infrastructure. But with their constant attacks on the freedom of the internet, they have exhausted my good-will.

So I am afraid I do not care about your liquid cooling on this day. I'm also not particularly impressed, I am sure the devil is in the details at this scale but there doesn't seem to be much novel here.

If a Googler happens to read this and it makes them sad, it's Google, not you. You might consider doing cool stuff somewhere else.

anon63622• 11 months ago

FAANGs have gone from wunderkind darlings that would pave the way for progress, like turn of the 20th-century-style, to monopolistic, feudal overlords rapaciously seeking to exploit land and resources with zero concern for the environment or locals' quality of life.

flakes• 11 months ago

In theory, a PC builder could try exchanging heat to a toilet tank, allowing for highly efficient cooling with each flush.

The future is here.

WJW• 11 months ago

In an interesting way, the reverse is also here: some district heating plants use heat pumps to extract waste heat from sewage to provide efficient heat for district heating systems.

See https://www.eneco.nl/en/about-us/what-we-do/sustainable-reso... for a basic overview.

lopis2• 11 months ago

I dream of a future where there's an efficient energy recapture in the house. We waste a lot of heat in the shower, and washing clothes and dishes. We also run the fridge all winter when it's less than 5C outside, and the fridge then produces a lot of heat. And when we're done cooking, the oven is still really hot. Could the home heating system of the future use all this heat (and cold) more efficiently?

bilater• 11 months ago

I’ve always wondered: what if we could actually use this heat? For example, could we take the heated water from cooling a data center and pump it to places that need it (Greenhouses, Industrial processes, Desalination and water treatment)?

xxpor• 11 months ago

Happens for urban data centers/internet exchanges:

www.aboutamazon.com/news/sustainability/the-super-efficient-heat-source-hidden-below-amazons-seattle-headquarters

rockostrich• 11 months ago

It's done sparingly. If you search for "data center waste heat uses" you'll find cases like you mention [1] [2] but the ROI on implementing it seems to be lower than renewable energy infrastructure.

[1] https://www.informationweek.com/sustainability/reusing-waste... [2] https://www.techspot.com/news/97995-data-center-uses-waste-h...

anon63622• 11 months ago

That's pretty much the way it had to go, other than doing full immersion cooling like GRC that never caught on. Meta has various rack-based chill water heat-exchanger cooling solutions but this looks a bit more integrated.

Google's Liquid Cooling

Related Stories