DRAM thermal issues reach crisis point

https://semiengineering.com/dram-thermal-issues-reach-crisis-point/

by rbanffy • 4 years ago

198 154 4 years ago

Victerius2• 4 years ago

A few overheated transistors may not greatly affect reliability, but the heat generated from a few billion transistors does. This is particularly true for AI/ML/DL designs, where high utilization increases thermal dissipation, but thermal density affects every advanced node chip and package, which are used in smart phones, server chips, AR/VR, and a number of other high-performance devices. For all of them, DRAM placement and performance is now a top design consideration.

I know this may not be a cheap solution, but why not start selling pre-built computers with active cooling systems? Refrigerant liquids like those used in refrigerators or water cooling could be an option. The article addresses this:

Although it sounds like a near-perfect solution in theory, and has been shown to work in labs, John Parry, industry lead, electronics and semiconductor at Siemens Digital Industries Software, noted that it’s unlikely to work in commercial production. “You’ve got everything from erosion by the fluid to issues with, of course, leaks because you’re dealing with extremely small, very fine physical geometry. And they are pumped. One of the features that we typically find has the lowest reliability associated with it are electromechanical devices like fans and pumps, so you end up with complexity in a number of different directions.”

So instead of integrating fluids within the computer, build powerful mini-freezers for computers and store the computer inside. Or split the warm transistors from the rest of the build and store only those inside the mini freezer, with cables to connect to the rest of the computer outside.

jackmott42• 4 years ago

Active cooling tends to have the challenge of controlling condensation, and then of course now you are drawing even MORE power from the wall.

mrtranscendence• 4 years ago

I've seen YouTube videos of overclockers employing refrigeration techniques (or coolants like liquid nitrogen), and it does seem like condensation is a major issue. Maybe that's not as much of a problem at more reasonable temperatures?

But yeah, I'd be just as or more concerned about the amount of power it would take to run a freezer like that ... I'm already drawing as much as 850 watts for my PC, with a max of a couple hundred watts for my OLED TV and speakers, and don't forget the modem and router, and a lamp to top it all off; would a powerful enough mini freezer to cool my PC even fit on the circuit?

Actually, it's even worse because I've got an air purifier running there too ... but I could move that, I suppose.

SketchySeaBeast5• 4 years ago

So you're taking a 300W-1000W space heater and putting it into a freezer that needs to be able to bleed that much heat? Going to need another breaker.

Victerius2• 4 years ago

I'm just brainstorming. I can troubleshoot my computer and write basic code but I'm not a computer engineer.

7speter2• 4 years ago

This has come up often in the comments section of articles I've seen about prospective 600-900w 40 series nvidia cards.

_jal• 4 years ago

For most commercial use, you're talking about refrigerated racks. They exist, but they're pretty niche.

In a typical data center, all this does is decentralize your cooling. Now you have many smaller (typically less robust) motors to monitor and replace, and many drain lines much closer to customer equipment and power.

Those units take up a lot more space, too, because of the insulation.

tbihl• 4 years ago

The elevated temperatures of the overheating components are such that fluid flow, not temperature difference, is the thing to go after, and it also has the advantage of being much simpler than adding a whole refrigeration cycle.

These problems start to read like problems from nuclear power, where sufficiently uniform flow is a huge deal so that various materials aren't compromised in the reactor.

beckingz2• 4 years ago

Condensation in most environments gets really rough on computers.

In theory you can eliminate condensation.

But in practice, there's a difference between theory and practice.

tonetheman• 4 years ago

i would hang the memory upside down so that condensation goes away from the electronics then put in a catch tray at the bottom for evaporation.

I am sure there is a lot more to it than that though... ha

dtx12• 4 years ago

why not integrate the ram into the package like apple does anyway and use a slightly larger SoC Cooling solution for the chips? Or just attach headspreaders to ram modules (like gaming modules) and add a fan for them like servers already do due to their general front to back airflow design. The only thing you can't do anymore is relying on the passive cooling of the chips own surface, something CPUs can't do anymore since the early 90s

SketchySeaBeast5• 4 years ago

Big old SoC really do seem like the future. CPU, GPU, RAM, motherboard controllers, throw all those different problem onto a big old die and optimize for cooling that guy.

AtlasBarfed• 4 years ago

Yeah I don't understand why a dedicated fan and other basic CPU cooling techniques don't apply here. It's probably because the DRAM industry doesn't want to change form factors and standards to a substantial degree...

... probably because they do the bare minimum to keep up with CPU design and routinely get busted for cartel price fixing and predatory pricing.

toast02• 4 years ago

Apple does a ram on top system right?

That's not going to be viable for servers ror two big reasons:

a) it would big a major capacity limitation; you're not fitting 8-16 DIMMs worth of ram ontop of the CPU. Sure, not everyone fills up their servers, but many do.

b) if you put the ram on top of the cpu, all of the cpu heat needs to transit the ram, which practically means you need a low heat cpu. This works for Apple, their laptop cooling design has never been appropriate for a high heat cpu, but servers manage to cool hundred watt chips in 1U through massive airflow, so high heat enables more computation.

Heatspreaders may make their way into server ram though (although not so big, cause a lot of servers are 1U)

Otoh, the article says

‘From zero to 85°C, it operates one way, and at 85° to 90°C, it starts to change,'” noted Bill Gervasi, principal systems architect at Nantero and author of the JEDEC DDR5 NVRAM spec. “From 90° to 95°C, it starts to panic. Above 95°C, you’re going to start losing data, so you’d better start shutting the system down.”

CPUs commonly operate in that temperature range, but RAM doesn't pull that much power, so it doesn't get too much above ambient as long as there's some airflow, and if ambient hits 50C, most people are going to shutdown their severs anyway.

snarfy• 4 years ago

The biological solution to leaks is clotting. Do we have cooling liquids that clot like blood does, say when exposed to oxygen?

chmod775• 4 years ago

It will clot inside the cooling circuit because of the air within it. Or will get within it.

However there are ways to prevent and detect leaks in current system with negative pressure: https://www.youtube.com/watch?v=UiPec2epHfc

xxpor• 4 years ago

the reliability there isn't particularly great ;)

kansface• 4 years ago

Great, not your computer can have a thrombosis or a stroke!

CoolGuySteve4• 4 years ago

I've always wondered why motherboards aren't placed at a slight angle like the wing of a car so that the air moving over it has a higher angle of incidence, higher pressure, and higher thermal capacity.

With the angle, you can also place cable connectors and whatnot on the bottom of the board so they don't obstruct airflow as much.

Basically, optimize PV = nRT inside the computer case at no extra cost other than a redesign.

SketchySeaBeast5• 4 years ago

I would assume because the things with the greatest heat have typically had such a requirement for active cooling that minor optimization wouldn't have helped much and for everything else you really didn't worry about (though my motherboard now has heat-pipes across the VRMs and my RAM and northbridge have got big old heat spreaders).

picture3• 4 years ago

It won't be a simple redesign to tilt boards "slightly" because manufacturing processes that are already honed in need to be completely retooled, with likely more complexity (man different length of standoffs per board?)

And additionally, there are only a few key components of a motherboard that need cooling. Most of the passive components like the many many decoupling capacitors don't generate significant heat. The components that do require access to cool air are already fitted with finned heat sinks and even additional fans. They interact with air enough to where a slight tilt cannot make a meaningful difference.

Basically just adding a small piece of aluminum to key areas will work better than angling the whole board

saltcured2• 4 years ago

I'm struggling slightly to envision the effect you are seeking. My motherboards don't tend to be flying through the air and so lack a well-defined angle of attack... :-) There already exist horizontal and vertical motherboard mounts in different computer cases, including ones that could be stood either way to suit the desktop. In my experience, this doesn't affect cooling that much.

I think the fan, internal baffle, and vent positions dominate the airflow conditions inside the case. So, rather than tilting a motherboard, wouldn't you get whatever you are after with just a slight change in these surrounding structures?

kllrnohj5• 4 years ago

You don't really need to pass any air over the PCB, though. Anything that needs cooling sticks up above it. Also the airflow through a case isn't perfectly parallel to the motherboard PCB anyway. GPU fans throw the air in all sorts of directions, including straight down into the motherboard. And so do CPU coolers.

Cables also don't really obstruct the airflow like at all.

dtgriscom3• 4 years ago

Car wings are angled because you're stuck with the direction the wind goes relative to the car (assuming you're driving with all wheels on a surface). Inside a computer, designers have a lot of freedom in where they direct the air flow. So, they plan the layout with space efficiency in mind, taking into account the needs of cooling.

kube-system2• 4 years ago

Heat pipes are the phase-change cooling solution that solves all of those issues. People don't really think of their cheap laptop as having a phase-change liquid cooling system, but it actually does.

dangrossman• 4 years ago

The article mentions that the automotive industry demands some of the largest temperature ranges for these parts. New cars are basically computers on wheels (especially something like a Tesla), and the cabin in a hot day under a glass roof can easily exceed 170F. Where will the freezer you build around all the computers go, and how will it be powered while the car is sitting parked in a lot?

outworlder• 4 years ago

I know this may not be a cheap solution, but why not start selling pre-built computers with active cooling systems? Refrigerant liquids like those used in refrigerators or water cooling could be an option.

Before going into water cooling, a change in form factor to allow for better airflow (and mounting of larger heat sinks) would be in order.

Water cooling would require a water cooling block, not sure how it would work with the current form factor.

So instead of integrating fluids within the computer, build powerful mini-freezers for computers and store the computer inside. Or split the warm transistors from the rest of the build and store only those inside the mini freezer, with cables to connect to the rest of the computer outside.

That's impractical. You are heat exchanging with the air, then you are cooling down the air? Versus exhaust the hot air and bringing more from the outside. You just need to dissipate heat, active cooling is not needed.

dtgriscom3• 4 years ago

Solutions that require increased complexity don't survive in the long run. I remember in the early '70s when my parents bought a new station wagon. This was right when the EPA was cracking down on emissions, and the manufacturers had to jump quick to compensate. Raise the hood on that thing, and you were greeted with a mass of hoses, pipes and components.

Since then, lots and lots of work has gone into making clean engines simpler. Open the hood of a gas-powered car these days and you'll find relative simplicity.

anthony_r• 4 years ago

we're lucky that this happens at 360 Kelvin and not at 200 Kelvin or even lower.

8jy89hui• 4 years ago

If our habitable temperature was cooler or hotter, we would use different materials to best reflect that environment. I’m not so sure it is luck

H8crilA• 4 years ago

So let's just use those that work up to 400K :)

somebodynew• 4 years ago

There is a bit of luck in even having any viable materials that work at the required temperature to choose from.

For example, humanity hasn't been able to find a single appropriate material for a superconductor at room temperature/atmospheric pressure despite significant research, but a civilization living below 100 K has a myriad of options to choose from. Superconductors are high technology to us, but if your planet is cold enough then superconducting niobium wire would be a boring household item like copper wire is for us.

tinus_hn2• 4 years ago

It’s lucky in the northern hemisphere there is an easily recognizable star pointing almost exactly at the North Pole, which makes navigation much easier.

It’s lucky some available material worked the right way to make a transistor.

It’s lucky some person smart enough to make that work got to work on that.

History is full of lucky coincidences like that. How many Einsteins have died out in the jungle, without access to our scientific knowledge or a way to add to it? For most of history and partly still today, being a scientist wasn’t possible for just about anyone, you had to be from the right family. It’s all about luck.

p1mrx• 4 years ago

Note that "kelvin" is lowercase: https://english.stackexchange.com/questions/329629/is-kelvin...

klodolph• 4 years ago

It's a combination of chemistry and geometry (and other factors). Maybe there's some luck.

There are ICs and components built for operating in extreme environments, like drilling. You can get SiC (silicon carbide) chips that operate above 200°C (473K), if that's important to you. There are also various semiconductors that are worse than silicon at handling high temperatures, like germanium. Old germanium circuits sometimes don't even work correctly on a hot day.

If we lived at 200K, I'm sure that there's a host of semiconductor materials which would be available to us which don't work at 300K.

YakBizzarro• 4 years ago

well, depends how you define lucky. at cryogenic temperature, the leakage current of a transistor is so small that you virtually don't require DRAM refresh. I tested DRAM cells with discharge times of hours, and the transistor was not at all optimized. See https://www.rambus.com/blogs/part-1-dram-goes-cryogenic/ (not my work)

brutusborn• 4 years ago

I loved this part at the end: "By contrast, allowing a temperature increase for chips in large data centers could have surprising environmental benefits. To this point, Keysight’s White recalled that a company once requested JEDEC increase the spec for an operating temperature by five degrees. The estimate of the potential savings was stunning. Based on how much energy they consumed annually for cooling, they calculated a five degree change could translate to shutting down three coal power plants per year. JEDEC ultimately compromised on the suggestion."

gjsman-10002• 4 years ago

Only three? That's not an immediate win. What if the temperature increase causes just so slightly more failures, causing so slightly more replacements, and each replacement requires energy to make, ship, install, replace, recycle, the effects of increased demand... What if it lasts slightly less long, causing more early failures and eWaste? After all that potential risk, is it a benefit still, and if so, how much?

We don't know and it is hard to know - but I don't blame JEDEC and would not call it a "compromise" on their part like it was a superior option.

picture3• 4 years ago

Cmon.. logically it has to be more than just three right? "Cooling efficiency" sometimes come in unit of W/°C difference, so I'd imagine that a few more degrees would be a huge deal.

uoaei• 4 years ago

One single company changing one single design parameter and enabling savings on the scale of multiple power plants? That is as immediate as wins get.

Spooky235• 4 years ago

For one company? That’s pretty impressive.

When I was on an architecture team that consolidated ~80 datacenter to 3 circa 2010, this was a key dollar driver. We raised the temperature ~6 degrees from the average temp, which meant kicking out a few vendors initially. The cost savings for doing that was essentially the total operational costs of 5 datacenters.

The annual failure rates for the hardware did not change at all by any metric. Number of service impacting hardware failures went to zero due to the consolidation.

In general, if you operate within the operating ranges of your hardware, you won’t have failure. You will have complaints from employees, because computers will operate at temperatures not comfortable for humans.

JJMcJ2• 4 years ago

I've heard of some large companies that run their data centers hot.

Cheaper to have a slightly higher failure rate, or have the computers throttle their clock speed, than to pay for extra air conditioning.

klysm• 4 years ago

Hard to do the math here though because you don't know the failure statistics in advance. Kind of a multi-armed bandit problem of sorts.

woleium• 4 years ago

Could also be more likely that it's cheaper to extend the life of an older DC by accepting higher temperatures and failure rates than to upgrade the hvac to accommodate newer higher density designs

__alexs6• 4 years ago

Maybe RAM will finally get more than a 4mm thermal pad and a random bit of Alu for cooling. Seems like most cooling designs have treated RAM as even more of an after thought than VRMs up until recently.

Even in most servers the accommodation for RAM cooling has basically just been orientating the DIMMs to line up with airflow. They are still packed together with minimal clearance.

AshamedCaptain• 4 years ago

I am not sure how much this is related to external cooling versus actually internal thermal dissipation. DDR JEDEC standards have actually decreased power consumption on every generation.

__alexs6• 4 years ago

They have reduced voltage but power consumption per sq-mm has gone up with increased densities. Many people run DRAM at above JEDEC speeds which usually requires higher voltages too.

Peak power consumption of DDR4 is around 375mW/GB @ 1.2V, DDR5 drops this about 10% but also increases the maximum density of a DIMM by 8x to 512GB which is like, 150W for a single DIMM.

deelowe2• 4 years ago

There's been talk of eliminating sockets for years. Something has got to give.

zeroth32• 4 years ago

more compact chips will have a higher failure rate. Not a great idea for servers.

jnwatson• 4 years ago

You can still actively cool socketed RAM.

dcow2• 4 years ago

cooling has basically just been orientating the DIMMs to line up with airflow

Isn't that server cooling in a nutshell? Ram high volumes of airflow through the chassis with stupidly loud fans and hope the parts stay cool?

dodobirdlord2• 4 years ago

You still need to conduct the heat away from the sources to a radiator of some sort, since cooling is proportional to surface area and it’s much easier to increase surface area by adding fins than by increasing airflow. You can only speed up the air to a certain point, past which better cooling becomes a matter of shaping the components for more contact with the air.

throwawaylinux• 4 years ago

I'm not in the field, but I have seen some of the work done by thermal engineers designing new server systems, and no that's not the case.

Cost, noise, and physical box size are to be minimized, and various compute/storoage/io capacity are to be maximized. Component placement and orientation relative to one another, baffles, fan characteristics heat sink design, all play a role.

They don't just throw parts in a box and add fans until all components are within operating temperature. Gross air flow is one lever, but it has costs and limits like all the others. Designs are modeled with heat and fluid simulations.

wallaBBB• 4 years ago

Question that comes to mind - Is M2 (with thermal issue on new Air) affected considering how RAM is packed there?

Toutouxc2• 4 years ago

What thermal issues? All I've seen so far are people who don't seem to understand how passive cooling works, despite the M1 Air being out for two years and working the same way.

webmobdev• 4 years ago

Perhaps OP came across this recent article - Reviewers agree: The M2 MacBook Air has a heat problem - https://www.digitaltrends.com/computing/m2-macbook-air-revie... .

nostrademons2• 4 years ago

The M2 chip generates more heat than the M1, with 20% more transistors and about a 12% higher clock speed. M2 Mac Pro has thermal issues compared to M1 Mac Pro as well, even with the fan.

alliao• 4 years ago

I think it's just expectations, people expected a brand new laptop should be able to perform fully without throttling. And that the throttling is designed as a crutch to prevent hardware damage.

One can draw different thresholds, ie. 100% performance for x amount of time before throttling happens etc..

nonrandomstring4• 4 years ago

Still waiting to see the first micro-engineered Stirling engine that can self-cool. Any physicists care to comment on why that won't work yet, or ever?

Chabsff• 4 years ago

You can only cool something by making something else warmer by a larger amount. The heat has to go somewhere, and moving that heat in any non-passive way will invariably produce yet more heat in the process.

ta8645• 4 years ago

You can only cool something by making something else warmer by a larger amount.

Why isn't it also true that you can only make something warmer, by cooling something else by a larger amount?

The movement of electricity generates waste heat, why isn't that process reversible? Making the heat disappear into a cold wire, rather than just dissipating into the atmosphere? (not suggesting it's would be easy or even practical).

nonrandomstring4• 4 years ago

I think some people are interpreting that as a joke. I'm not talking about a net gain of energy or any crazy perpetual motion machine. Think of something like a "heat brake". Differential heat energy can be converted to mechanical work. Some of that can be used to cool the system elsewhere, creating a negative feedback loop. Another way to think of such a system is like the "reluctance" of an inductor.

With present thermoelectric effects, using a Seebeck junction to generate current for a fan is hopelessly ineffective. But is that necessarily the case for all designs which could help to hold a system under a critical temperature when heat spikes.

bilsbie• 4 years ago

It’s at least partly caused by climate change too

jhallenworld• 4 years ago

Maybe DRAM becomes non-viable, so switch to SRAM. Which is denser, 14 nm DRAM or 5 nm SRAM?

55873445216111• 4 years ago

SRAM is ~10x higher cost per bit (due to memory cell size) than DRAM

to11mtm2• 4 years ago

DRAM.

IIRC TSMC's 135MBit 5nm example is 79.8mm^2, although that's got other logic.

In the abstract, a 0.021 square-micrometer-per-bit size [1] says you'd need about 21mm^2 for a gigabit (base 10) of 5nm SRAM, without other logic.

Micron claimed 0.315Gb/mm^2 on their 14nm process, [2] so somewhere between a factor of 6 and 7.

That said, my understanding is that there is some sort of wall around 10nm, where we can't really make smaller capacitors and thus the limitation on things. (This may have changed since I last was aware however.)

(There is also the way than 'nm' works these days... but I'm not qualified to speak on that)

Also, AFAIK SRAM is still broadly speaking more power hungry than DRAM (I may be completely out of date on this though...)

[1] https://fuse.wikichip.org/news/3398/tsmc-details-5-nm/

[2] https://semiengineering.com/micron-d1%CE%B1-the-most-advance...

superkuh• 4 years ago

RAM has been parallel for ages. IBM's new POWER10 achitecture switches to serial control of ram with firmware running on the ram sticks. As long as complex mitigations and monitoring are going to be required this might be the way to go.

ip262• 4 years ago

How does that help? Adding a SERDES in the middle rarely decreases total power.

dylan604• 4 years ago

Next, we'll have a generation of mobile devices that will be liquid cooled. Of course because of the miniturization, there will be no way to refill the liquid coolant without getting a new device. This will naturally happen before the batteries die creating an even shorter life cycle in devices. Sounds like a perfect pitch for an upcoming WWDC type of event.

dusted2• 4 years ago

Sounds like nothing a little liquid nitrogen can't fix.

(as a standard metric, about once every 64 milliseconds)

64 milliseconds? wow.. I thought they'd need refreshing way more often

rowanG077• 4 years ago

A lot more refresh commands are issued. A single cell holds data for 64 milliseconds. But a refresh only refreshes a subset of the RAM, a full row. So to refresh an entire chip you need to refresh all rows which can be hundreds to thousands of rows. So in practice you need to issue refreshes every few microseconds to ensure you hit every cell at least once every 64ms.

dusted2• 4 years ago

Thanks for clarifying :)

userbinator• 4 years ago

Does DRAM really get that hot now? I remember the huge amount of products aimed at cooling DIMMs appeared around 15-20 years ago and largely dismissed them as unnecessary "bling", and closer to a decade ago, on a regular desktop PC, had some that came with heatspreaders; nonetheless, after a weekend-long Linpack run to ensure stability of an overclock, during which the CPU was constantly well above ambient at around 70C, reached in and cautiously felt the heatspreaders on the DIMMs --- and was surprised to discover that they were only slightly warm, probably around 40C.

kasabali• 4 years ago

Run a RAM heavy stress test like memtest86 and you'll see heatspreaders on RAM do get hot.

RustyRussell• 4 years ago

From TFA:

"And it’s not just servers. With about 8 billion transistors on a single die, a mobile phone can get so hot it might need to spend a few minutes in a refrigerator. When that happens, the apps will fail to function correctly."

Such a weird way to phrase this! All I can think is ".. and it's hard to use your phone when it's in the fridge"?

p1necone• 4 years ago

I have one of those magnetic phone holders that attach to the aircon vents in your car. Every time I use it with the air con on in the summer I think about how my phone could probably use the extra cooling for a performance boost, except it's not really needed just for running google maps.

And then when I do the same thing in the winter I have to be careful about turning the heater on because the hot air will make my phone auto shut down from the heat.

walrus01• 4 years ago

They could do a lot by just putting 1.5mm height aluminum fin heatsinks on DIMMs as compared to the extremely cheap flat heat spreaders commonly used now.

hoseja• 4 years ago

It sucks being stuck in the silicon hole. Somebody, seriously invest in high-temperature semiconductors!

DRAM thermal issues reach crisis point

Related Stories