The Dark Silicon Problem and What It Means for CPU Designers (2013)

http://www.informit.com/articles/article.aspx?p=2142913

by plainOldText • 8 years ago

78 68 8 years ago

Why don't they start making CPUs 3 Dimensional like a cube with 6 "processors" each with multiple cores as its "sides" with the pins on the opposite sides of the cube wall. Seems to me more internal volume might allow for more cleaver head distribution channels

asfasgasg• 8 years ago

If I understand correctly, neither heat dissipation nor existing manufacturing techniques are amenable to this approach. Also modern CPUs do have more than a dozen layers IIRC.

woah• 8 years ago

I think op is talking about 6 flat normal chips as the sides. This would allow for cooling stuff inside the cube. Maybe having only 5 of the sides as chips would make it even easier to have a heat sink. The center of the cube could be copper or something.

gameswithgo• 8 years ago

less surface area per transistor makes the heat problem worse. they are already a little bit 3d though.

blackflame70004• 8 years ago

What if the cooling came from the pin side of the chip? Then the surface area is the same

neilmovva• 8 years ago

That still means multiple silicon dies, which we have known how to do for a while (see: Intel Core 2 Quad from 2006, and more recently AMD Epyc).

Having more dies lets you dissipate more heat, but then it's kinda hard to build low-latency / high-bandwidth interconnects between the dies. Inter-die buses go over a PCB or interposer, which impose higher parasitic capacitance and make it difficult/expensive to run wide interfaces. That's why techniques like "dark silicon" allocation are important - it allows us to get more perf in a single die.

wongarsu• 8 years ago

For heat dissipation you want the most surface area per volume (because you can only transfer heat away in the surface area). The optimal arrangement for that would be a huge, flat, one atom thick surface.

Another goal we have is low latency (and high clock rate, which is depenend on low latency), which suggests putting everything in a qube or even a shere. So we compromise somewhere in the middle with a square with a few layers.

perl4ever3• 8 years ago

"For heat dissipation you want the most surface area per volume (because you can only transfer heat away in the surface area). The optimal arrangement for that would be a huge, flat, one atom thick surface."

Made me think of this:

https://en.wikipedia.org/wiki/Menger_sponge

Say we have roughly 300 sq mm, that's about 17.32 mm square, which is 8.24e+7 silicon atoms (0.21 nm) across.

Then we have a surface area of roughly 1.36e+16 atoms - the area x 2.

If we make a fractal sponge down to the limit of single atoms, then that's about 16.5 cycles of removing cubes from a 17.32 mm cube. Let's ignore the difficulty of doing it half a time. According to the formulas from wikipedia, the result has a volume of about 0.7% of the original, with 2.4e28 "sides" of atoms exposed.

So the third dimension gets you about 1.8 million times the surface area. I suppose this isn't nearly as good as 4e10 flat sheets with 1 atom separation between each, but you could argue it's more practical because everything is connected...

throwwit• 8 years ago

I think a dimpled fabrication would be best... with whatever amounts to heat sinks on both sides.

deepnotderp6• 8 years ago

1. Heat dissipation is now ~n x m times worse where n is your transistor layer count. And where m is the increased thermal resistance*

2. Power delivery is now ~n times worse where n is your transistor layer count. *

3. Connections between chips are very slow, power hungry and expensive. Fabrication of "monolithic" 3D is temperature wise painful and usually results in crummier transistors.

With that being said, innovative 3D integration methods in specific applications can help a lot. Shameless plug: we at Vathys do this for deep learning chips.

* to a first order of approximation

tormeh• 8 years ago

The heat can be tackled in part by pumping water through holes in the CPU. I believe it was IBM that came up with this. Can't tell if it's feasible or not.

deaddodo• 8 years ago

They do use some 3D techniques inside dies. Tri-gate transistors and 3DICs, for insurance

deepnotderp6• 8 years ago

Such a "side cube" configuration would probably lead to longer wires than is useful.

wolfgke• 8 years ago

Why don't they start making CPUs 3 Dimensional like a cube with 6 "processors" each with multiple cores as its "sides" with the pins on the opposite sides of the cube wall.

Not directly an answer to this specific question, but a (somewhat) colleague who writes his PhD thesis about 3-dimensional chip design made a popular scientific lecture about this topic. As I understood it, the central problem is that it is very hard to produce chips with multiple (lots of) layers (where you want to have interconnects inbetween). In particular producing the interconnects between the layers is really hard if they can lie "everywhere" instead of only at the border. There exist multiple ideas how this might be done (e.g. drill holes with high-precission lasers into the substrate and try to fill them with something conductive), but none of them as of today "really works" (at least if we are talking about chips with somewhat more than 4 layers and interconnects everywhere inbetween).

deepnotderp6• 8 years ago

Fun fact: the through silicon via was invented by William Shockley himself

burnte• 8 years ago

Just the reverse. As volume increases, the relative amount of surface area decreases.

Phillipharryt• 8 years ago

From the article "The heat generation per unit area of an integrated circuit passed the surface of a 100-watt light bulb in the mid 1990s, and now is somewhere between the inside of a nuclear reactor and the surface of a star. "

I can't tell if this is hyperbole or not, it amazes me but no amount of googling is coming up with a useful answer. Is anyone able to confirm or deny it for me?

andlier• 8 years ago

After a quick napkin calculation+google it seems the sun has around 20kW of power per square centimeter. So not entirely unfeasible that a cooler star, or a nuclear reactor is closer to the typical 10-100W/cm2 of a modern cpu/gpu. Still some orders of magnitude off from our closest star. (Hope calculation is correct)

perl4ever3• 8 years ago

Looking at the wikipedia page on red dwarf stars[1], it appears a small star (M9V) might have 8% of the sun's radius and 0.015% of its luminosity. Thus, it would have about 156 times less area, and 2.3% of the output per area of the sun. So taking your figure as given, that means it would be less than 500W / cm^2.

[1]https://en.wikipedia.org/wiki/Red_dwarf

ars2• 8 years ago

20kW of power per square centimeter

I'm getting 6300 W/cm^2 (see: https://news.ycombinator.com/item?id=17589371)

Cybiote• 8 years ago

The temperature of the sun at its surface is ~5772 Kelvin. To get power per unit area, use Stefan-Boltzman: σ * T^4, σ * (5772 K)^4 ≈ 6294 W/cm^2. Dividing the sun's luminosity (power) by its surface area will also give a similar value.

A 815 mm^2, 250 W GPU will be 250 W / 8.15 cm^2 ≈ 31 W / cm^2.

Zanni2• 8 years ago

Temperature inside a nuclear reactor: 300 degrees Celsius. [1]

Temperature on the surface of the sun: 5,600 degrees Celsius. [2]

Temperature of a CPU: ~75 degrees Celsius. [3]

[1] http://academic.brooklyn.cuny.edu/physics/sobel/Nucphys/pile...

[2] http://coolcosmos.ipac.caltech.edu/ask/7-How-hot-is-the-Sun-

[3] https://www.computerhope.com/issues/ch000687.htm

readams• 8 years ago

not sure why you're being downvoted, but it's worth mentioning that the temperature is not what matters here but rather the heat generation per area. The temperature is a function of how quickly the heat can be pulled away as well as the heat generation.

earenndil• 8 years ago

per unit area

A CPU is a lot smaller than both a nuclear reactor and the sun.

a_wild_dandan• 8 years ago

(Intensity) = (Power) / (Unit Area)

(Sun's Radius) = 695 x 10^6 m

(Sun's Power) = 4 x 10^26 W

(CPU Output) = 75 W

(Die Size) = 37 mm x 37 mm

(Sun Intensity) = (4 x 10^26 W ) / (4 x pi x (695 x 10^6 m)^2) ~ 66 x 10^6

(CPU Intensity) = (75 W) / (0.0014 m^2) ~ 53 x 10^3

I'm getting a several orders of magnitude higher W/m output for the sun. Perhaps I made an algebra mistake?

namibj2• 8 years ago

Yes. You can handle 100W on a 300mm^2 chip. Or 300W on a large one, compare AMD Ryzen/ Nvidia Volta (MXM/V100).

Tuna-Fish• 8 years ago

Your CPU is huge. For example, the highest-performance single-chip Pinnacle ridge (AMD Ryzen 2 2700X) is 213 mm² and has a TDP of 105W. That's roughly an order of magnitude difference.

ars2• 8 years ago

From:

http://www.wolframalpha.com/input/?i=Sun+%7C+luminosity+%2F+...

The sun is: 6300 W/cm^2

inside of a nuclear reactor

Inside means volume? That's not the same unit, so I'm not really sure how to calculate that.

Interestingly, on a per-volume bases, the sun's output is quite low. The sun is just very very large.

See: http://www.wolframalpha.com/input/?i=Sun+%7C+luminosity+%2F+...

And you find the sun is only: 1.383×10^-6 horsepowers per gallon :) Or if you insist: 0.2725 W/m^3, which as you can see is really really low.

zaarn• 8 years ago

Intel's latest CPU innovation, the i9 "Industrial Cooling Required" Edition with 28 cores, would come close to a nuclear reactor and does manage to replicate the heat output usually experienced during atmospheric reentry from orbit.

I doubt it comes close to the surface of the average star but certainly close to some of the cooler red dwarf stars and probably most brown dwarf stars.

dnautics3• 8 years ago

keep in mind that 'the surface of a star', for various definitions of 'surface' is actually "cooler than you might think", because it's rather diffuse and more than made up for in total output by the sheer size of the star.

jarym2• 8 years ago

We just need a clueless CIO to turn up and ask if 'Cloud' will solve the problem :D

stephengillie• 8 years ago

Cloud is busy facing off against Sephiroth. What's Batman up to?

melbourne_mat• 8 years ago

I love the cynicism :-)

jarym2• 8 years ago

hehe - not everyone got it from the downvotes I got!

bogomipz5• 8 years ago

I had a question - the author states:

"The most obvious is the instruction decoder, which is near the start of the pipeline, and is responsible (in the loosest possible terms) for passing the inputs to each of the execution units."

Why would it "in the loosest possible terms"? Isn't this "precisely" the job of the decoder?

kabdib• 8 years ago

IIRC in many chips the decoder stage translates native instructions into micro-ops, which are RISC-like and the main food for execution units. The translation is not necessarily a simple one (one native instruction is often more than one micro-op, and it's possible to collapse multiple native instructions -- especially stuff like prefixes -- into one or more micro-ops).

bogomipz5• 8 years ago

Ah OK that make sense. This is likely what the authors means here by "loosely." Cheers.

bogomipz5• 8 years ago

I had a couple of question about this bit of history mentioned in the article. I'm hoping someone could shed some light on this:

"You can emulate floating-point arithmetic by using integer instructions—but taking 10–100 times as long."

Exactly how is/was floating point arithmetic emulated using only integers?

Why is that range given an order of magnitude? Is this dependent on the precision I'm guessing?

phkahler• 8 years ago

We had floating point when programming in BASIC on old 6502 8bit computers. There were software routines for doing the math. You know, multiply the mantissa, add the exponents... If someone gave you pointers to a couple 4-byte chunks of data and told you to write code to do floating point multiply on the contents using only C-char variables, what would you write? That's why it's 100 times slower than a nice modern fmul. It wasn't quite that bad, they could use the carry flag which isn't available in C.

sehugg• 8 years ago

Yep, here's the old Woz/Roy Rankin 6502 code for log, exp, conversions, and basic math: http://www.6502.org/source/floats/wozfp1.txt

bogomipz5• 8 years ago

Thanks these are all good reads and explanations. I guess I just have never had to think about FPU emulation before and it kind of threw me for a loop. It's amazing what we can take for granted now I guess :)

Cheers.

brandmeyer• 8 years ago

If you have a copy of Knuth's Art of Computer Programming, the section on seminumerical algorithms covers software floating point.

Another source is the Handbook of Floating Point Arithmetic, which covers both hardware and software methods in detail.

An implementation in source code may be found in libgcc. IIRC their ARM assembly code versions are some of the fastest routines available anywhere for software floating point emulation. While they are in assembly, ARM assembly is fairly intelligible.

Finally, you can probably come up with 90% or more of the correct solutions yourself if you start with the definition of a floating-point number as (-1)^s * 1.m * 2^(e - bias) and grind through the algebra with the bias as a constant. The theorems that prove the minimum number and nature of the guard bits to get correct rounding are another matter, but you can start off by computing the equivalent infinite-precision terms and then rounding.

pkaye• 8 years ago

A floating point number consists of 3 parts: mantissa, exponent and sign. These are usually packing into a 32-bit or 64-bit value. So you can split them back into 3 integers and do the integer adds, multiply and shifts to get the result and pack them back. It is not hard to implement a basic version but to make it fast and comply with the standard is a bit more work.

JudasGoat• 8 years ago

The author states "For every watt of power the CPU consumes, it must dissipate a watt of heat." It was always my understanding that that in electrical devices, (with the exception of heaters) the amount of heat produced was inversely proportional to the efficiency of the device. So is it really true that all energy provided to the CPU or SOC is dissapated as heat?

Elrac2• 8 years ago

The author's sentence is essentially a truism, because fiddling with information _as such_ and in theory doesn't use up any energy. As practically implemented in current CPU electronics, though, information needs to be communicated from point A to point B as a change in voltage. To convey that voltage change means having to move some amount of electrical charge into or out of the tiny capacitor that is a transistor's gate, via the non-perfect conductor that is the doped-silicon trace between the two points. Resistance saps part of the energy moving those electrons around and converts it to heat.

dnautics3• 8 years ago

fiddling with information _as such_ and in theory doesn't use up any energy.

that's not true at all. Any time you destroy information (for example an and gate can destroy information) you use energy.

jamesough• 8 years ago

Yep, it all ends up as heat. Energy is conserved, so if it didn't end up as heat, it would have to go somewhere, and there isn't really anywhere else it could go.

Related topic: reversible computing. You might wonder if a computation need take any energy at all: it turns out that an irreversible computation (e.g. NOR, XOR, AND etc) is physically guaranteed to waste energy, but if you make sure your compute steps are always reversible (i.e. each input maps to one and only one output) then you can theoretically compute for free (the Feynmann lectures on computation cover this well).

But these energies are negligible compared to the wattage that goes through a standard CPU.

cgdcraig• 8 years ago

That's true only of devices that don't convert the energy into work(motors), light(LEDs) or other voltages(transformers). All other electronics convert all the power into heat. Basically any computer is a heater that so happens is able to do math as well. Energy is consumed for doing a computation but it's 6-7 orders smaller than what the heat produced

The Dark Silicon Problem and What It Means for CPU Designers (2013)

Related Stories