Knight’s Landing: Atom with AVX-512

https://chipsandcheese.com/2022/12/08/knights-landing-atom-with-avx-512/

by rbanffy • 4 years ago

75 34 4 years ago

rektide2• 4 years ago

Finally! One of the excellent excellent Chips & Cheese coverages of a really weird chip! Nice!

Intel spends nearly 40% of the core’s die area to implement wide AVX-512 execution, so Knight’s Landing gets some incredibly high throughput for a low power architecture. In fact, it almost feels like a small OoO core built around feeding giant vector units

Well said!

There's some interesting details here. I love how the article tests SMT1 (aka no SMT), SMT2, and SMT4, and shows how things change. The re-order resources getting subdivided as SMT scales up is obvious but a neat hack, a real neat hack, for repartitioning core resources- on a core (Atom) that wasnt doing reordering much at all, at that point.

Also a good reminder that this almost 10 year old chip was using DDR4. I'd forgotten that DDR4 has been around for so so long!

The chip has some really monstrous capabilities. Modern flagship consumer GPUs are hitting the 1Tbps mark in memory bandwidth. AMD's upcoming Navi 3 has 96MB of L3 cache good for 5.3TB/s. Meanwhile, here's a chip that's 10 years older that has 16GB of onboard DRAM that C&C gets up to 4.5TB/s. Damn ya'll. And with huge AVX-512x2 per core, as mentioned in the top quote... this thing was such a wonderful & fascinating beast.

I didn't realize the previous incarnation- which I bought used/hella-cheap a long time ago & wanted to toy with but never did- was P54C based, aka an original-ish Pentium. That's wild.

This article is just so good. On and on, with every little characteristic and quirk. This is like a perfect "Speaking for the Dead", out of Enders Game. Alas that the world just wasn't awesome enough to put this shit to use at real volume. It's like an ultra-flexible on-the-fly reconfiguring DSP?

klauspost• 4 years ago

Reading it, it seems very similar to the Playstation 3 Cell, with its fate mirroring it very much.

A highly specialized processor that has very high computational throughput for specialized operations, but a quite limited scalar unit.

In both cases you really have to write software specifically for it to get it to perform with any reasonable speed. If you do that, you get great value, but any existing code will need significant modifications to perform.

Granted AVX512 is (now) more common than SPE code ever became. It is slightly better than the Itanium approach, but scalar performance (especially single threaded) will have limited the value you get from this CPU, unless you write your own software and it can utilize AVX-512.

rbanffy5• 4 years ago

It was much easier to program though. The Cell had two different ISAs, one for the PPUs and one for the SPUs. The SPUs also didn't have direct access to memory and the PPUs had to manage task assignment and completion, as well as setting up DMA transfers between SPU memory and main memory.

The Phi was closer to the Sun Niagara family - lots and lots of simple, slow, cores, with the note that, in the case of the Phi, the weakling x86's had mighty SIMD abilities while the Niagara had more or less standard SPARC stuff.

Neither will have amazing performance unless you have at least as many threads running as you have cores, and most of the time, at least twice as many. For single-threaded code, they were on the slow side.

Still, I always suggested people use Phis to develop because they'd get a taste of future computers. Nowadays a decent laptop will have half a dozen cores and, unless your mail client has that many threads, it'll not feel as fast as it could be.

janwas5• 4 years ago

Great in-depth article. One especially interesting data point relevant to ongoing discussions about AVX-512 area cost:

40% of 2.93 mm2 per core is AVX-512, so 1.14 mm2. This is a large fraction, but as the article says the core is basically a minimum wrapper around the vector unit, with rather weak L1i/branch predictor/store buffers.

Let's put that in the context of modern chips. 14nm density was 44.67 MTr/mm2 so that's 50.9 MTr for AVX-512. To compare with 5nm, let's use density of TSMC N5 (138.2 MTr/mm2) to get 0.37 mm2.

So that's about 10% of an Apple M1 Firestorm core to enable 5-10x speedups vs scalar code. Sounds worthwhile to me. We can now stop saying that "AVX-512 is a huge fraction of modern cores", and "give us more cores instead". Let's instead use the hardware we have :)

celrod2• 4 years ago

Good analysis. It's also worth pointing out that this is for 2x 512 bit FMA, which is more than client Ice/Tiger/Rocket lake or Zen4 have.

Personally, I bought HEDT (Skylake-X and Cascadelake) because I wanted 2x 512 bit AVX512. Glad it's cheap in terms of area, and I'm hoping we'll get more options with great vector performance in the future.

janwas5• 4 years ago

Good point about the second FMA. I'm not certain it's the best tradeoff, Genoa only has two half-width FMA.

I share your hope for more focus on vectors. It's also up to us software devs, CPUs will not invest as heavily if we don't use it.

moonchild• 4 years ago

give us more cores instead

That should be a pretty obvious conclusion--higher-clocked cores is better than wider cores, and wider cores is better than more cores.

Communication cost between concurrency domains for superscalar/supervector: min <1ns

Communication cost between concurrency domains for multiprocessors: min 10-100ns

janwas5• 4 years ago

:) Obvious or not, one does still occasionally hear "give us more cores instead".

I agree about the communication cost. Higher clocks are harder - quadratic increase in power. For shared-nothing problems, an array of wimpy (low frequency) cores is pretty good. It seems to me that wider, lower-frequency cores are a good compromise: beefy enough (thanks to vectors) to reduce the number of cores required, while still power-efficient due to both vectors and lower clocks.

phonon• 4 years ago

Knight’s Landing AVX-512 units run at about 1.5 GHz at peak (even when only a few are in use.)

The ones in Sapphire Rapids run 2x+ faster. Likely there is more pipelining and many more transistors used to reach those clock speeds.

janwas5• 4 years ago

Physical design is outside my area of expertise, but I understand that a large part of the increase may be due to improved fab. 15-20% speed improvement per node would account for the majority of the clock speed gain. Also, with smaller feature sizes comes the ability to trade away some of the density for further increases in the frequency, without actually requiring more transistors.

rbanffy5• 4 years ago

"AVX-512 is a huge fraction of modern cores", and "give us more cores instead". Let's instead use the hardware we have

If what you do can use a wide SIMD pipeline, then, by all means, get AVX-512.

If what you want is to have lots of branchy processes running serving different things, then more cores is a better idea.

I'd suggest the best of both worlds - asymmetric cores. Put a couple that have wide SIMD pipelines, alongside a couple others that don't, but may be smaller and more numerous.

AVX-512 tasks that are scheduled to non-AVX-512 trap and are scheduled to AVX-512 cores. Tasks that don't trap continue running wherever the kernel thinks is best.

ksec• 4 years ago

Yes. That was part of the plan for Intel. Remember that Intel was suppose to have 7nm by 2019, which is roughly equal to TSMC N5.

Moissanite• 4 years ago

I have a book ("Intel Xeon Phi Processor High Performance Programming") which has sat un-loved on my shelf since 2018 - alas I never got chance to play with one of these things. The last batch was put to good use by an oil and gas company though: https://www.hpcwire.com/2019/03/13/oil-and-gas-supercloud-ta...

rbanffy5• 4 years ago

It's a shame mere mortals never got to see the Knights Mill family. Since KNL they could be the main CPU of your workstation and Supermicro even had motherboards that could use it. And Knights Mill added virtualization, which made them a lot more useful for a developer.

Unfortunately, I never saw used ones on eBay.

rektide2• 4 years ago

Some comments on the submission from 16 hours back: https://news.ycombinator.com/item?id=33916480

rbanffy5• 4 years ago

Can they get merged?

dang• 4 years ago

I've merged the threads into the current one. It's later, but it's still on the front page.

muziq• 4 years ago

Great read, and always loved these processors.. My first introduction to AVX-512 and the beginnings of a love affair :)

convolvatron• 4 years ago

didn't this whole processor line get cancelled?

wmf• 4 years ago

Yes, they were canceled several years ago.

bell-cot• 4 years ago

Not denying that there are use cases...but my first reaction to "Atom with AVX-512" is "Fiat 500 with a 5th Wheel".

https://en.wikipedia.org/wiki/Fiat_500_(2007)

https://en.wikipedia.org/wiki/Fifth-wheel_coupling

mk_stjames• 4 years ago

The Xeon Phi coprocessor pci-e cards with similar CPU can be had on ebay for like $60 these days. Sadly I haven't seen any good writeups on setting up an environment to run such cards in a homebuilt system. I think it would be neat to build a little mini-supercomputer with 8 of them stacked in one of those mining motherboards with 8 pci-e slots. But I'd have no idea where to start with the toolchain needed to compile an run code across them.

thechao• 4 years ago

The original cards couldn't be thermally throttle, safely. Whenever the union guys came to our floor they were required to wear PPE (helmet, goggles, earplugs, earmuffs, and a vest). It was as loud as a high powered low quality vacuum cleaner. Staying within 5' of the towers, when running, without ear protection was an OSHA violation.

Y_Y• 4 years ago

I remember playing with them at the local supercomputer facility. I thought it was so funny that you were expected to ssh into a pcie device.

cpgxiii• 4 years ago

The only Xeon Phi products to be available in PCIe form are the first-gen KNC cards. These are not Atom-based, they do not share the x86_64 ABI, and the only effective toolchain for them is older versions of the proprietary Intel compiler.

While they were quirky at the time (and I got some neat simulations out of them), they were a massive pain in the ass in every other way:

- Requires a motherboard that supports large PCIe BAR addressing, much more common now, not at release time (and good luck getting this working with multiple Xeon Phi coprocessors on a non-server motherboard) - You likely need enough host memory to cover each of the cards (so 8GB * number of cards) - You'll need to patch the Intel MPSS kernel driver to work with newer kernels - You'll need to patch the Intel MPSS tools to work on anything that isn't ~2015 RHEL - You'll want to buy the water cooling kit that alphacool made for them, it's the only way to keep temperatures and noise to a reasonable level

Solve all of those, and you get a (fairly cheap) coprocessor reasonably good at parallel yet branchy tasks. It's much easier to use the second generation of KNL (or third-generation KNM) which share the standard X86_64 ABI and can boot a modern Linux distro and you can use a normal toolchain).

rbanffy5• 4 years ago

or third-generation KNM

Never found Knights Mill available anywhere. My understanding is that they were only available to integrators in the HPC space.

shoo• 4 years ago

Tom Forsyth gave a talk at handmade seattle 2019 titled "SMACNI to AVX512 the life cycle of an instruction set"

https://guide.handmade-seattle.com/c/2019/talks/lifecycle-of...

sorenjan2• 4 years ago

I wonder how well this would run a raytracer, aren't those pretty branchy?

berkut• 4 years ago

Fairly well, Intel's Embree raytracing kernels have support for Xeon Phi, although going more than 8-wide with incoherent rays (i.e. pathtracing) starts to require more and more sorting to batch things up for decent utilisation...

sorenjan2• 4 years ago

Would you mind giving an easy explanation of the difference between raytracing and pathtracing? I find it a bit confusing.

chazeon• 4 years ago

My colleague are still using KNL cpus on Stampede2 to run molecular dynamics simulations. They are just “cheap” in terms of job accounting, because there unloved because they are old and cannot catch up with state of art AMD cpus today. Butter they have a shorter queue for it also.

But it is very interesting to see that these CPUs’ optimizations for high throughput by adopting near-die MCDRAM. This strategy is very similar to what we see in Apple’s M1 series chips today.

Knight’s Landing: Atom with AVX-512

Related Stories