AMD 3rd Gen EPYC Milan Review

https://www.anandtech.com/print/16529/amd-epyc-milan-review

by pella • 5 years ago

208 103 5 years ago

modzu• 5 years ago

how is a 300W cpu cooled in a server environment? just high rpms and good environmentals? ive stayed on intel with my workstation so i can keep a virtually passive and quiet heatsink without having to go water

Latty• 5 years ago

Yeah, noise is relatively unimportant so they tend to use a ton of extremely high RPM fans and big hunks of copper, from what I've seen.

wffurr3• 5 years ago

And the DC staff wears hearing protection when they're working among the racks.

willis936• 5 years ago

Servers are in 4U rack mounted enclosures with (relatively) low height heatsinks and huge amounts of airflow. Intake in front, exhaust in rear. Most clients will have beefy air conditioning to keep the ambient intake temp and humidity low.

wtallis• 5 years ago

I think 4U servers are quite rare these days, except for models designed to accommodate large numbers of either GPUs or 3.5" hard drives. Most 2-socket servers with up to a few dozen SSDs are 2U designs.

epmaybe2• 5 years ago

Performance per watt is better on amd right now, at least last I heard.

unicornfinder• 5 years ago

Performance per watt is better but unfortunately the idle power consumption is fairly high.

formerly_proven• 5 years ago

Typical servers are 1U or 2U high boxes. 1U=44.5 mm and each box is "full depth", so around 700 mm long (sometimes more), and ~450 mm wide. The 1U boxes typically just have a block of copper with a dense bunch of fins on them as a CPU cooler, while 2U designs usually incorporate heatpipes.

Lots of airflow.

High-density systems are usually 2U, but with four dual-socket systems per chassis.

ben-schaaf• 5 years ago

To put it in perspective, the Intel Xeon 6258R is rated at 205W. Servers have oodles of (likely airconditioned) airflow so this isn't a problem.

folago• 5 years ago

When I worked in HPC in Tromsoe, arctic Norway, one of local supercomputers was liquid cooled and the heat used warmed up some buildings on the university campus.

Out_of_Characte• 5 years ago

a cooling unit provides a delta between surface area temparature and the ambient temparature. Epyc chips are significantly larger and more spread out with the chiplets so the heat density of these chips are relatively similar. So the cooler doesn't have to provide a larger temparature delta and ony has to be slightly larger if at all.

Overall heat density on chips has increased due to litography changes so the chiplet architecture in a way is only a stopgap.

numpad0• 5 years ago

By an array of fans specced like 8cm 10W 10krpm, like six fans wide two fans deep, blowing into a passive heatsink, with help of air ducts inside the chassis

Intel or AMD or SPARC or ARM the setup is all the same for rackmount hardwares, pure copper passive heatsink and high power axial fans

rodgerd2• 5 years ago

Servers are horrifyingly loud, and datacentres will destroy your hearing in no time flat. Airflow management in datacentres is quite the art form, as well: proper packing of the racks for airflow, height of raised floors to accommodate blown air, and so on; some people are starting to go back to the future with things like liquid-cooled racks as well.

greggyb2• 5 years ago

Don't forget that you could feasibly use a 280W 64-core processor to replace a dual-socket 2x205W 2x28-core Intel box with 8 cores to spare. Obviously, there will be performance deltas based on specific use cases, but this should help frame the comparison.

rbanffy5• 5 years ago

Why so little L3 cache on the competition?

ChuckNorris89• 5 years ago

Limitations due to die size and power consumption since Intel Xeon is still on the ye olde 14nm++ process.

Also since Xeon dies are monolithic, unlike AMDs chiplet design, means that increasing the size of certain components on the die, like cache for example, increases the risk of defects wich reduces the yields, making them unprofitable.

rbanffy5• 5 years ago

True, but the ARM ones have just 32MB for up to 80 threads.

I wonder if we could get numbers for L3 misses and cycles spent waiting for main memory under realistic workloads.

greggyb2• 5 years ago

FYI, 'ye olde' is actually using the letter 'y' to represent an Old English 'thorn' character. The thorn character represents the 'th' sound. Printing presses gained prominence in non-English speaking regions earlier than in English speaking regions. The 'y' was substituted for the 'thorn' due to some amount of similarity in shape.

Thus, 'ye olde' would be pronounced 'the old'. 'Ye' is just a different form of 'the' from a time when 'y' represented 'th'.

So, 'the ye olde' would be read as 'the the olde'.

dragontamer13• 5 years ago

EPYC is a split L3 cache. Any particular core only benefits from 32MBs of L3, the 33rd MB is "on another chip". (EDIT: Zen2 was 16MBs, Zen3 is 32MBs. Fixed numbers for Zen3)

As such, AMD can make absolutely huge amounts of L3 cache (well, many parallel L3 clusters), while other CPU designers need to figure out how to combine the L3 so that a single core can benefit it from it all.

xirbeosbwo12342• 5 years ago

That's not quite accurate. Every core has access to the entire L3, including the L3 on an entirely different socket. CPUs communicate through caches, so if a core just plain couldn't talk to another core's cache then cache coherency algorithms wouldn't work. Though a core can access the entire cache, the latency is higher when going off-die. It is really high when going to another socket.

The first generation of Epyc had a complicated hierarchy that made latency quite hard to predict, but the new architecture is simpler. A CPU can talk to a cache in the same package but on a different die with reasonably low latency.

(I don't have numbers. Still reading.)

rbanffy5• 5 years ago

What AMD does is not magic and is not beyond what others can do. My question is why they chose to have just 32MB for up to 80 cores when AMD can choose to have 32MB per 8-core chiplet.

As a comparison, an IBM z15 mainframe CPU has 10 cores and 256MB per socket.

xirbeosbwo12342• 5 years ago

First off, it's not a direct comparison. The Epyc has one L3 cache per chiplet. This means that latency is not uniform across the entire L3 cache. This was a serious concern on the first generation of Epyc, where accessing L3 could take anywhere from zero to three hops across an internal network. AMD has greatly improved the problem on the more recent generations by switching to a star topology with more predictable latency.

That said, there are two major reasons:

1. Epyc is on a chiplet architecture. Large chips are harder to make than small ones. Building two 200mm^2 chips is cheaper than building one 400mm^2 chip. Since Epyc has a chiplet architecture, this means they can put more silicon into a chip for the same price. This means that Epyc can be just plain bigger than the competition. This comes with some complexity and inefficiency but has, so far, paid off in spades for them.

2. Epyc is on a newer process. This means AMD can fit more transistors even with the same area. Intel has had serious problems with their newer processes, so this is not an advantage AMD expected to have when designing the part. The use of a cutting-edge process was, in part, enabled by the chiplet architecture. It is possible to fabricate several small chips on a 7nm process even though one large chip would be prohibitively expensive, and AMD has been able to use a 14nm process in parts of the CPU that wouldn't benefit from a 7nm process to cut costs.

The first point is serious cleverness on the part of AMD. The second point is mostly that Intel dropped the ball.

totalZero• 5 years ago

What is the likelihood that mixed-process chiplets become the state of the art?

rbanffy5• 5 years ago

There must be a typo on the 74F3 price. US$2900 for it is a steal.

tecleandor• 5 years ago

I found it in the original press release (price for !K units, of course)

https://ir.amd.com/news-events/press-releases/detail/993/amd...

masklinn2• 5 years ago

Still seems like a typo, it doesn't make any sense that the 24 / 48 would be priced between the 8 / 16 and the 16 / 32. Either the prices of the 73 and 74 were swapped or the tag is just plain wrong. "2900" is also very suspiciously round compared to every other price on the press release.

wffurr3• 5 years ago

RTFA:

“ Users will notice that the 16-core processor is more expensive ($3521) than the 24 core processor ($2900) here. This was the same in the previous generation, however in that case the 16-core had the higher TDP. For this launch, both the 16-core F and 24-core F have the same TDP, so the only reason I can think of for AMD to have a higher price on the 16-core processor is that it only has 2 cores per chiplet active, rather than three? Perhaps it is easier to bin a processor with an even number of cores active”

coder5433• 5 years ago

I really don't think the article's speculation there is helpful... it's really reaching.

As I said below the article in the comments:

If I were to speculate, I would strongly guess that the actual reason is licensing. AMD knows that more people are going to want the 16 core CPUs in order to fit into certain brackets of software licensing, so AMD charges more for those to maximize profit and availability of the 16 core parts. For those customers, moving to a 24 core processor would probably mean paying significantly more for whatever software they're licensing.

This is the more compelling reason to me, and it matches with server processors that Intel and AMD have charged more for in the past.

"Even vs odd" affecting the difficulty of the binning process just sounds extremely arbitrary... definitely not likely to affect customer prices, given how many other products are in AMD's stack that don't show this same inverse pricing discrepancy.

cm2187• 5 years ago

I am building a single socket server right now, I can't really justify more than twice the price of a 7443P for a marginally higher base clock and twice the cache. Does the cache makes that much of a difference? I thought these are already very large caches vs lots of Intel CPUs.

dragontamer13• 5 years ago

Hmm, with AMD Threadripper, you're already looking at TLB issues at these L3 sizes. So if you actually want to take advantage of lots of L3, you need either many cores, or hugepages.

Case in point: AMD Zen2 has 2048 TLB entries (L2), under a default (in Linux and Windows) of 4kB per TLB entry. That's 8MBs of TLB before your processor starts to page-walk.

Emphasis: Your application will pagewalk when the data still fits in L3 cache.

------------

I'm looking at some of this lineup with 3-cores per CCX (32MBs L3 cache), which means under default 4kB pages, those cores will always require a pagewalk to just read/write its 32MBs L3 cache effectively.

With that being said: 2048 TLB entries for Zen2 processors. Maybe AMD has increased the TLB entries for Zen3. Either way, you probably should start looking at hugepage configuration settings...

These L3 cache sizes are absurd, to the point where its kind of unwieldy. I mean, with enough configuration / programming, you can really make these things fly. But its not exactly plug-and-play.

stillbourne2• 5 years ago

I'm waiting for news on genesis peak, I'd love to get 4th gen threadripper on my next box.

MrFoof• 5 years ago

Computex 2021, June 1st.

* Computex 2017 is when Zen Threadripper was announced.

* Computex 2019 is when Zen 2 Threadripper was announced.

* Computex 2020 is when Zen 2 Threadripper PRO was going to be announced. After the show was delayed, and then officially canceled, AMD did their own announcement shortly after.

So odds are it'll be Computex 2021 on June 1st. This fits in with the usual release timing, as Threadripper tends to follow EPYC by a few months. I'd expect retail channel availability for Threadripper as early as late July, as there's usually a few months between announce and channel availability.

Trust me, I've been waiting to build a 5970X as well. :)

stillbourne2• 5 years ago

Dude, thanks, I've been looking for any scrap of rumor or release date.

whatshisface• 5 years ago

That logarithmic curve fit to the samples from a step function...

bryanlarsen• 5 years ago

Hopefully they can fix their idle power consumption with a firmware tweak or a new stepping, that's a massive regression. It looks like that causes a significant performance degradation too -- more of the power budget going to IO means less to the compute cores.

ece2• 5 years ago

...The core-to-core latency within a quadrant this generation has improved from a worst-case 112ns to 99ns, about a 10ns improvement. Access to remote quadrants has been reduced from up to 142ns to up to 114ns, which is actually a 24% improvement, which is considerable.
What’s really interesting is that inter-socket latencies have also seen very notable reductions. Whereas the Rome part these went up to 272ns, the new Milan part reduces this down to 202ns, again a large 25% improvement between generations.

Good I/O is going to have a cost in terms of die area and power. Granted what Anandtech observed is interesting for 2P systems. Maybe there's a NPS4 vs NPS1 difference at work here, or AMD just isn't doing as much power gating on the IOD as it could in one or either case.

zhdc1• 5 years ago

It looks like Zen 2 processors about about to become even more of a bargain than they already are.

I'll take a 7702P at $2-3K over a 7713P at $5K ten times out of ten.

dragontamer13• 5 years ago

Considering both are made from the 7nm TSMC process, AMD probably aren't going to make any more Zen2 processors at this point.

I think you're right: that buying a generation old or so can offer gross cost savings. But that's only true for the time-period where those chips are available.

IanCutress2• 5 years ago

AMD is going to keep Zen 2 EPYC sales going for a good while yet. Both families will co-exist in the market.

blinkingled• 5 years ago

   INVLPGB New instruction to use instead of inter-core 
   unterrupts to broadcast page invalidates, requires 
   OS/hypervisor support
   
   VAES / VPCLMULQDQ AVX2 Instructions for 
   encryption/decryption acceleration
   
   SEV-ES Limits the interruptions a malicious hypervisor may 
   inject into a VM/instance

   Memory Protection Keys Application control for access- 
   disable and write-disable settings without TLB management
  
   Process Context ID (PCID) Process tags in TLB to reduce 
   flush requirements

Interruptions (Instructions) and Unterrupts (Interrupts) aside (the article obviously was pushed out as fast as AT could lol) - these additions seem like they would help with performance when it comes to mitigations of all the speculation vulnerabilities in an hypervisor env?

fvv2• 5 years ago

Benchmarks https://www.phoronix.com/scan.php?page=article&item=epyc-700...

IanCutress2• 5 years ago

The article linked at the top has pages of benchmarks. Did.... you miss them?

mattashii2• 5 years ago

There are benchmarks, yes, but Anandtechs' bechmarks are mostly synthetic benches from systems almost no-one uses (I don't run SPEC benchmarks daily, and ok, I compile stuff, but not clang, nor using clang)

But Phoronix has synthetic benchmarks from systems quite a few people use (e.g. databases, rendering, simulation). SPEC is nice, but I wouldn't know where to look if I would want to apply that to e.g. postgres. With Phoronix' results, at least there is an idea of how I could apply the results of the benchmark to my usage.

So I find this link quite useful.

That's not to say that I think Anandtechs benchmarks are not valuable (I love reading the nitty gritty details like cache timing specifics), but I can't apply them to anything other than a generic CPU workload, not even slightly in the direction of my specific workload.

ChuckMcM• 5 years ago

Nice bump in specs. Perhaps now they will announce the Zen3 Threadripper :-). As others have mentioned the TR can starve itself on memory accesses when doing a lot of cache invalidation (think pointer chasing through large datasets). If the EPYC improvement of having the chiplet CPUs all share L3 cache moved into the TR space (which one might assume it would[1]) then this could be a reason to upgrade.

[1] I may be wrong here but the TR looks to me like an EPYC chip with the mult-CPU stuff all pulled off. It would be interesting to have a decap with the chiplets identified.

gameswithgo• 5 years ago

yes TR will have the new cache configuration, just like regular ryzen and epyc do.

mcdevilkiller• 5 years ago

In fact, all Zen chips are equal in the cores part (except for clockspeed and other tuning) thanks to the reusability of chiplets. The only thing that differs between Ryzen, TR amd EPYC is the IO die. And the pins (TR and EPYC use the same package, yep).

kllrnohj4• 5 years ago

With the notably exception of ryzen mobile, which doesn't use chiplets.

eqvinox3• 5 years ago

Kinda wish there was a 3rd gen 7282 equivalent, for those of us just building their home servers. 2 sockets with <$750/CPU was kinda the sweet spot there.

(... really need to replace my 2×E5-2630v2, it's getting quite old... but I guess a 2×7282 will do.)

[Ed.: nevermind, hadn't read to the "idle consumption" graphs yet... nope, I'll stick with Rome ...]

wmf4• 5 years ago

Why would two weak sockets be better than a single-socket system?

jedbrown• 5 years ago

16 channels of DDR4-3200, versus 8 in single socket EPYC or 4 for Threadripper.

eqvinox3• 5 years ago

Have you even looked at the offerings? The P-series/single socket CPUs are, at the same specs, only slightly cheaper than the dual socket ones. 2×7282 gives me 32 cores at $1300 list price. 7443P is $1337 for 24 cores. 7502P is even more expensive.

Two 7282s is pretty much the best perf/power/money option as far as I can see.

fomine32• 5 years ago

This is most interesting part of interview posted at the same time.

https://www.anandtech.com/show/16548/interview-with-amd-forr...

IC: While AMD increases the performance on its processor product line, the bandwidth out to DRAM remains constant. Is there an ideal intercept point where higher bandwidth memory makes sense for a customer?
FN: I think you’re absolutely right, and really at the top of the stack, depending on the workload, that can be the performance limiter. If you’re comparing top of the stack parts in certain workloads, you’re not going to see as much of a performance gain from generation to generation, just because you are memory bandwidth limited at the end of the day.
That’s going to continue as we keep increasing the performance of cores, and keep increasing the number of cores. But you should expect us to continue to increase the amount of bandwidth and memory support. DDR5 is coming, which has quite a bit of headroom to DDR4. We see more and more interest in using high bandwidth memory, for an on-package solution. I think you will see SKU’s in the future from a variety of companies incorporating HBM, especially for AI. That will initially be fairly specialized to be to be candid, because HBM is extremely expensive. So for most the standard DDR memory, even DDR5 memory, means that HBM is going to be confined initially to applications that are incredibly memory latency sensitive, and then you know, it’ll be interesting to how it plays out over time.

ece2• 5 years ago

I would love to see desktop systems move to 4 channels. 2 DIMMs/channel can die as far as I'm concerned, lets have 2 channels/DIMM for laptops. Memory channel density needs to increase along with core density (wider and/or more cores).

wmf4• 5 years ago

They won't. DDR5-6400 doubles memory bandwidth and that's what we'll get for the next decade or so.

ksec• 5 years ago

They are still using GF 14nm on their IOD due to Wafer Supply Agreement. We will have to wait and see how they fit their iOD with PCI-Express 5 and DDR5 with Zen 4.

And I cant wait to see Netflix plays around with these in their FreeBSD box.

It is interesting the biggest upgrade from Zen 4 won't actually the Core part but the iOD. As I am expecting Zen 4 to be some tweaks and a Die Shrink to 5nm. TSMC also somehow unexpectedly announce doubling their 5nm capacity with new Fab capacity being built. My guess would be an aggregate demand from Apple and AMD exceed certain threshold to be worth doing.

Const-me• 5 years ago

Core chiplets only send signal to I/O die, the distance is measured in mm and the parameters of these wires is tightly controlled by AMD.

I/O die sends signals to the rest of hardware. Wires are way longer, often with multiple connectors in them. I/O chiplet needs to produce and consume more of these milli-amps of electrical current.

One can produce larger transistors with finer processes e.g. by adding more fins to FinFETs, but since they need larger transistors anyway to handle more current, I’m not sure there’s much value in upgrading process of the I/O chiplet.

freedom422• 5 years ago

A bit offtopic. All AMD, Intel and ARM processors come with binary blobs that can't be disabled or removed which could potentially control your whole computer (Intel Management Engine, AMD Platform Security Processor).

Really powerful desktop and server-grade processors are available with fully open source firmware and hardware (even fully open BMC). It's not that expensive, comes with PCIe 4.0 and other modern features [1].

They are POWER9 processors, not x86_64. But many software work on it, checkout 'talos-workstation' on IRC to chat with the users [can access IRC through Element Matrix app] - someone is running Android Studio on it (yeah so much for privacy but still).

[1] https://www.raptorcs.com/

These systems are currently being used by businesses and developers alike, including Google.

wmf4• 5 years ago

Honestly, Power9 is not really powerful compared to Milan; it's 1/3 to 1/2 the performance at best. https://openbenchmarking.org/vs/Processor/AMD%20EPYC%207763%...

The desktop-grade Power9 gets rekt by a Core i3: https://openbenchmarking.org/vs/Processor/POWER9%204-Core,In...

freedom422• 5 years ago

Edit: Commented something without reading, deleted.

The advantage of POWER9 is fully open firmware and hardware while being really powerful. They won't beat the latest AMD in performance. But AMD can never be open. The RaptorCS products are FSF RYF (respects your freedom) certified.

https://ryf.fsf.org/vendors/raptor

Some benchmarks from 2019: https://www.phoronix.com/scan.php?page=article&item=rome-pow...

AMD 3rd Gen EPYC Milan Review

Related Stories