The Future Google Rackspace Power9 System

For a platform to succees, it should provide low barrier of entry - may be a low capacity P9 system at a low cost ~ 1K ? Will be a better strategy for OF.

I know Power is targeting cloud computing applications, but IBM should consider low cost entry level gear for getting some market share at the lower level which can transition to higher margin markets .

transfire• 10 years ago

I am surprised. I thought 64-bit ARM was the newness headed to the server farms.

DannyBee2• 10 years ago

It will be there eventually. It is definitely not there now, despite what some may have you think :)

StreamBright2• 10 years ago

Do you have some data to back this prediction? What is the biggest advantage over x86 server processors?

wyldfire• 10 years ago

ARM instruction set is pretty mature but the system architecture for servers is less so, IMO. I think there's little commonality for bootstrapping the various SoCs.

bogomipz12• 10 years ago

Is there enough juice in ARM chips to power servers and compete with Intel offerings like Skylake/Haswell?

virtuallynathan• 10 years ago

I wonder if the inclusion of NVLink in Power 8+ will cause Power to excel in ML applications. It could well be quite a bit faster than x86 just due to the memory/interconnect bandwidth.

PeCaN2• 10 years ago

NVLink and CAPI[1] both have huge potential for machine learning. However, a lot of the benefits of NVLink for ML come from GPU-to-GPU NVLink, which doesn't require CPU support.

1. CAPI doesn't seem to get mentioned to much around here, but imagine an FPGA directly accessing some shared system memory. It's neat.

DiabloD35• 10 years ago

It does seem to require support in the PCI-E host controller, which for both modern Intel and those POWER machines, is on die on the CPU.

So, it "requires" CPU support, just not in the way usually meant.

mikehollinger2• 10 years ago

Yeah, it's neat. (I work on stuff that exploits this). We open-sourced the software side of our first flash IO accelerator last year. [1]

You can do some pretty cool things from a HW designer's perspective inside the accelerator, and in the main application. Since the accelerator is cache-coherent, and able to map the same virtual addresses as a given process (and attach to multiple processes' address spaces) the device can do "simple" things like follow pointers, which used to require building a command / data packet, DMA'ing it to the device, and then waiting for a response packet. This, effectively, frees up the main CPU to do other things, rather than wrangle data. It also means that bottlenecks move.

[1] https://github.com/open-power/capiflash

Symmetry3• 10 years ago

For huge datasets the GPU-CPU links might become more important with Pascal now that the GPUs are allowed to trigger page faults.

bogomipz12• 10 years ago

Oh is CAPI an onboard FPGA with a memory controller?

bogomipz12• 10 years ago

Can you elaborate on CAPI/NVlink would be beneficial to an ML workload?

bluedino3• 10 years ago

IBM reps love to throw around the "Google is switching to IBM" line. Can they possibly compete with IBM on price? Why isn't AMD trying to reach this market?

rdtsc2• 10 years ago

AMD would still have an AMD64 architecture though? Or are you thinking they should come up with a new competing architecture.

derefr• 10 years ago

These POWER chips are an open design; thus, anyone who wants Google as a customer and has a chip fab ready to go (like, say, AMD) could, in theory just fab some up to sell to Google.

bryanlarsen• 10 years ago

"Why isn't AMD trying to reach this market?"

https://en.wikipedia.org/wiki/Zen_(microarchitecture)

voltagex_• 10 years ago

What's the current state of Power development in the Linux kernel like? I thought it was only IBM holding the fort (via ozLabs) but this could be a big boost.

cpeterso• 10 years ago

Why does Facebook's Open Rack use a nonstandard rack size? That seems like an obvious barrier for adoption of hardware that was designed to be a commodity.

DiabloD35• 10 years ago

Racks were designed to fit telecom hardware originally. The Open Rack size is designed around common computer hardware sizes.

It isn't a barrier for adoption because swapping racks out of a datacenter is easy, and they fit on standard datacenter floor tiles.

What is a barrier is that damned 48V.

Disclaimer: I run a hosting company.

rdtsc2• 10 years ago

What is a barrier is that damned 48V.

Interesting what is the issue with 48V, I saw equipment for it seemed to be overpriced. Remember pricing out some stuff and as soon as 48V option for power came into play then price rose quite a bit.

Or is it that voltage is not high enough to be efficient for a large data center?

jzwinck• 10 years ago

Swapping racks may not be possible if you don't own the racks. Lots of datacenters are built with their own racks. You might be able to ask for extra deep racks but that's it.

bogomipz12• 10 years ago

Can you explain how and why 48V is a barrier to begin with?

bluedino3• 10 years ago

The Open Rack’s equipment bay has been widened from 19 inches to 21 inches, but it can still accommodate standard 19-inch equipment. A wider 21-inch bay, however, enables some “interesting configurations”, like installing three motherboards or five 3.5-inch disk drives side-by-side in one chassis. The outer width of the rack has remained a standard 24 inches to accommodate standard floor tiles.

ksec6• 10 years ago

They are going up against the coming Xeon E5 Broadwell + FPGA. Power9 do offer more memory per Rack. But I dont see how Intel cant adopt with better memory controller.

To simply put, what are the incentive to switch over to Power9 platform?

dman• 10 years ago

Having viable options to Intel would be very helpful for Google when negotiating bulk rates for the Xeon processors.

thrownaway2424• 10 years ago

Yeah exactly. Google waves these around from time to time to have something over which to negotiate.

petra• 10 years ago

So this raises all sort of questions: Can Intel can be fast enough in integrating Altera(sw+hw+corporate...) ? What is the better FPGA development environment, with more developer share, etc ? FPGA's can be cannibalistic to Intel's business - will they have an incentive problem ? Do some companies(say in china) prefer an open processor, like POWER, and this will create some ecosystem advantage ? Are there any advantageous startups to buy like kandou-bus(faster interconnect) and who will buy them ?

So it's not certain Intel will win.

homero• 10 years ago

Fpga is trash compared to asics

ksec6• 10 years ago

Yes, and I think Intel is not certain to win, just much more likely. The Power9 is here is targeting 2H 2017 release. Which is actually up against Intel Skylake/Kabylake Xeon Purley Platform in similar timeframe.

Purley Platform, Skylake Xeon offers:

Up to 8 Socket and 28 Core per Socket 6 Channel Memory Controller, 12 DIMM per Socket Support of Intel Xpoint NVDIMM 48 PCI-E Gen3, OmniPath 100G Connection

Offers up to 1.5TB Memory on a 2S Server, or 6TB Xpoint. If you push to the limit of 128GB TSV DRAM and 512GB Xpoint, that is potential of 3TB on 2S Server and 12TB of Xpoint.

Not to mention Intel's Network Controller. The Whole ecosystem from Intel Cloud is actually quite amazing. Both from Hardware innovation and Software Compiler they are working on. It is the same lock in as the PC Windows industry, and unless you get a dramatic new way of doing things. You cant simply switch the Mobile Industry to x86 or vice versa all by yourself. Even if you are as big as Google. Then you get 10nm Intel Server in 2018/2019.

Again, I dont see the incentive making the switch.

vegabook• 10 years ago

would be interested to hear more about this IO issue. Am I right to assume that because of the genesis of the X86 architecture in desktop computing, it is not optimized for server-class IO, and that this permeates the design (ie difficult to catch up with a ground-up server architecture)? If that's true then this is a big deal for Power. Certainly my big-data workflows are usually memory-IO bound, not compute bound.

cm3• 10 years ago

If one has the financial option to diversify, one would be wise to use x86, ARM and POWER at the same time. There aren't many examples where monoculture has been beneficial to anyone but the artificially selected culture.

bogomipz12• 10 years ago

Broadwell + FPGA? Is there an Intel design that has both? Can you clarify?

Also what is the issue with the memory controller?

ksec6• 10 years ago

Yes, First design will be out this year. A simple Google Search should bring your lots of info. It will be separate die but on same package. A true integrated single die solution is on track for 2017.

Power8 or Power9 has better memory controller then Intel Xeon. More memory channel, higher bandwidth, and higher memory capacity.

mozumder• 10 years ago

Kinda amazing that they can fit 2 Power9's as well as 2 FHFL PCIEx16 slots, along with 15 drives and 2TB of memory in 1 rack unit.

xiaopingguo• 10 years ago

Wasn't Google at one point all about commodity/consumer level hardware for their servers? Seems a huge turnaround.

sargun• 10 years ago

This is still largely at commodity prices / performance points. It's been quite some time since any of their hardware has looked consumer-oriented, but comparing this to what enterprises buy, it's apples and oranges.

[1] http://shop.oreilly.com/product/0636920041528.do

[2] http://research.google.com/pubs/pub35290.html

nl• 10 years ago

TheNextPlatform is a pretty bad site. They rehash - badly - information available elsewhere, and add a hyperactive spin on it all.

Here's the truth:

Google uses lots of compute power (insightful!)

Google isn't shifting to Power.

Google does have an active R&D program looking at Power.

TheNextPlatform misses the whole point here: That Zaius board has 32 DDR4 slots (commercially available servers from eg Dell max out at 24) and it has 2 NVLINK slots! (!!)

Those NVLINK slots are what Intel should be worried about, because that's where Google is prepared to pay money. They are building computers that lock themselves into NVidia and doing it gladly.

Intel better find a way to compete with NVidia on deep learning.

exhilaration• 10 years ago

For anyone else wondering what NVLINK is, some links to save you a Google search:

https://blogs.nvidia.com/blog/2014/11/14/what-is-nvlink/

http://www.tomshardware.com/news/nvidia-nvlink-boosts-perfor...

I'd be curious to hear what Intel is developing to compete with this.

bogomipz12• 10 years ago

Do you have similar links for CAPI? I feel like that along with NVLINK seem to be present in all the POWER literature.

oneweekwonder• 10 years ago

To add to the above, From Wikipedia[0]:

NVLink is a communications protocol developed by Nvidia. NVLink specifies a point-to-point connection between a CPU and a GPU and also between a GPU and another GPU. NVLink products introduced to date focus on the high-performance application space.

and [1]:

NVLink – a power-efficient high-speed bus between the CPU and GPU, and between multiple GPUs. Allows much higher transfer speeds than those achievable by using PCI Express; estimated to provide between 80 and 200 GB/s.

[0] https://en.wikipedia.org/wiki/NVLink

[1] https://en.wikipedia.org/wiki/GeForce#PASCAL

The Future Google Rackspace Power9 System

Related Stories