Inside Pascal: Nvidia 's Newest Computing Platform

https://devblogs.nvidia.com/parallelforall/inside-pascal/

by jonbaer • 10 years ago

122 45 10 years ago

devblogs.nvidia.com

wyldfire2• 10 years ago

Half precision ftw! ML is the use case they're designing for, but we all get to reap the benefits.

pavlov2• 10 years ago

Hasn't half precision (16-bit float) been in NVidia GPUs forever? I could swear it was available back in the very first shader-capable Geforce FX days already.

wyldfire2• 10 years ago

IIRC what's new here is native support in the ALUs, etc. I think the older support was probably in software.

bsprings2• 10 years ago

(Post author here.) Yes, FP16 has been supported in NVIDIA GPUs as a texture format "forever" -- since before it was incorporated into the IEEE 754 standard. Indeed what is new in GP100 is hardware ALU support (and note denormals are full speed, which is even more important for lower precision formats).

gnuvince• 10 years ago

Please say it's programmable in Pascal.

overcast• 10 years ago

Hush you!

venomsnake• 10 years ago

That will be only for the Turbo models. It was awesome language.

m_mueller2• 10 years ago

I'm not sure whether it was called Turbo Pascal, but we had this Pascal editor with built in 2D draw windows to learn programming in high school. I'm still looking for something equivalent (but maybe a bit more modern/portable) to give to my kid when he's showing some interest. Scratch is nice, but the visual programming becomes limiting very quickly. Is there anything like this today?

gnoway• 10 years ago

Well, Delphi can call functions in external assemblies so...technically?

https://rosettacode.org/wiki/Call_a_function_in_a_shared_lib...

boznz• 10 years ago

+1 Nostalgia. :-)

analognoise• 10 years ago

Calling out to FreePascal and Lazarus here - if they provide a DLL, yup!

I was just looking at controlling NGSpice from FreePascal - one of the examples of running a shared instance of NGSpice is done in a Pascal dialect:

http://ngspice.sourceforge.net/shared.html

I like Pascal much better than C++ and think the portable Lazarus GUI toolkit is pretty damn trick. Check it out: http://www.lazarus-ide.org/

sklogic• 10 years ago

Why not? Pascal to PTX should be trivial (e.g., there is a lot of LLVM Pascal frontends).

marmaduke3• 10 years ago

Is there a description somewhere without all the bla bla hype? Comparison with past architectures would also be welcome, also without hype.

dr_zoidberg2• 10 years ago

What I got from the article:

* ~5.5 TFLOPs on FP64

* "About 2x" performance on FP32 (so 11 TFLOPs)

* "Up to 2x" performance on FP16 (compared to FP32, so about 22 TFLOPs).

* FP16 is also aimed to neural nets training, because when the weights of the net are FP16, it's a more compact representation.

* 3840 general purpose processors.

* More/better texture units, memory units, etc. So it's not about raw power, but also about a better design.

Guess that's about the important stuff. I just skimmed the article over the top, reading a bit here and there, but that seemed to be the most remarkable stuff.

ghshephard2• 10 years ago

How does performance like this compare to the TFLOPs on a supercomputer? I.E. how does one draw parallels to the performance you can get versus what we've been historically able to do with http://www.top500.org/statistics/perfdevel/?

pklausler• 10 years ago

The article has several tables that juxtapose the specs of the previous, current, and new generations, and I think that you will enjoy reading it.

marmaduke3• 10 years ago

I have a fermi board which, despite worse numbers, easily outpaces a kepler and maxwell board, for my workload.

So, yeah, those tables are hype too. I am asking about benchmarks on real workloads.

ansible• 10 years ago

I think the unified memory is one of the big deals here. This makes working on large data sets much easier.

drewm1980• 10 years ago

Comparison to CPU is also important IMHO, and for that you need to be aware that terminology is very different.

What nvidia calls a "core" is more like one entry in a SIMD unit on a cpu. What nvidia calls a "SM" is closer to a CPU core.

There is more to it than that, i.e. gpu cores are more independent than entries in a cpu vector unit, but on the other hand, but gpu "SM"s are less independent than cpu cores.

It's also worth keeping in mind that mediocre cpu code will run circles around mediocre gpu code. To get the gpu magic you have to invest a lot of effort in tuning for the architecture.

JustSomeNobody• 10 years ago

300 Watts.

Toasty.

tomkinstinch• 10 years ago

Wow. It has 15.3 billion transistors. It's amazing we can buy something with that many engineered parts. Even if the transistors are the result of duplication and lithography, it's an astonishing number. Creating the mask must have taken a while.

Does anyone know what the failure rate is for the transistors (or transistors of a similar production process)? Do they all have to function to spec for a GPU, or are malfunctioning transistors disabled or corrected? What does the QC process look like?

djcapelis2• 10 years ago

Exact failure and bin rates for most semiconductor companies are considered deep dark internal trade secret. Other than pure scale, yield rates are one of the biggest factors in semiconductor cost and profit margin.

And the answer is it depends. If you lose some transistors, you expect to lose the entire chip. But the vast majority of the transistors on each chip are part of cache or many many duplicate GPU cores, which if they fail to pass tests, can be disabled or downclocked and then the chip is binned into the appropriate product line.

With GPUs this is much easier than other types of chips, because the level of functional duplication that exists allows a lot of flexibility. If a core is bad, you use a different one, and GPU cores are small enough they'd be stupid not to put some spares on each chip. Same with memories.

Generally one can safely assume:

* Most chips that come off the line are binned into a lower category and do not function at max spec for everything, which is why the price jump is so high at the extreme upper end of a hardware series.

* With ASIC lithography most transistor malfunction isn't correctable, you mostly have to either downclock (some types of faults) or disable (the rest) that piece.

* Rates of transistor malfunction is still incredibly fucking amazingly phenomenally low. Like with 15B transistors on a chip, you have trouble affording a failure rate of even one in a billion.

So your line has to be, as the kids say: on fleek.

Salgat• 10 years ago

Is there an approximation you can give, such as a magnitude? Is it around 1 transistor per million, per billion, per thousand?

ckozlowski• 10 years ago

Spot on.

I used to work for an OEM, and the Intel and AMD engineers would quietly explain to us how this worked on a number of occasions.

The AMD X3 chips I think were the best example of this being done. These were quad-core parts that AMD was manufacturing at the time, but had defect that made one core faulty. So that core was disabled, and sold as triple-core part. http://www.zdnet.com/article/why-amds-triple-core-phenom-is-...

Retric• 10 years ago

An interesting note is processes improve over time, so most companies end up binning processors much more conservatively over time. Price discrimination means company's want to restrict their highest bin even if they could sell it for less. https://en.wikipedia.org/wiki/Price_discrimination

tcas• 10 years ago

I do not have the answers for your questions (and I don't think anyone can share actual failure rates), but I would direct you this video which goes over a lot of modern chip fabrication techniques, circa 2009:

https://www.youtube.com/watch?v=NGFhc8R_uO4

It's crazy stuff.

There are wafer test machines which will interface with the wafer directly and do some testing (which are $$$$), JTAG type tests, which access parts of the chip out of band, and functional testing. Some products, like SD Cards actually have a microcontroller on board that will provide the test routines and error correction without the need of an expensive machine. Design for test is extremely important.

I'm by no means an expert however, I mostly deal with JTAG and functional tests.

cottonseed• 10 years ago

I just read that the Xilinx XCVU440 FPGA has >20B transistors, and that's one generation old (20nm, UltraScale+ is on 16nm finfet). Insane.

mtgx• 10 years ago

IBM's Power9 and its future Power 3.0 ISA CPUs, which should increasingly focus on deep-learning/big data optimization combined with Nvidia's GPUs which will increasingly optimize for the same, should make an interesting match over the next 5+ years.

On the gaming side, I do hope they continue to optimize for VR. I think AMD is even slightly ahead of them on that.

timeu• 10 years ago

Sorry but does it run Crysis ? ;-)

But seriously quite impressive piece of hardware.

Inside Pascal: Nvidia 's Newest Computing Platform

Related Stories