Nvidia Announces PCIe Tesla V100

http://www.anandtech.com/show/11559/nvidia-formally-announces-pcie-tesla-v100-available-later-this-year?utm_source=twitter&utm_medium=social

by endorphone • 9 years ago

52 41 9 years ago

anandtech.com

polskibus2• 9 years ago

I wonder when will such briefs include information about cryptocurrency mining performance on a regular basis.

fleetwood2• 9 years ago

Probably quite a while, as NVIDIA currently doesn't mention cryptocurrency mining publicly to keep investors from worrying about the sustainability of their recent revenue growth.

polskibus2• 9 years ago

I wonder whether cryptomining sells more gpus than deep learning, outside public cloud providers.

wmf3• 9 years ago

It is rumored that NVIDIA will announce mining-only cards within weeks: http://wccftech.com/nvidia-pascal-gpu-cryptocurrency-mining-...

colordrops• 9 years ago

I was under the impression that ASICs were far better for mining. Have GPUs caught up again?

dantillberg3• 9 years ago

GPUs are definitely dead for bitcoin. You'll get something like 10^9th more efficient SHA256 hashing from an ASIC than a GPU.

But for many other coins, there are two big factors bringing GPUs back into the game for mining:

- Lots of alt-coins competing for popularity; many might have ASIC mining implementations in the future, but there either hasn't been enough time or enough money to be made by doing it yet.

- There's been various research done to reduce the potential gain in efficiency from an ASIC implementation over using commodity hardware. This was done in response to the concentrating effect of ASIC mining operations, which put a greater percentage of global hashpower in a smaller set of hands. The main way ethereum implements this "ASIC resistance" is by exercising memory bandwidth -- an area where GPUs are already quite optimized -- in the hashing algorithm. https://github.com/ethereum/wiki/wiki/Ethash goes into detail.

petra• 9 years ago

What else besides deep learning can the tensor core do well ?

krona• 9 years ago

multiplying and adding matrices are fairly fundamental operations.

Here is a starting point: https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_op...

science404• 9 years ago

Yes but they are limited in size and in precision to FP16, which is not sufficient for most applications, especially scientific computing... What this person is asking I think is: Can you actually think of another concrete application?

thearn4• 9 years ago

CFD and other high-fidelity multiphysics simulations can be efficient when GPU-bound. It all comes down to being able to write a matrix and stream a bunch of vectors to multiply it by. When considering Krylov subspace methods, this extends to covering a large part of the problem space of linear algebra.

I think the Tesla cards in particular are double precision, which is not necessarily the best for deep learning applications.

tanderson923• 9 years ago

High-fidelity multiphysics seems incompatible with FP16, unless you're talking some kind of mixed precision method.

lvspiff• 9 years ago

I remember a time back before the turn of the century me and my friends joking how cool it would be to run doom on a CRAY - who knew that less than 20 years later we could be...

astrodust• 9 years ago

DOOM wouldn't run very well on a CRAY machine for multiple reasons, but one of which is it doesn't have the right video hardware.

A Raspberry Pi has way more compute power than the first generation Cray machine and a 386-vintage machine has better single-core, non-vectorized performance.

modeless5• 9 years ago

Don't get too excited about V100. P100 was announced over 1 year ago and is still today not available from AWS, GCE, or Azure (despite various announcements). Although you can now, finally, order one at retail for the low, low price of $12k+ [1] [2].

The upcoming Volta-based consumer GPUs are going to be your best value for machine learning, not V100.

[1] https://www.amazon.com/NVIDIA-Tesla-P100-computing-processor... [2] http://accessories.us.dell.com/sna/productdetail.aspx?c=us&l...

jamesfmilne• 9 years ago

You can get the Quadro GP100, which is pretty much identical (I've got 2).

http://images.nvidia.com/content/pdf/quadro/data-sheets/3020...

Agreed re consumer GPUs for machine learning though.

modeless5• 9 years ago

Interesting, I hadn't heard of GP100, only released in February. Still, at $6000, as expensive as eight 1080 Tis.

endorphone3• 9 years ago

Many of the early P100s went to supercomputers. The V100 is interesting because they implemented "tensor cores", and claim that it will offer 12 times the deep learning performance of the already class leading P100.

I'm not sure how consumer GPUs that don't exist can be better, though. That seems highly presumptive, especially given that most machine learning setups are heat, cooling and space limited.

modeless5• 9 years ago

most machine learning setups are heat, cooling and space limited.

You forgot "cost". V100, like P100, will cost literally more than 10 times what consumer cards do, for a likely speedup of less than 2 times. For the vast majority of people, that will not be worth it.

sgt101• 9 years ago

Tensorflow 1.2 has integrated new instructions from Intel, does anyone have any figures on 1.2 showing CPU performance vs GPU's?

visionscaper3• 9 years ago

The "NVIDIA Tesla Family Specification Comparison" table indicates 112 TFLOPS "Tensor Performance (Deep Learning)" for Tesla V100 (PCIe).

Is that double precision?

The Nvidia 1080 Ti has a double precision performance of 332 GFLOPS [1]. If the above number is for double precision computing, the Tesla V100 (PCIe) would be about 337 times as fast (!!)

Does anyone have more insight into these numbers?

[1] https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_proces...

UPDATE : I should have read the article more carefully, it seems to be a mix of FP16 (half precision) and FP32 (single precision). That would likely mean a factor ~10 in computation performance (specifically for deep learning)

wmf3• 9 years ago

"...matrix operations with FP16 inputs and FP32 accumulation..." http://images.nvidia.com/content/volta-architecture/pdf/Volt...

visionscaper3• 9 years ago

Thanks, I also read the article better now. I updated my comment.

davesque• 9 years ago

On this topic, hasn't there been some research that has shown that lesser precision values can be used in neural network algorithms without much loss in performance?

visionscaper3• 9 years ago

There had been a lot of research on this topic (just Google it), with good results.

redtuesday• 9 years ago

With the incredible die size of Volta, and the cost associated with it, I wonder if the rumors are true that AMDs next gen GPU codenamed Navi is using the same strategy as their Zen CPUs (Ryzen, Threadripper, Epyc) - multiple smaller dies connected for better yields and lower cost.

Would something like that be even possible with GPUs? I know the Exascale paper from AMD mentions GPU chiplets which sound like this.

Nvidia Announces PCIe Tesla V100

Related Stories