An Interview with Nvidia CEO Jensen Huang About AI’s iPhone Moment

https://stratechery.com/2023/an-interview-with-nvidia-ceo-jensen-huang-about-ais-iphone-moment/

by pps • 3 years ago

197 116 3 years ago

stratechery.com

bottlepalm3• 3 years ago

Man how lucky is Nvidia? crypto winds down just as AI is ramps up.

greatpostman• 3 years ago

I wouldn’t call it luck, just a great product that’s enabling societal change

ralphc• 3 years ago

NVidia got lucky, at least in the long term. They spent years perfecting SIMD to make spaceships blow up and, by coincidence, that's the same technology that enabled coin mining then deep learning.

orangepanda• 3 years ago

Is Nvidia taking advantage of the crypto/AI trends, or enabling them?

rchaud• 3 years ago

they're selling shovels in a gold rush, that's the best business to be in.

danpalmer3• 3 years ago

They have been investing in this for a long time, CUDA is 15 years old. That said, they have plenty of competition now and that's only going to increase.

- Apple have got their own chips now for mobile AI applications for iPhones.

- Google have got their own chips for Android and servers.

- AMD are gaining on Intel in the server space, and have their own GPUs, being able to sell CPUs and GPUs that complement each other may be a good strategy, plus AMD have plenty of their own experience with OpenCL.

imwithstoopid2• 3 years ago

Nvidia is in an excellent position - they have CUDA as you point out, and they are moving that into server room dominance in this application space

Google has TPUs but have these even made a tiny dent in Nvidia's position?

I assume anything Apple is cooking is using Nvidia in the server room already

Intel seems completely absent from this market

AMD seems content to limit its ambitions to punching Intel

its Nvidia's game to lose at this point...I wonder when they start moving in the other direction and realize they have the power to introduce their own client platform (I secretly wish they would try to mainstream a linux laptop running on Nvidia ARM but obviously this is just a fantasy)

if anything, I think Huang may not be ambitious enough!

bottlepalm3• 3 years ago

They've been investing for a long time, but it's only blown up in the past year due to recent breakthroughs. Good timing for Nvidia.

jacooper3• 3 years ago

AMD GPUs are a joke for anything but gaming

noogle2• 3 years ago

How much of a moat is CUDA?

It's indeed ages beyond any of their competitors. However, most ML/DS people interact with CUDA via a higher-level framework. In recent years this community has consolidated around a few (and even only one platform, PyTorch) framework. For some reason AMD had not invested in platform backends, but there is no network effect or a vendor lock-in to hinder a shift from CUDA to ROCm if it is supported equally well.

mastax• 3 years ago

I'm not convinced AMD has a good AI play. (Disclaimer: I hold a long position in AMD).

AI hardware seems like it can be much simpler than GPGPUs, given the successful implementations by many companies including small startups.

AI hardware software seems like it is extremely difficult. Making a simple programmer and ops interface over a massively parallel, distributed, memory bandwidth constrained system that needs to be able to compile and run high performance custom code out of customers' shifting piles of random python packages.

AMD has continuously struggled at (2) and hasn't seemed to recommit to doing it properly. AMD certainly has silicon design expertise, but given (1) I don't think that is enough.

Xilinix is in interesting alternative path for products or improving AMDs software/devex. I'm not sure what to expect from that, yet.

A4ET8a8uTh0• 3 years ago

Yeah... but have you tried using AMD GPUs for any LLMs for example? All the interesting stuff that is publicly released is for Nvidia. I would love to be able to focus on AMD since Nvidia has been adding some anti-user features lately.

pier252• 3 years ago

Google have got their own chips for Android

What chips for Android?

raverbashing• 3 years ago

And Intel is, as usual, doing nothing

jitl2• 3 years ago

They nerfed hash rate on their cards multiple times

selectodude• 3 years ago

And drug manufacturers spend a lot of money to nerf their medications to make them harder to inject. Not all customers are good customers.

PartiallyTyped• 3 years ago

Nvidia is part of the reason why it happened and why it happened now, they didn't get lucky by any stretch of the imagination wrt to AI.

ww5202• 3 years ago

It's not just luck but good strategy. In the past 10 to 15 years, Nvidia has been leveraging its core GPU to go beyond gaming video card to expand into different peripheral areas where massive parallel computing is needed, such as super computing, cloud computing, animation farm, CAD, visualization, simulation, car, VR, AI, and Crypto. They have been able to catch/enable one wave or the other because it's part of their roadmap.

tpmx2• 3 years ago

They have been going pretty much everywhere with their CUDA runtime. LLMs was a random hit.

At the same time, it doesn't seem like a great moat - I think AMD should be to able compete pretty well soon. I think TSMC/Samsung/ASML will capture quite a lot of the profit in the boom to come.

enos_feedler• 3 years ago

I worked at NV on compilers when the CUDA toolchain was brand new. Nobody would have saw this magnitude of parallel applications. Maybe some computer vision stuff. At the time it seemed like diversification from gaming to try and break into HPC, etc.

squokko• 3 years ago

I mean NVIDIA designed the hardware and software for massively parallel computation. That's not really luck.

shubham-rawat5• 3 years ago

NVIDIA's top management may have had a vision but in coming years the competition will be rising exponentially in the AI/ML space as AMD+XILINX, INTEL and some potential dedicated AI hardware startups could become a big threat to dominance of nvidia. nvidia has just a GPU and they fully control it from the architecture to the whole software ecosystem that been built around their GPUs but AMD has complete ground up control over all the components like x86 cpu, gpu, xilinx fpga and they could be big challenger to NV.

seanmcdirmid2• 3 years ago

AI (on GPUs) has been ramping up for more than the last 10 years, crypto was a fly by night thing in comparison.

px432• 3 years ago

GPU mining was only really a thing in the cryptocurrency space for a few years, maybe 2011-2013. Miners moved on to FPGAs for most things, and then ASICs shortly after. There were a few mildly GPU friendly cryptocurrencies, but the vast majority of GPU power used in the past decade has been for deep learning, which really took off in 2008, before Bitcoin was a thing.

px432• 3 years ago

I find it funny that a global trillion dollar economy counts as being "wound down" to some people. The total economic value of the cryptocurrency space at the moment is just between the total economic value of Russia (1.06 trillion) and Switzerland (1.26 trillion). Yes, that's a lot lower than it was, but still a virtual global superpower.

You're not wrong, it's just a fun sign of how far the cryptocurrency space has come.

throwaway1777• 3 years ago

The relevant fact is that GPU mining is dead, not how big crypto is overall.

sagarpatil• 3 years ago

Don’t forget Covid where everyone and their grandmother was gaming. There was a 6 months waiting for RTX 3080. Although now that I think of it , some of it can be attributed to the disruption in supply chain.

ttul• 3 years ago

Never mind TSMC…

tmsh• 3 years ago

Lots of great quotes like:

  "Inference will be the way software is operated in the future. Inference is simply a piece of software that was written by a computer instead of a piece of software that was written by a human and every computer will just run inference someday."

ww5202• 3 years ago

You can argue compilers have been doing that. And JIT was the next step to enable runtime rewriting of software. AI directed JIT or compilers probably will be next.

abudabi1234• 3 years ago

Semiotics Lisp Machine Software 2.0⸻make it happen.

cubefox• 3 years ago

The piece of software that was written by a computer (by an ML algorithm, to be precise) is called a "model". "Inference" is the execution of this software.

rhtgrg• 3 years ago

Isn't this proof that the CEO doesn't understand the tech? Or am I misunderstanding something?

khazhoux• 3 years ago

He's simplifying for the press.

programmer_dude• 3 years ago

LOL.

flangola7• 3 years ago

"It's just an auto complete"

koheripbal• 3 years ago

We're all just auto-completes.

crop_rotation• 3 years ago

Nvidia seems in a really great position. They are to AI what Intel was to the PC (and unlike the PC era there is not a single Microsoft here who controls the entire ecosystem). CUDA still has no alternative. Yes Google has TPUs but outside of Google NVIDIA still dominates and enjoys the network effects (support in all kinds of libs and framework). They face the same problems that Intel faced, as in the market just wants the software and if the software works the same, the hardware is replaceable. It will be interesting to see how they adapt.

TechnicolorByte2• 3 years ago

Interesting comparison with Intel in the PC market:

the market just wants the software and if the software works the same, the hardware is replaceable

And likely to be true given how much competition is heating up in the AI hardware space. Granted, many of these competitors and startups especially have existed for years and haven’t made much of a dent. Even Google’s TPU doesn’t seem that much better than Nvidia’s stuff based on their limited MLPerf score releases. Maybe this “iPhone moment” for AI will change that and force competitors to finally put some real effort in it.

As for Nvidia, looks like they are trying to adapt by selling their own enterprise software solutions such as Omniverse and their AI model customization stuff. Will be interesting to see if they can transform into more of a software solutions provider going forward.

xiphias2• 3 years ago

AMD would have been a natural competitor, but they seem to be slow to catch up.

Also segmenting gaming GPUs from AI GPUs is just not the best idea: devs don't want to buy a gaming GPU and an AI GPU separately when they can just buy a stronger NVIDIA GPU that does both.

Giorgi• 3 years ago

Why is this text though? Is there video?

TechnicolorByte2• 3 years ago

Looks like there’s a paid podcast available. I’m sure there’s some TTS apps that could take in the text and make it consumable that way, though.

visarga• 3 years ago

NaturalReaders is a good one

jaflo• 3 years ago

What’s wrong with text?

rubans• 3 years ago

You can't easily listen to text while doing something else like cooking or laundry. Text to speech is possible, but not amazing

_qua• 3 years ago

This is a lot to read. It says there is a podcast link in the email but I don't see a link. Do you need to be a subscriber?

lsy• 3 years ago

On Nvidia seemingly being in perfect position to ride successive hype cycles of technology: their product category is one of the few types of hardware with significant technical advancement in the last decade, making it only natural that people would seek to leverage that category of hardware with new software efforts. So less that they have been brilliant strategically and more that they have been brilliant technically and that has been creating the market so to speak. What is not clear is whether, as an industry, we are constrained by this technology into seeking solutions that can be massively parallelized.

More simply, when every leap in compute is a massively parallel architecture, every problem seems like it needs to be solved by a massively parallel system. But I'm guessing before this cycle is over we start to see the limitations of that.

davidwu• 3 years ago

Jensen Huang is awesome and in lots of ways underrated compared to a lot of his contemporaries. Per Wikipedia, he founded the company on his 30th birthday in February, 1993. That means he's spent more of his life running Nvidia than not. What a stellar run.

The two Acquired episodes about Nvidia are fantastic and worth listening to:

Nvidia: The GPU Company (1993-2006): https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993...

Nvidia: The Machine Learning Company (2006-2022): https://www.acquired.fm/episodes/nvidia-the-machine-learning...

An Interview with Nvidia CEO Jensen Huang About AI’s iPhone Moment

Related Stories