AI: Nvidia Is Taking All the Money

https://seekingalpha.com/article/4609485-ai-nvidia-is-taking-all-the-money

by TradingPlaces • 3 years ago

76 30 3 years ago

I am surprised that AMD is lagging behind so badly. Their GPUs have plenty of ram available yet they are pretty useless for ml.

javchz2• 3 years ago

The main issue it's CUDA. Most ML developers use it, or use libraries that depends on CUDA as an envoirments.

There are brand agnostic alternatives, but they are less popular and usually slower.

TradingPlaces3• 3 years ago

OP here. That’s exactly right. In the article, I describe CUDA and the rest of the software suite as their competitive moat to hardware competition from AMD or AI accelerators.

lmpdev• 3 years ago

I really hope CUDA doesn't end up being the PostScript of ML

LegitShady• 3 years ago

I don’t think its CUDA. If it was CUDA people would use AMDs Radeon Open Compute and save tons of money on cards with more memory.

The issue is that the Radeon platform isn’t well developed, and AMD is charging as much as they can get for their cards just like Nvidia. if they had working competitive compute they’d also charge as much as Nvidia, and make more money on fewer cards sold.

I think there’s just something with card architecture that makes it not as good as Nvidia for this purpose. Just like mining bitcoins or VR, sometimes card architecture makes a difference.

WeMoveOn• 3 years ago

Eh its more so that the entire ecosystem is built on CUDA. I don't think most developers would care or even notice if you swapped out CUDA for something that'll work seamlessly with the existing Python libraries. Sadly such an alternative doesn't exist and probably won't exist.

adam_arthur• 3 years ago

There wasn't a strong incentive to get either ML training or inference working on non-nvidia devices before. You can sure bet there is now though.

To expect that NVDA will be the only player for years to come is quite likely to be proven false. Google already has their foot in this game too via their TPU. The financial incentives have recently gotten an order of magnitude stronger

bob10292• 3 years ago

I have a tiny suspicion that a new technique is going to emerge soon that will cause some frustration for Nvidia's investors. For instance, observing how little quantization error seems to matter makes me believe there are probably some other massive wins lurking throughout. Before I'd invest, I'd consider the "doomsday" scenario wherein someone develops a technique for running an OpenAI, GPT-4-scale model on a MacBook or iPhone.

I know it probably seems impossible to many on HN that there could be another bucket of 3-4 orders of magnitude sitting on the table, but progress over the last ~6 months seems to provide a compelling argument against that point of view.

There are also radically-different architectures that have seen zero serious effort put towards them. Mostly things that are CPU-bound. In my view, GPU is starting to cause more harm than good with regard to development of new concepts for problem solving. There are neural network architectures that simply don't work well across the PCIe bus. Eventually, someone is going to start playing around with these ideas as GPU scarcity rages onwards.

heyitsguay• 3 years ago

Yeah but the challenge lately has been that efficiency gains have also made the capabilities of large compute clusters even more powerful. Transformers were introduced as an efficient alternative to RNNs and conv nets, then it turned out that however much they supercharged a single GPU, they were dramatically more powerful deployed at datacenter scale.

Tuna-Fish• 3 years ago

I'd consider the "doomsday" scenario wherein someone develops a technique for running an OpenAI, GPT-4-scale model on a MacBook or iPhone.

That's not a doomsday scenario for Nvidia. There is essentially infinite demand for better AI, only limited by what can be provided at acceptable cost. If you can run GPT-4 on a macbook, you can do even better with a more massive model. If you can run a good image model on a macbook, then the next frontier is running a good video model, etc.

The real doomsday scenario for Nvidia is that there seems to very little differentiating their hardware. Their lead in the space exists because they developed good software support early, which lead to everyone standardizing on them. But AI is not like graphics where the problem domain is complex and the APIs are very ill-defined, and you can do all these tricks to make it faster and better. Instead, AI is almost entirely doing just a handful of very simple operations. Other vendors should be capable of providing good software support eventually, and at that point, what is Nvidia's moat? What justifies their margin?

awaythrow4832• 3 years ago

There is essentially infinite demand for better AI,

New paradigm stuff. Infinite demand. It will be able to create a perpetual motion machine easily.

hakfoo• 3 years ago

The risk point is always "good enough."

Yes, you can run a better model if you have more hardware, but does a better model translate to a more compelling user experience?

Will people be willing to pay for slightly better AI images of giant cats using the Empire State Building as a scratching post if good-enough cats can be rendered locally.

SnorkelTan• 3 years ago

I think Windows is a pretty good example of why just a set of software libraries can be a pretty formidable competitive edge.

paulddraper• 3 years ago

1. There is not a 3-4 order of magnitude improvement on the table. 1? Maybe.

2. If AI gets more capable/efficient, the demand will increase not decrease.

Did weapons manufactures go out of business when machine guns were invented?

bob10292• 3 years ago

Did weapons manufactures go out of business when machine guns were invented?

Some certainly did.

stevenhuang• 3 years ago

The obvious counter example is that our brains operate many orders of magnitudes more efficient and capable than what currently exists.

That alone proves your first point wrong.

m463• 3 years ago

I thought the whole deal with machine learning was that inference is easy, but training is hard.

So creating the models from the data is what takes the billions of transistors on many cards.

running the models doesn't need so much hardware.

also... with respect to powerful processing - I haven't seen graphics cards tapering off. There's a ravenous demand for better graphics hardware each year. For every technique that is commoditized, the next year there's a new way of doing things that is better. I remember things like lighting, or realistic hair, or physics or whatever making a new graphics card better. Why wouldn't AI stuff be any different?

rahimnathwani• 3 years ago

  inference is easy, but training is hard

That's often true for classification models (like image recognition), but it the generation models (SD, Llama and GPT-4) everyone is excited about use a lot of compute for inference.

ilaksh• 3 years ago

Is there an open software standard that different AI-specific accelerators can all use?

Is there any investment in consumer-focused AI expansion cards or devices?

What about research into memristors and things? You would think if it were feasible at all that now there would be a huge research push to make radical compute-in-memory paradigms that into real products.

andrewstuart• 3 years ago

AMD should clone Cuda.

It may be hard but no reason why it can’t be done.

Except AMD doesn’t seem very good at doing software.

Thought maybe an open source effort could do it.

touisteur• 3 years ago

HIP is their effort to do that. Not sure they're putting enough dev and optimization effort to extract actual max perf from their hardware. So you're putting one (or 16) 300W board in your system with lower efficiency, risk of no support, bad long term support. And for (easier, more efficient) training you need large memory and hug interconnect. Only NVIDIA has this combination. I'll wait until I see XeLink bridged/switched servers actually in prod, benchmarkable by any noob, before I touch Intel GPUs for training. And if you were thinking H100s were unobtainium, try buying Intel Gaudi stuff...

AI: Nvidia Is Taking All the Money

Related Stories